[PATCH mptcp-net v2 0/2] mptcp: pm: in-kernel: usable client side with C-flag

[PATCH mptcp-net v2 0/2] mptcp: pm: in-kernel: usable client side with C-flag

[Matthieu Baerts (NGI0)]

When servers set the C-flag in their MP_CAPABLE to tell clients not to
create subflows to the initial address and port, clients will likely not
use their other endpoints. That's because the in-kernel path-manager
uses the 'subflow' endpoints to create subflows only to the initial
address and port.

The first patch fixes that, and the second one validates it.

More details: https://github.com/multipath-tcp/mptcp_net-next/issues/536

Please see the comment in the first patch, and don't hesitate to reply :)

Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
---
Changes in v2:
- patch 1: move cond to new helper (Geliang) + rename var, squash comm.
- patch 2: move accidental .c modif to the parent commit (Geliang)
- Link to v1: https://lore.kernel.org/r/20250915-pm-c-flag-client-default-v1-0-6bb241e4d991@kernel.org

---
Matthieu Baerts (NGI0) (2):
      mptcp: pm: in-kernel: usable client side with C-flag
      selftests: mptcp: join: validate C-flag + def limit

 net/mptcp/pm.c                                  |  7 +++--
 net/mptcp/pm_kernel.c                           | 37 +++++++++++++++++++++++++
 net/mptcp/protocol.h                            |  7 +++++
 tools/testing/selftests/net/mptcp/mptcp_join.sh | 11 ++++++++
 4 files changed, 60 insertions(+), 2 deletions(-)
---
base-commit: 01d3ba86f43b6fe8e91cfd00ba178d6997095a04
change-id: 20250912-pm-c-flag-client-default-f676d8acfdc7

Best regards,
-- 
Matthieu Baerts (NGI0) <matttbe@kernel.org>

[PATCH mptcp-net v2 1/2] mptcp: pm: in-kernel: usable client side with C-flag

[Matthieu Baerts (NGI0)]

When servers set the C-flag in their MP_CAPABLE to tell clients not to
create subflows to the initial address and port, clients will likely not
use their other endpoints. That's because the in-kernel path-manager
uses the 'subflow' endpoints to create subflows only to the initial
address and port.

If the limits have not been modified to accept ADD_ADDR, the client
doesn't try to establish new subflows. If the limits accept ADD_ADDR,
the routing routes will be used to select the source IP.

The C-flag is typically set when the server is operating behind a legacy
Layer 4 load balancer, or using anycast IP address. Clients having their
different 'subflow' endpoints setup, don't end up creating multiple
subflows as expected, and causing some deployment issues.

A special case is then added here: when servers set the C-flag in the
MPC and directly sends an ADD_ADDR, this single ADD_ADDR is accepted.
The 'subflows' endpoints will then be used with this new remote IP and
port. This exception is only allowed when the ADD_ADDR is sent
immediately after the 3WHS, and makes the client switching to the 'fully
established' mode. After that, 'select_local_address()' will not be able
to find any subflows, because 'id_avail_bitmap' will be filled in
mptcp_pm_create_subflow_or_signal_addr(), when switching to 'fully
established' mode.

Fixes: df377be38725 ("mptcp: add deny_join_id0 in mptcp_options_received")
Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/536
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
---
Notes:
- I tried to find a simple solution that can hopefully be backported to
  cover the 'CDN' use-case.
- I tried to find a solution respecting the 'accepted add addr' limit,
  but I don't see how to... That's why here I limit the exception a
  maximum: when this limit is 0 (default case), only the first ADD_ADDR,
  etc. Feel free to share what you think about that.
- I have some additional code doing some cleanup, and I started to look
  at #503, which is a bit linked to that.
- v2: move conditions to new helper (Geliang) + rename var, squash comm.
---
 net/mptcp/pm.c        |  7 +++++--
 net/mptcp/pm_kernel.c | 37 +++++++++++++++++++++++++++++++++++++
 net/mptcp/protocol.h  |  7 +++++++
 3 files changed, 49 insertions(+), 2 deletions(-)

diff --git a/net/mptcp/pm.c b/net/mptcp/pm.c
index 02dfb379417e2843301f121039ec0d370b040ef7..edaf93fe6f86b32aee8e9bb88318b1dc2a7fcb5e 100644
--- a/net/mptcp/pm.c
+++ b/net/mptcp/pm.c
@@ -637,9 +637,12 @@ void mptcp_pm_add_addr_received(const struct sock *ssk,
 		} else {
 			__MPTCP_INC_STATS(sock_net((struct sock *)msk), MPTCP_MIB_ADDADDRDROP);
 		}
-	/* id0 should not have a different address */
+	/* - id0 should not have a different address
+	 * - special case for C-flag: linked to fill_local_addresses_vec()
+	 */
 	} else if ((addr->id == 0 && !mptcp_pm_is_init_remote_addr(msk, addr)) ||
-		   (addr->id > 0 && !READ_ONCE(pm->accept_addr))) {
+		   (addr->id > 0 && !READ_ONCE(pm->accept_addr) &&
+		    !mptcp_pm_add_addr_c_flag_case(msk))) {
 		mptcp_pm_announce_addr(msk, addr, true);
 		mptcp_pm_add_addr_send_ack(msk);
 	} else if (mptcp_pm_schedule_work(msk, MPTCP_PM_ADD_ADDR_RECEIVED)) {
diff --git a/net/mptcp/pm_kernel.c b/net/mptcp/pm_kernel.c
index f30df8e884a0b3de9fb092e5774cafe08f6350ce..d7cd89fa6a11a1ea7703edbfbdf2bbe86a6a3054 100644
--- a/net/mptcp/pm_kernel.c
+++ b/net/mptcp/pm_kernel.c
@@ -389,10 +389,12 @@ static unsigned int fill_local_addresses_vec(struct mptcp_sock *msk,
 	struct mptcp_addr_info mpc_addr;
 	struct pm_nl_pernet *pernet;
 	unsigned int subflows_max;
+	bool c_flag_case;
 	int i = 0;
 
 	pernet = pm_nl_get_pernet_from_msk(msk);
 	subflows_max = mptcp_pm_get_subflows_max(msk);
+	c_flag_case = remote->id && mptcp_pm_add_addr_c_flag_case(msk);
 
 	mptcp_local_address((struct sock_common *)msk, &mpc_addr);
 
@@ -409,6 +411,10 @@ static unsigned int fill_local_addresses_vec(struct mptcp_sock *msk,
 			locals[i].flags = entry->flags;
 			locals[i].ifindex = entry->ifindex;
 
+			if (c_flag_case)
+				__clear_bit(locals[i].addr.id,
+					    msk->pm.id_avail_bitmap);
+
 			/* Special case for ID0: set the correct ID */
 			if (mptcp_addresses_equal(&locals[i].addr, &mpc_addr, locals[i].addr.port))
 				locals[i].addr.id = 0;
@@ -419,6 +425,37 @@ static unsigned int fill_local_addresses_vec(struct mptcp_sock *msk,
 	}
 	rcu_read_unlock();
 
+	/* Special case: peer sets the C flag, accept one ADD_ADDR if default
+	 * limits are used -- accepting no ADD_ADDR -- and use subflow endpoints
+	 */
+	if (!i && c_flag_case) {
+		unsigned int local_addr_max = mptcp_pm_get_local_addr_max(msk);
+
+		while (msk->pm.local_addr_used < local_addr_max &&
+		       msk->pm.subflows < subflows_max) {
+			struct mptcp_pm_local *local = &locals[i];
+
+			if (!select_local_address(pernet, msk, local))
+				break;
+
+			__clear_bit(local->addr.id, msk->pm.id_avail_bitmap);
+
+			if (!mptcp_pm_addr_families_match(sk, &local->addr,
+							  remote))
+				continue;
+
+			if (mptcp_addresses_equal(&local->addr, &mpc_addr,
+						  local->addr.port))
+				continue;
+
+			msk->pm.local_addr_used++;
+			msk->pm.subflows++;
+			i++;
+		}
+
+		return i;
+	}
+
 	/* If the array is empty, fill in the single
 	 * 'IPADDRANY' local address
 	 */
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index 9b5a248bad40491b678671b53f5f540d396a2a63..dd0662defd41c84474e44c559c571e3594b85d9e 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -1196,6 +1196,13 @@ static inline void mptcp_pm_close_subflow(struct mptcp_sock *msk)
 	spin_unlock_bh(&msk->pm.lock);
 }
 
+static inline bool mptcp_pm_add_addr_c_flag_case(struct mptcp_sock *msk)
+{
+	return READ_ONCE(msk->pm.remote_deny_join_id0) &&
+	       msk->pm.local_addr_used == 0 &&
+	       mptcp_pm_get_add_addr_accept_max(msk) == 0;
+}
+
 void mptcp_sockopt_sync_locked(struct mptcp_sock *msk, struct sock *ssk);
 
 static inline struct mptcp_ext *mptcp_get_ext(const struct sk_buff *skb)

-- 
2.51.0

Re: [PATCH mptcp-net v2 1/2] mptcp: pm: in-kernel: usable client side with C-flag

[Matthieu Baerts]

Hello,

On 16/09/2025 13:01, Matthieu Baerts (NGI0) wrote:
> When servers set the C-flag in their MP_CAPABLE to tell clients not to
> create subflows to the initial address and port, clients will likely not
> use their other endpoints. That's because the in-kernel path-manager
> uses the 'subflow' endpoints to create subflows only to the initial
> address and port.
> 
> If the limits have not been modified to accept ADD_ADDR, the client
> doesn't try to establish new subflows. If the limits accept ADD_ADDR,
> the routing routes will be used to select the source IP.
> 
> The C-flag is typically set when the server is operating behind a legacy
> Layer 4 load balancer, or using anycast IP address. Clients having their
> different 'subflow' endpoints setup, don't end up creating multiple
> subflows as expected, and causing some deployment issues.
> 
> A special case is then added here: when servers set the C-flag in the
> MPC and directly sends an ADD_ADDR, this single ADD_ADDR is accepted.
> The 'subflows' endpoints will then be used with this new remote IP and
> port. This exception is only allowed when the ADD_ADDR is sent
> immediately after the 3WHS, and makes the client switching to the 'fully
> established' mode. After that, 'select_local_address()' will not be able
> to find any subflows, because 'id_avail_bitmap' will be filled in
> mptcp_pm_create_subflow_or_signal_addr(), when switching to 'fully
> established' mode.
> 
> Fixes: df377be38725 ("mptcp: add deny_join_id0 in mptcp_options_received")
> Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/536
> Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
> ---
> Notes:
> - I tried to find a simple solution that can hopefully be backported to
>   cover the 'CDN' use-case.
> - I tried to find a solution respecting the 'accepted add addr' limit,
>   but I don't see how to... That's why here I limit the exception a
>   maximum: when this limit is 0 (default case), only the first ADD_ADDR,
>   etc. Feel free to share what you think about that.
> - I have some additional code doing some cleanup, and I started to look
>   at #503, which is a bit linked to that.
> - v2: move conditions to new helper (Geliang) + rename var, squash comm.

(...)
> diff --git a/net/mptcp/pm_kernel.c b/net/mptcp/pm_kernel.c
> index f30df8e884a0b3de9fb092e5774cafe08f6350ce..d7cd89fa6a11a1ea7703edbfbdf2bbe86a6a3054 100644
> --- a/net/mptcp/pm_kernel.c
> +++ b/net/mptcp/pm_kernel.c

(...)

> @@ -409,6 +411,10 @@ static unsigned int fill_local_addresses_vec(struct mptcp_sock *msk,
>  			locals[i].flags = entry->flags;
>  			locals[i].ifindex = entry->ifindex;
>  
> +			if (c_flag_case)

Small note: here, it should be:

  if (c_flag_case && (entry->flags & MPTCP_PM_ADDR_FLAG_SUBFLOW))

(but this doesn't change the global idea of this patch, I can apply it
when applying the patch)

> +				__clear_bit(locals[i].addr.id,
> +					    msk->pm.id_avail_bitmap);
> +
>  			/* Special case for ID0: set the correct ID */
>  			if (mptcp_addresses_equal(&locals[i].addr, &mpc_addr, locals[i].addr.port))
>  				locals[i].addr.id = 0;

(...)

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.

Re: [PATCH mptcp-net v2 1/2] mptcp: pm: in-kernel: usable client side with C-flag

[Geliang Tang]

Hi Matt,

Thanks for this v2.

On Tue, 2025-09-16 at 13:01 +0200, Matthieu Baerts (NGI0) wrote:
> When servers set the C-flag in their MP_CAPABLE to tell clients not
> to
> create subflows to the initial address and port, clients will likely
> not
> use their other endpoints. That's because the in-kernel path-manager
> uses the 'subflow' endpoints to create subflows only to the initial
> address and port.
> 
> If the limits have not been modified to accept ADD_ADDR, the client
> doesn't try to establish new subflows. If the limits accept ADD_ADDR,
> the routing routes will be used to select the source IP.
> 
> The C-flag is typically set when the server is operating behind a
> legacy
> Layer 4 load balancer, or using anycast IP address. Clients having
> their
> different 'subflow' endpoints setup, don't end up creating multiple
> subflows as expected, and causing some deployment issues.
> 
> A special case is then added here: when servers set the C-flag in the
> MPC and directly sends an ADD_ADDR, this single ADD_ADDR is accepted.
> The 'subflows' endpoints will then be used with this new remote IP
> and
> port. This exception is only allowed when the ADD_ADDR is sent
> immediately after the 3WHS, and makes the client switching to the
> 'fully
> established' mode. After that, 'select_local_address()' will not be
> able
> to find any subflows, because 'id_avail_bitmap' will be filled in
> mptcp_pm_create_subflow_or_signal_addr(), when switching to 'fully
> established' mode.
> 
> Fixes: df377be38725 ("mptcp: add deny_join_id0 in
> mptcp_options_received")
> Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/536
> Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
> ---
> Notes:
> - I tried to find a simple solution that can hopefully be backported
> to
>   cover the 'CDN' use-case.
> - I tried to find a solution respecting the 'accepted add addr'
> limit,
>   but I don't see how to... That's why here I limit the exception a
>   maximum: when this limit is 0 (default case), only the first
> ADD_ADDR,
>   etc. Feel free to share what you think about that.
> - I have some additional code doing some cleanup, and I started to
> look
>   at #503, which is a bit linked to that.
> - v2: move conditions to new helper (Geliang) + rename var, squash
> comm.
> ---
>  net/mptcp/pm.c        |  7 +++++--
>  net/mptcp/pm_kernel.c | 37 +++++++++++++++++++++++++++++++++++++
>  net/mptcp/protocol.h  |  7 +++++++
>  3 files changed, 49 insertions(+), 2 deletions(-)
> 
> diff --git a/net/mptcp/pm.c b/net/mptcp/pm.c
> index
> 02dfb379417e2843301f121039ec0d370b040ef7..edaf93fe6f86b32aee8e9bb8831
> 8b1dc2a7fcb5e 100644
> --- a/net/mptcp/pm.c
> +++ b/net/mptcp/pm.c
> @@ -637,9 +637,12 @@ void mptcp_pm_add_addr_received(const struct
> sock *ssk,
>  		} else {
>  			__MPTCP_INC_STATS(sock_net((struct sock
> *)msk), MPTCP_MIB_ADDADDRDROP);
>  		}
> -	/* id0 should not have a different address */
> +	/* - id0 should not have a different address
> +	 * - special case for C-flag: linked to
> fill_local_addresses_vec()
> +	 */
>  	} else if ((addr->id == 0 &&
> !mptcp_pm_is_init_remote_addr(msk, addr)) ||
> -		   (addr->id > 0 && !READ_ONCE(pm->accept_addr))) {
> +		   (addr->id > 0 && !READ_ONCE(pm->accept_addr) &&
> +		    !mptcp_pm_add_addr_c_flag_case(msk))) {
>  		mptcp_pm_announce_addr(msk, addr, true);
>  		mptcp_pm_add_addr_send_ack(msk);
>  	} else if (mptcp_pm_schedule_work(msk,
> MPTCP_PM_ADD_ADDR_RECEIVED)) {
> diff --git a/net/mptcp/pm_kernel.c b/net/mptcp/pm_kernel.c
> index
> f30df8e884a0b3de9fb092e5774cafe08f6350ce..d7cd89fa6a11a1ea7703edbfbdf
> 2bbe86a6a3054 100644
> --- a/net/mptcp/pm_kernel.c
> +++ b/net/mptcp/pm_kernel.c
> @@ -389,10 +389,12 @@ static unsigned int
> fill_local_addresses_vec(struct mptcp_sock *msk,
>  	struct mptcp_addr_info mpc_addr;
>  	struct pm_nl_pernet *pernet;
>  	unsigned int subflows_max;
> +	bool c_flag_case;
>  	int i = 0;
>  
>  	pernet = pm_nl_get_pernet_from_msk(msk);
>  	subflows_max = mptcp_pm_get_subflows_max(msk);
> +	c_flag_case = remote->id &&
> mptcp_pm_add_addr_c_flag_case(msk);
>  
>  	mptcp_local_address((struct sock_common *)msk, &mpc_addr);
>  
> @@ -409,6 +411,10 @@ static unsigned int
> fill_local_addresses_vec(struct mptcp_sock *msk,
>  			locals[i].flags = entry->flags;
>  			locals[i].ifindex = entry->ifindex;
>  
> +			if (c_flag_case)
> +				__clear_bit(locals[i].addr.id,
> +					    msk-
> >pm.id_avail_bitmap);
> +
>  			/* Special case for ID0: set the correct ID
> */
>  			if (mptcp_addresses_equal(&locals[i].addr,
> &mpc_addr, locals[i].addr.port))
>  				locals[i].addr.id = 0;
> @@ -419,6 +425,37 @@ static unsigned int
> fill_local_addresses_vec(struct mptcp_sock *msk,
>  	}
>  	rcu_read_unlock();
>  
> +	/* Special case: peer sets the C flag, accept one ADD_ADDR
> if default
> +	 * limits are used -- accepting no ADD_ADDR -- and use
> subflow endpoints
> +	 */
> +	if (!i && c_flag_case) {
> +		unsigned int local_addr_max =
> mptcp_pm_get_local_addr_max(msk);
> +
> +		while (msk->pm.local_addr_used < local_addr_max &&
> +		       msk->pm.subflows < subflows_max) {
> +			struct mptcp_pm_local *local = &locals[i];
> +
> +			if (!select_local_address(pernet, msk,
> local))
> +				break;
> +
> +			__clear_bit(local->addr.id, msk-
> >pm.id_avail_bitmap);
> +
> +			if (!mptcp_pm_addr_families_match(sk,
> &local->addr,
> +							  remote))
> +				continue;
> +
> +			if (mptcp_addresses_equal(&local->addr,
> &mpc_addr,
> +						  local->addr.port))
> +				continue;
> +
> +			msk->pm.local_addr_used++;
> +			msk->pm.subflows++;
> +			i++;
> +		}
> +
> +		return i;
> +	}

In the subsequent patches, this was refactored into a separate helper.
I think that change is well-implemented and could be directly squashed
into this patch. We can define this fill_local_addresses_vec_c_flag
here directly within this patch. WDYT?

There's no need to send a v3; the changes can be made in the squash-to
patch.

Thanks,
-Geliang

> +
>  	/* If the array is empty, fill in the single
>  	 * 'IPADDRANY' local address
>  	 */
> diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
> index
> 9b5a248bad40491b678671b53f5f540d396a2a63..dd0662defd41c84474e44c559c5
> 71e3594b85d9e 100644
> --- a/net/mptcp/protocol.h
> +++ b/net/mptcp/protocol.h
> @@ -1196,6 +1196,13 @@ static inline void
> mptcp_pm_close_subflow(struct mptcp_sock *msk)
>  	spin_unlock_bh(&msk->pm.lock);
>  }
>  
> +static inline bool mptcp_pm_add_addr_c_flag_case(struct mptcp_sock
> *msk)
> +{
> +	return READ_ONCE(msk->pm.remote_deny_join_id0) &&
> +	       msk->pm.local_addr_used == 0 &&
> +	       mptcp_pm_get_add_addr_accept_max(msk) == 0;
> +}
> +
>  void mptcp_sockopt_sync_locked(struct mptcp_sock *msk, struct sock
> *ssk);
>  
>  static inline struct mptcp_ext *mptcp_get_ext(const struct sk_buff
> *skb)

Re: [PATCH mptcp-net v2 1/2] mptcp: pm: in-kernel: usable client side with C-flag

[Matthieu Baerts]

Hi Geliang,

On 19/09/2025 05:01, Geliang Tang wrote:
> Hi Matt,
> 
> Thanks for this v2.
> 
> On Tue, 2025-09-16 at 13:01 +0200, Matthieu Baerts (NGI0) wrote:
>> When servers set the C-flag in their MP_CAPABLE to tell clients not
>> to
>> create subflows to the initial address and port, clients will likely
>> not
>> use their other endpoints. That's because the in-kernel path-manager
>> uses the 'subflow' endpoints to create subflows only to the initial
>> address and port.
>>
>> If the limits have not been modified to accept ADD_ADDR, the client
>> doesn't try to establish new subflows. If the limits accept ADD_ADDR,
>> the routing routes will be used to select the source IP.
>>
>> The C-flag is typically set when the server is operating behind a
>> legacy
>> Layer 4 load balancer, or using anycast IP address. Clients having
>> their
>> different 'subflow' endpoints setup, don't end up creating multiple
>> subflows as expected, and causing some deployment issues.
>>
>> A special case is then added here: when servers set the C-flag in the
>> MPC and directly sends an ADD_ADDR, this single ADD_ADDR is accepted.
>> The 'subflows' endpoints will then be used with this new remote IP
>> and
>> port. This exception is only allowed when the ADD_ADDR is sent
>> immediately after the 3WHS, and makes the client switching to the
>> 'fully
>> established' mode. After that, 'select_local_address()' will not be
>> able
>> to find any subflows, because 'id_avail_bitmap' will be filled in
>> mptcp_pm_create_subflow_or_signal_addr(), when switching to 'fully
>> established' mode.
>>
>> Fixes: df377be38725 ("mptcp: add deny_join_id0 in
>> mptcp_options_received")
>> Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/536
>> Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
>> ---
>> Notes:
>> - I tried to find a simple solution that can hopefully be backported
>> to
>>   cover the 'CDN' use-case.
>> - I tried to find a solution respecting the 'accepted add addr'
>> limit,
>>   but I don't see how to... That's why here I limit the exception a
>>   maximum: when this limit is 0 (default case), only the first
>> ADD_ADDR,
>>   etc. Feel free to share what you think about that.
>> - I have some additional code doing some cleanup, and I started to
>> look
>>   at #503, which is a bit linked to that.
>> - v2: move conditions to new helper (Geliang) + rename var, squash
>> comm.
>> ---
>>  net/mptcp/pm.c        |  7 +++++--
>>  net/mptcp/pm_kernel.c | 37 +++++++++++++++++++++++++++++++++++++
>>  net/mptcp/protocol.h  |  7 +++++++
>>  3 files changed, 49 insertions(+), 2 deletions(-)
>>
>> diff --git a/net/mptcp/pm.c b/net/mptcp/pm.c
>> index
>> 02dfb379417e2843301f121039ec0d370b040ef7..edaf93fe6f86b32aee8e9bb8831
>> 8b1dc2a7fcb5e 100644
>> --- a/net/mptcp/pm.c
>> +++ b/net/mptcp/pm.c
>> @@ -637,9 +637,12 @@ void mptcp_pm_add_addr_received(const struct
>> sock *ssk,
>>  		} else {
>>  			__MPTCP_INC_STATS(sock_net((struct sock
>> *)msk), MPTCP_MIB_ADDADDRDROP);
>>  		}
>> -	/* id0 should not have a different address */
>> +	/* - id0 should not have a different address
>> +	 * - special case for C-flag: linked to
>> fill_local_addresses_vec()
>> +	 */
>>  	} else if ((addr->id == 0 &&
>> !mptcp_pm_is_init_remote_addr(msk, addr)) ||
>> -		   (addr->id > 0 && !READ_ONCE(pm->accept_addr))) {
>> +		   (addr->id > 0 && !READ_ONCE(pm->accept_addr) &&
>> +		    !mptcp_pm_add_addr_c_flag_case(msk))) {
>>  		mptcp_pm_announce_addr(msk, addr, true);
>>  		mptcp_pm_add_addr_send_ack(msk);
>>  	} else if (mptcp_pm_schedule_work(msk,
>> MPTCP_PM_ADD_ADDR_RECEIVED)) {
>> diff --git a/net/mptcp/pm_kernel.c b/net/mptcp/pm_kernel.c
>> index
>> f30df8e884a0b3de9fb092e5774cafe08f6350ce..d7cd89fa6a11a1ea7703edbfbdf
>> 2bbe86a6a3054 100644
>> --- a/net/mptcp/pm_kernel.c
>> +++ b/net/mptcp/pm_kernel.c
>> @@ -389,10 +389,12 @@ static unsigned int
>> fill_local_addresses_vec(struct mptcp_sock *msk,
>>  	struct mptcp_addr_info mpc_addr;
>>  	struct pm_nl_pernet *pernet;
>>  	unsigned int subflows_max;
>> +	bool c_flag_case;
>>  	int i = 0;
>>  
>>  	pernet = pm_nl_get_pernet_from_msk(msk);
>>  	subflows_max = mptcp_pm_get_subflows_max(msk);
>> +	c_flag_case = remote->id &&
>> mptcp_pm_add_addr_c_flag_case(msk);
>>  
>>  	mptcp_local_address((struct sock_common *)msk, &mpc_addr);
>>  
>> @@ -409,6 +411,10 @@ static unsigned int
>> fill_local_addresses_vec(struct mptcp_sock *msk,
>>  			locals[i].flags = entry->flags;
>>  			locals[i].ifindex = entry->ifindex;
>>  
>> +			if (c_flag_case)
>> +				__clear_bit(locals[i].addr.id,
>> +					    msk-
>>> pm.id_avail_bitmap);
>> +
>>  			/* Special case for ID0: set the correct ID
>> */
>>  			if (mptcp_addresses_equal(&locals[i].addr,
>> &mpc_addr, locals[i].addr.port))
>>  				locals[i].addr.id = 0;
>> @@ -419,6 +425,37 @@ static unsigned int
>> fill_local_addresses_vec(struct mptcp_sock *msk,
>>  	}
>>  	rcu_read_unlock();
>>  
>> +	/* Special case: peer sets the C flag, accept one ADD_ADDR
>> if default
>> +	 * limits are used -- accepting no ADD_ADDR -- and use
>> subflow endpoints
>> +	 */
>> +	if (!i && c_flag_case) {
>> +		unsigned int local_addr_max =
>> mptcp_pm_get_local_addr_max(msk);
>> +
>> +		while (msk->pm.local_addr_used < local_addr_max &&
>> +		       msk->pm.subflows < subflows_max) {
>> +			struct mptcp_pm_local *local = &locals[i];
>> +
>> +			if (!select_local_address(pernet, msk,
>> local))
>> +				break;
>> +
>> +			__clear_bit(local->addr.id, msk-
>>> pm.id_avail_bitmap);
>> +
>> +			if (!mptcp_pm_addr_families_match(sk,
>> &local->addr,
>> +							  remote))
>> +				continue;
>> +
>> +			if (mptcp_addresses_equal(&local->addr,
>> &mpc_addr,
>> +						  local->addr.port))
>> +				continue;
>> +
>> +			msk->pm.local_addr_used++;
>> +			msk->pm.subflows++;
>> +			i++;
>> +		}
>> +
>> +		return i;
>> +	}
> 
> In the subsequent patches, this was refactored into a separate helper.
> I think that change is well-implemented and could be directly squashed
> into this patch. We can define this fill_local_addresses_vec_c_flag
> here directly within this patch. WDYT?

I think it would be better not to add this new helper in this patch, and
keep it as it is: it will be easier to backport that, and probably
easier to review.

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.

Re: [PATCH mptcp-net v2 1/2] mptcp: pm: in-kernel: usable client side with C-flag

[Geliang Tang]

On Fri, 2025-09-19 at 10:17 +0200, Matthieu Baerts wrote:
> Hi Geliang,
> 
> On 19/09/2025 05:01, Geliang Tang wrote:
> > Hi Matt,
> > 
> > Thanks for this v2.
> > 
> > On Tue, 2025-09-16 at 13:01 +0200, Matthieu Baerts (NGI0) wrote:
> > > When servers set the C-flag in their MP_CAPABLE to tell clients
> > > not
> > > to
> > > create subflows to the initial address and port, clients will
> > > likely
> > > not
> > > use their other endpoints. That's because the in-kernel path-
> > > manager
> > > uses the 'subflow' endpoints to create subflows only to the
> > > initial
> > > address and port.
> > > 
> > > If the limits have not been modified to accept ADD_ADDR, the
> > > client
> > > doesn't try to establish new subflows. If the limits accept
> > > ADD_ADDR,
> > > the routing routes will be used to select the source IP.
> > > 
> > > The C-flag is typically set when the server is operating behind a
> > > legacy
> > > Layer 4 load balancer, or using anycast IP address. Clients
> > > having
> > > their
> > > different 'subflow' endpoints setup, don't end up creating
> > > multiple
> > > subflows as expected, and causing some deployment issues.
> > > 
> > > A special case is then added here: when servers set the C-flag in
> > > the
> > > MPC and directly sends an ADD_ADDR, this single ADD_ADDR is
> > > accepted.
> > > The 'subflows' endpoints will then be used with this new remote
> > > IP
> > > and
> > > port. This exception is only allowed when the ADD_ADDR is sent
> > > immediately after the 3WHS, and makes the client switching to the
> > > 'fully
> > > established' mode. After that, 'select_local_address()' will not
> > > be
> > > able
> > > to find any subflows, because 'id_avail_bitmap' will be filled in
> > > mptcp_pm_create_subflow_or_signal_addr(), when switching to
> > > 'fully
> > > established' mode.
> > > 
> > > Fixes: df377be38725 ("mptcp: add deny_join_id0 in
> > > mptcp_options_received")
> > > Closes:
> > > https://github.com/multipath-tcp/mptcp_net-next/issues/536
> > > Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
> > > ---
> > > Notes:
> > > - I tried to find a simple solution that can hopefully be
> > > backported
> > > to
> > >   cover the 'CDN' use-case.
> > > - I tried to find a solution respecting the 'accepted add addr'
> > > limit,
> > >   but I don't see how to... That's why here I limit the exception
> > > a
> > >   maximum: when this limit is 0 (default case), only the first
> > > ADD_ADDR,
> > >   etc. Feel free to share what you think about that.
> > > - I have some additional code doing some cleanup, and I started
> > > to
> > > look
> > >   at #503, which is a bit linked to that.
> > > - v2: move conditions to new helper (Geliang) + rename var,
> > > squash
> > > comm.
> > > ---
> > >  net/mptcp/pm.c        |  7 +++++--
> > >  net/mptcp/pm_kernel.c | 37 +++++++++++++++++++++++++++++++++++++
> > >  net/mptcp/protocol.h  |  7 +++++++
> > >  3 files changed, 49 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/net/mptcp/pm.c b/net/mptcp/pm.c
> > > index
> > > 02dfb379417e2843301f121039ec0d370b040ef7..edaf93fe6f86b32aee8e9bb
> > > 8831
> > > 8b1dc2a7fcb5e 100644
> > > --- a/net/mptcp/pm.c
> > > +++ b/net/mptcp/pm.c
> > > @@ -637,9 +637,12 @@ void mptcp_pm_add_addr_received(const struct
> > > sock *ssk,
> > >  		} else {
> > >  			__MPTCP_INC_STATS(sock_net((struct sock
> > > *)msk), MPTCP_MIB_ADDADDRDROP);
> > >  		}
> > > -	/* id0 should not have a different address */
> > > +	/* - id0 should not have a different address
> > > +	 * - special case for C-flag: linked to
> > > fill_local_addresses_vec()
> > > +	 */
> > >  	} else if ((addr->id == 0 &&
> > > !mptcp_pm_is_init_remote_addr(msk, addr)) ||
> > > -		   (addr->id > 0 && !READ_ONCE(pm-
> > > >accept_addr))) {
> > > +		   (addr->id > 0 && !READ_ONCE(pm->accept_addr)
> > > &&
> > > +		    !mptcp_pm_add_addr_c_flag_case(msk))) {
> > >  		mptcp_pm_announce_addr(msk, addr, true);
> > >  		mptcp_pm_add_addr_send_ack(msk);
> > >  	} else if (mptcp_pm_schedule_work(msk,
> > > MPTCP_PM_ADD_ADDR_RECEIVED)) {
> > > diff --git a/net/mptcp/pm_kernel.c b/net/mptcp/pm_kernel.c
> > > index
> > > f30df8e884a0b3de9fb092e5774cafe08f6350ce..d7cd89fa6a11a1ea7703edb
> > > fbdf
> > > 2bbe86a6a3054 100644
> > > --- a/net/mptcp/pm_kernel.c
> > > +++ b/net/mptcp/pm_kernel.c
> > > @@ -389,10 +389,12 @@ static unsigned int
> > > fill_local_addresses_vec(struct mptcp_sock *msk,
> > >  	struct mptcp_addr_info mpc_addr;
> > >  	struct pm_nl_pernet *pernet;
> > >  	unsigned int subflows_max;
> > > +	bool c_flag_case;
> > >  	int i = 0;
> > >  
> > >  	pernet = pm_nl_get_pernet_from_msk(msk);
> > >  	subflows_max = mptcp_pm_get_subflows_max(msk);
> > > +	c_flag_case = remote->id &&
> > > mptcp_pm_add_addr_c_flag_case(msk);
> > >  
> > >  	mptcp_local_address((struct sock_common *)msk,
> > > &mpc_addr);
> > >  
> > > @@ -409,6 +411,10 @@ static unsigned int
> > > fill_local_addresses_vec(struct mptcp_sock *msk,
> > >  			locals[i].flags = entry->flags;
> > >  			locals[i].ifindex = entry->ifindex;
> > >  
> > > +			if (c_flag_case)
> > > +				__clear_bit(locals[i].addr.id,
> > > +					    msk-
> > > > pm.id_avail_bitmap);
> > > +
> > >  			/* Special case for ID0: set the correct
> > > ID
> > > */
> > >  			if
> > > (mptcp_addresses_equal(&locals[i].addr,
> > > &mpc_addr, locals[i].addr.port))
> > >  				locals[i].addr.id = 0;
> > > @@ -419,6 +425,37 @@ static unsigned int
> > > fill_local_addresses_vec(struct mptcp_sock *msk,
> > >  	}
> > >  	rcu_read_unlock();
> > >  
> > > +	/* Special case: peer sets the C flag, accept one
> > > ADD_ADDR
> > > if default
> > > +	 * limits are used -- accepting no ADD_ADDR -- and use
> > > subflow endpoints
> > > +	 */
> > > +	if (!i && c_flag_case) {
> > > +		unsigned int local_addr_max =
> > > mptcp_pm_get_local_addr_max(msk);
> > > +
> > > +		while (msk->pm.local_addr_used < local_addr_max
> > > &&
> > > +		       msk->pm.subflows < subflows_max) {
> > > +			struct mptcp_pm_local *local =
> > > &locals[i];
> > > +
> > > +			if (!select_local_address(pernet, msk,
> > > local))
> > > +				break;
> > > +
> > > +			__clear_bit(local->addr.id, msk-
> > > > pm.id_avail_bitmap);
> > > +
> > > +			if (!mptcp_pm_addr_families_match(sk,
> > > &local->addr,
> > > +							 
> > > remote))
> > > +				continue;
> > > +
> > > +			if (mptcp_addresses_equal(&local->addr,
> > > &mpc_addr,
> > > +						  local-
> > > >addr.port))
> > > +				continue;
> > > +
> > > +			msk->pm.local_addr_used++;
> > > +			msk->pm.subflows++;
> > > +			i++;
> > > +		}
> > > +
> > > +		return i;
> > > +	}
> > 
> > In the subsequent patches, this was refactored into a separate
> > helper.
> > I think that change is well-implemented and could be directly
> > squashed
> > into this patch. We can define this fill_local_addresses_vec_c_flag
> > here directly within this patch. WDYT?
> 
> I think it would be better not to add this new helper in this patch,
> and
> keep it as it is: it will be easier to backport that, and probably
> easier to review.

Sure, let's keep it as is.

> 
> Cheers,
> Matt

[PATCH mptcp-net v2 2/2] selftests: mptcp: join: validate C-flag + def limit

[Matthieu Baerts (NGI0)]

The previous commit adds an exception for the C-flag case. The
'mptcp_join.sh' selftest is extended to validate this case.

In this subtest, there is a typical CDN deployment with a client where
MPTCP endpoints have been 'automatically' configured:

- the server set net.mptcp.allow_join_initial_addr_port=0

- the client has multiple 'subflow' endpoints, and the default limits:
  not accepting ADD_ADDRs.

Without the parent patch, the client is not able to establish new
subflows using its 'subflow' endpoints. The parent commit fixes that.

The 'Fixes' tag here below is the same as the one from the previous
commit: this patch here is not fixing anything wrong in the selftests,
but it validates the previous fix for an issue introduced by this commit
ID.

Fixes: df377be38725 ("mptcp: add deny_join_id0 in mptcp_options_received")
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
---
Notes:
- v2: move accidental .c modif to the parent commit (Geliang) + limit+s
---
 tools/testing/selftests/net/mptcp/mptcp_join.sh | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh
index 6055ee5762e13108e5e2924a0e77d58da584d008..a94b3960ad5e009dbead66b6ff2aa01f70aa3e1f 100755
--- a/tools/testing/selftests/net/mptcp/mptcp_join.sh
+++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh
@@ -3306,6 +3306,17 @@ deny_join_id0_tests()
 		run_tests $ns1 $ns2 10.0.1.1
 		chk_join_nr 1 1 1
 	fi
+
+	# default limits, server deny join id 0 + signal
+	if reset_with_allow_join_id0 "default limits, server deny join id 0" 0 1; then
+		pm_nl_set_limits $ns1 0 2
+		pm_nl_set_limits $ns2 0 2
+		pm_nl_add_endpoint $ns1 10.0.2.1 flags signal
+		pm_nl_add_endpoint $ns2 10.0.3.2 flags subflow
+		pm_nl_add_endpoint $ns2 10.0.4.2 flags subflow
+		run_tests $ns1 $ns2 10.0.1.1
+		chk_join_nr 2 2 2
+	fi
 }
 
 fullmesh_tests()

-- 
2.51.0

Re: [PATCH mptcp-net v2 0/2] mptcp: pm: in-kernel: usable client side with C-flag

[MPTCP CI]

Hi Matthieu,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal: Success! ✅
- KVM Validation: debug: Unstable: 1 failed test(s): packetdrill_mp_capable 🔴
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/17764130273

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/cbd9718b7233
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1002880


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)

Re: [PATCH mptcp-net v2 0/2] mptcp: pm: in-kernel: usable client side with C-flag

[Geliang Tang]

Hi Matt,

On Tue, 2025-09-16 at 13:01 +0200, Matthieu Baerts (NGI0) wrote:
> When servers set the C-flag in their MP_CAPABLE to tell clients not
> to
> create subflows to the initial address and port, clients will likely
> not
> use their other endpoints. That's because the in-kernel path-manager
> uses the 'subflow' endpoints to create subflows only to the initial
> address and port.
> 
> The first patch fixes that, and the second one validates it.
> 
> More details:
> https://github.com/multipath-tcp/mptcp_net-next/issues/536
> 
> Please see the comment in the first patch, and don't hesitate to
> reply :)
> 
> Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>

Thanks for this v2.

LGTM!

Reviewed-by: Geliang Tang <geliang@kernel.org>

-Geliang

> ---
> Changes in v2:
> - patch 1: move cond to new helper (Geliang) + rename var, squash
> comm.
> - patch 2: move accidental .c modif to the parent commit (Geliang)
> - Link to v1:
> https://lore.kernel.org/r/20250915-pm-c-flag-client-default-v1-0-6bb241e4d991@kernel.org
> 
> ---
> Matthieu Baerts (NGI0) (2):
>       mptcp: pm: in-kernel: usable client side with C-flag
>       selftests: mptcp: join: validate C-flag + def limit
> 
>  net/mptcp/pm.c                                  |  7 +++--
>  net/mptcp/pm_kernel.c                           | 37
> +++++++++++++++++++++++++
>  net/mptcp/protocol.h                            |  7 +++++
>  tools/testing/selftests/net/mptcp/mptcp_join.sh | 11 ++++++++
>  4 files changed, 60 insertions(+), 2 deletions(-)
> ---
> base-commit: 01d3ba86f43b6fe8e91cfd00ba178d6997095a04
> change-id: 20250912-pm-c-flag-client-default-f676d8acfdc7
> 
> Best regards,

Re: [PATCH mptcp-net v2 0/2] mptcp: pm: in-kernel: usable client side with C-flag

[Matthieu Baerts]

Hi Geliang,

On 19/09/2025 12:15, Geliang Tang wrote:
> Hi Matt,
> 
> On Tue, 2025-09-16 at 13:01 +0200, Matthieu Baerts (NGI0) wrote:
>> When servers set the C-flag in their MP_CAPABLE to tell clients not
>> to
>> create subflows to the initial address and port, clients will likely
>> not
>> use their other endpoints. That's because the in-kernel path-manager
>> uses the 'subflow' endpoints to create subflows only to the initial
>> address and port.
>>
>> The first patch fixes that, and the second one validates it.
>>
>> More details:
>> https://github.com/multipath-tcp/mptcp_net-next/issues/536
>>
>> Please see the comment in the first patch, and don't hesitate to
>> reply :)
>>
>> Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
> 
> Thanks for this v2.
> 
> LGTM!
> 
> Reviewed-by: Geliang Tang <geliang@kernel.org>

Thank you for the review!

Now in our tree. I decided to add them in the section for net-next: we
are close to the merge window, and linked to the comment from patch 1,
it is certainly safer to delay the backport to stable a bit.

New patches for t/upstream:
- 0b324b1e22a0: mptcp: pm: in-kernel: usable client side with C-flag
- b366bcbbad02: "squashed" patch 1/10 in "mptcp: pm: in-kernel: usable
client side with C-flag"
- c3ca688abf65: selftests: mptcp: join: validate C-flag + def limit
- Results: fe9c68017366..8a1a30937c29 (export)

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.

Re: [PATCH mptcp-net v2 0/2] mptcp: pm: in-kernel: usable client side with C-flag

[Matthieu Baerts]

On 19/09/2025 12:47, Matthieu Baerts wrote:
> Hi Geliang,
> 
> On 19/09/2025 12:15, Geliang Tang wrote:
>> Hi Matt,
>>
>> On Tue, 2025-09-16 at 13:01 +0200, Matthieu Baerts (NGI0) wrote:
>>> When servers set the C-flag in their MP_CAPABLE to tell clients not
>>> to
>>> create subflows to the initial address and port, clients will likely
>>> not
>>> use their other endpoints. That's because the in-kernel path-manager
>>> uses the 'subflow' endpoints to create subflows only to the initial
>>> address and port.
>>>
>>> The first patch fixes that, and the second one validates it.
>>>
>>> More details:
>>> https://github.com/multipath-tcp/mptcp_net-next/issues/536
>>>
>>> Please see the comment in the first patch, and don't hesitate to
>>> reply :)
>>>
>>> Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
>>
>> Thanks for this v2.
>>
>> LGTM!
>>
>> Reviewed-by: Geliang Tang <geliang@kernel.org>
> 
> Thank you for the review!
> 
> Now in our tree. I decided to add them in the section for net-next: we
> are close to the merge window, and linked to the comment from patch 1,
> it is certainly safer to delay the backport to stable a bit.
> 
> New patches for t/upstream:
> - 0b324b1e22a0: mptcp: pm: in-kernel: usable client side with C-flag
> - b366bcbbad02: "squashed" patch 1/10 in "mptcp: pm: in-kernel: usable
> client side with C-flag"
> - c3ca688abf65: selftests: mptcp: join: validate C-flag + def limit
> - Results: fe9c68017366..8a1a30937c29 (export)

I forgot the last bit:

- Results: fe9c68017366..169aef388767 (export)

Tests are now in progress:

- export:
https://github.com/multipath-tcp/mptcp_net-next/commit/df63a09d28d2808cbe27f96a1937ad904035d2d6/checks

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.