[PATCH mptcp-next v2 0/5] mptcp_join userspace pm tests

[PATCH mptcp-next v2 0/5] mptcp_join userspace pm tests

[Geliang Tang]

This patch set add add address, remove address, create subflow, and
destroy subflow test cases for userspace pm in mptcp_join script.

v2:
 - add more tests in mptcp_join.sh
 - patch 4, add kill_wait helper as Matt suggested.

Geliang Tang (5):
  mptcp: update MIB_RMSUBFLOW in cmd_sf_destroy
  selftests: mptcp: userspace pm address tests
  selftests: mptcp: userspace pm subflow tests
  selftests: mptcp: avoid Terminated messages in userspace_pm
  selftests: mptcp: update pm_nl_ctl usage header

 net/mptcp/pm_userspace.c                      |  2 +
 .../testing/selftests/net/mptcp/mptcp_join.sh | 84 ++++++++++++++++++-
 tools/testing/selftests/net/mptcp/pm_nl_ctl.c |  2 +-
 .../selftests/net/mptcp/userspace_pm.sh       | 40 +++++----
 4 files changed, 108 insertions(+), 20 deletions(-)

-- 
2.35.3

[PATCH mptcp-next v2 1/5] mptcp: update MIB_RMSUBFLOW in cmd_sf_destroy

[Geliang Tang]

This patch increases MPTCP_MIB_RMSUBFLOW mib counter in userspace pm
destroy subflow function mptcp_nl_cmd_sf_destroy() when removing subflow.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 net/mptcp/pm_userspace.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/net/mptcp/pm_userspace.c b/net/mptcp/pm_userspace.c
index f56378e4f597..ebbab5200290 100644
--- a/net/mptcp/pm_userspace.c
+++ b/net/mptcp/pm_userspace.c
@@ -5,6 +5,7 @@
  */
 
 #include "protocol.h"
+#include "mib.h"
 
 void mptcp_free_local_addr_list(struct mptcp_sock *msk)
 {
@@ -418,6 +419,7 @@ int mptcp_nl_cmd_sf_destroy(struct sk_buff *skb, struct genl_info *info)
 
 		mptcp_subflow_shutdown(sk, ssk, RCV_SHUTDOWN | SEND_SHUTDOWN);
 		mptcp_close_ssk(sk, ssk, subflow);
+		__MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RMSUBFLOW);
 		err = 0;
 	} else {
 		err = -ESRCH;
-- 
2.35.3

Re: [PATCH mptcp-next v2 1/5] mptcp: update MIB_RMSUBFLOW in cmd_sf_destroy

[Matthieu Baerts]

Hi Geliang,

On 11/06/2022 16:54, Geliang Tang wrote:
> This patch increases MPTCP_MIB_RMSUBFLOW mib counter in userspace pm
> destroy subflow function mptcp_nl_cmd_sf_destroy() when removing subflow.
> 
> Signed-off-by: Geliang Tang <geliang.tang@suse.com>
> ---
>  net/mptcp/pm_userspace.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/net/mptcp/pm_userspace.c b/net/mptcp/pm_userspace.c
> index f56378e4f597..ebbab5200290 100644
> --- a/net/mptcp/pm_userspace.c
> +++ b/net/mptcp/pm_userspace.c
> @@ -5,6 +5,7 @@
>   */
>  
>  #include "protocol.h"
> +#include "mib.h"
>  
>  void mptcp_free_local_addr_list(struct mptcp_sock *msk)
>  {
> @@ -418,6 +419,7 @@ int mptcp_nl_cmd_sf_destroy(struct sk_buff *skb, struct genl_info *info)
>  
>  		mptcp_subflow_shutdown(sk, ssk, RCV_SHUTDOWN | SEND_SHUTDOWN);
>  		mptcp_close_ssk(sk, ssk, subflow);
> +		__MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RMSUBFLOW);

It looks like this instruction is causing quite a bit of issues
according to the public CI:

- KVM Validation: normal:
  - Critical: 7 Call Trace(s) ❌:
  - Task: https://cirrus-ci.com/task/5443636765655040
  - Summary:
https://api.cirrus-ci.com/v1/artifact/task/5443636765655040/summary/summary.txt


I guess you should use MPTCP_INC_STATS() instead.


Side note: I don't know if it is a good idea to increment MIB counters
from the PM code (any pm*.c files). I think this should only be done
from "core" functions in protocol.c and subflow.c (+ options.c of
course). Or from new helpers declared in protocol.h. WDYT?

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net

Re: [PATCH mptcp-next v2 1/5] mptcp: update MIB_RMSUBFLOW in cmd_sf_destroy

[Mat Martineau]

[-- Attachment #1: Type: text/plain, Size: 1914 bytes --]

On Mon, 13 Jun 2022, Matthieu Baerts wrote:

> Hi Geliang,
>
> On 11/06/2022 16:54, Geliang Tang wrote:
>> This patch increases MPTCP_MIB_RMSUBFLOW mib counter in userspace pm
>> destroy subflow function mptcp_nl_cmd_sf_destroy() when removing subflow.
>>
>> Signed-off-by: Geliang Tang <geliang.tang@suse.com>
>> ---
>>  net/mptcp/pm_userspace.c | 2 ++
>>  1 file changed, 2 insertions(+)
>>
>> diff --git a/net/mptcp/pm_userspace.c b/net/mptcp/pm_userspace.c
>> index f56378e4f597..ebbab5200290 100644
>> --- a/net/mptcp/pm_userspace.c
>> +++ b/net/mptcp/pm_userspace.c
>> @@ -5,6 +5,7 @@
>>   */
>>
>>  #include "protocol.h"
>> +#include "mib.h"
>>
>>  void mptcp_free_local_addr_list(struct mptcp_sock *msk)
>>  {
>> @@ -418,6 +419,7 @@ int mptcp_nl_cmd_sf_destroy(struct sk_buff *skb, struct genl_info *info)
>>
>>  		mptcp_subflow_shutdown(sk, ssk, RCV_SHUTDOWN | SEND_SHUTDOWN);
>>  		mptcp_close_ssk(sk, ssk, subflow);
>> +		__MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RMSUBFLOW);
>
> It looks like this instruction is causing quite a bit of issues
> according to the public CI:
>
> - KVM Validation: normal:
>  - Critical: 7 Call Trace(s) ❌:
>  - Task: https://cirrus-ci.com/task/5443636765655040
>  - Summary:
> https://api.cirrus-ci.com/v1/artifact/task/5443636765655040/summary/summary.txt
>
>
> I guess you should use MPTCP_INC_STATS() instead.

+1

>
>
> Side note: I don't know if it is a good idea to increment MIB counters
> from the PM code (any pm*.c files). I think this should only be done
> from "core" functions in protocol.c and subflow.c (+ options.c of
> course). Or from new helpers declared in protocol.h. WDYT?
>

Could you elaborate some more on this? There's some existing MIB counter 
code in the pm*.c files that doesn't seem too out of place - is the idea 
that moving calls to the core it would reduce duplicated code in 
pm_netlink.c and pm_userspace.c?


--
Mat Martineau
Intel

Re: [PATCH mptcp-next v2 1/5] mptcp: update MIB_RMSUBFLOW in cmd_sf_destroy

[Matthieu Baerts]

Hi Mat,

On 14/06/2022 01:04, Mat Martineau wrote:
> On Mon, 13 Jun 2022, Matthieu Baerts wrote:
> 
>> Hi Geliang,
>>
>> On 11/06/2022 16:54, Geliang Tang wrote:
>>> This patch increases MPTCP_MIB_RMSUBFLOW mib counter in userspace pm
>>> destroy subflow function mptcp_nl_cmd_sf_destroy() when removing
>>> subflow.
>>>
>>> Signed-off-by: Geliang Tang <geliang.tang@suse.com>
>>> ---
>>>  net/mptcp/pm_userspace.c | 2 ++
>>>  1 file changed, 2 insertions(+)
>>>
>>> diff --git a/net/mptcp/pm_userspace.c b/net/mptcp/pm_userspace.c
>>> index f56378e4f597..ebbab5200290 100644
>>> --- a/net/mptcp/pm_userspace.c
>>> +++ b/net/mptcp/pm_userspace.c
>>> @@ -5,6 +5,7 @@
>>>   */
>>>
>>>  #include "protocol.h"
>>> +#include "mib.h"
>>>
>>>  void mptcp_free_local_addr_list(struct mptcp_sock *msk)
>>>  {
>>> @@ -418,6 +419,7 @@ int mptcp_nl_cmd_sf_destroy(struct sk_buff *skb,
>>> struct genl_info *info)
>>>
>>>          mptcp_subflow_shutdown(sk, ssk, RCV_SHUTDOWN | SEND_SHUTDOWN);
>>>          mptcp_close_ssk(sk, ssk, subflow);
>>> +        __MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RMSUBFLOW);
>>
>> It looks like this instruction is causing quite a bit of issues
>> according to the public CI:
>>
>> - KVM Validation: normal:
>>  - Critical: 7 Call Trace(s) ❌:
>>  - Task: https://cirrus-ci.com/task/5443636765655040
>>  - Summary:
>> https://api.cirrus-ci.com/v1/artifact/task/5443636765655040/summary/summary.txt
>>
>>
>>
>> I guess you should use MPTCP_INC_STATS() instead.
> 
> +1
> 
>>
>>
>> Side note: I don't know if it is a good idea to increment MIB counters
>> from the PM code (any pm*.c files). I think this should only be done
>> from "core" functions in protocol.c and subflow.c (+ options.c of
>> course). Or from new helpers declared in protocol.h. WDYT?
>>
> 
> Could you elaborate some more on this? There's some existing MIB counter
> code in the pm*.c files that doesn't seem too out of place - is the idea
> that moving calls to the core it would reduce duplicated code in
> pm_netlink.c and pm_userspace.c?

Yes indeed, this is linked and probably more code should be refactored.

My point is that if you need to do a few "independent" calls to
functions from the core API to do a certain type of action from
different sides, it seems likely to have this code being out of sync
after a bit of time or in new code. I'm not sure I'm explaining clearly
what I have in mind so here is a "silly" example to explain this simple
thing:

If you want to destroy a subflow you might want to do:

    msk = get_msk(ssk); // you need the msk later, normal to have this

    // these two functions are linked: here we need to use both to have
    // the desired behaviour. We might not want to close the subflow
    // after the shutdown so fine without helpers (except if it is more
    // common to do both)
    shutdown_ssk(msk, ssk)
    close_ssk(msk, ssk);

    // it is easy to forget counters
    inc_stats(msk, REMOVE_ADDRESS); // maybe too specific: add helper
                                    // to avoid dup code in pm*.c and
                                    // inc the counter only in 1 place?
    dec_stats(msk, SSK_COUNTERS);   // this could go in core API func


But OK, now when reading the code in pm_netlink.c, adding a helper would
not really help so it is probably easier to keep this stat being
increased here I suppose.

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net

[PATCH mptcp-next v2 2/5] selftests: mptcp: userspace pm address tests

[Geliang Tang]

This patch adds userspace pm tests support for mptcp_join.sh script. Add
userpace pm add_addr and rm_addr test cases in userspace_tests().

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 .../testing/selftests/net/mptcp/mptcp_join.sh | 45 ++++++++++++++++++-
 1 file changed, 44 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh
index a4406b7a8064..c3cea1d0d245 100755
--- a/tools/testing/selftests/net/mptcp/mptcp_join.sh
+++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh
@@ -654,6 +654,9 @@ do_transfer()
 
 	local port=$((10000 + TEST_COUNT - 1))
 	local cappid
+	local userspace_pm=0
+	local evts_ns1
+	local evts_ns1_pid
 
 	:> "$cout"
 	:> "$sout"
@@ -690,12 +693,24 @@ do_transfer()
 		extra_args="-r ${speed:6}"
 	fi
 
+	if [[ "${addr_nr_ns1}" = "userspace_"* ]]; then
+		userspace_pm=1
+		addr_nr_ns1=${addr_nr_ns1:10}
+	fi
+
 	if [[ "${addr_nr_ns2}" = "fastclose_"* ]]; then
 		# disconnect
 		extra_args="$extra_args -I ${addr_nr_ns2:10}"
 		addr_nr_ns2=0
 	fi
 
+	if [ $userspace_pm -eq 1 ]; then
+		evts_ns1=$(mktemp)
+		:> "$evts_ns1"
+		ip netns exec ${listener_ns} ./pm_nl_ctl events >> "$evts_ns1" 2>&1 &
+		evts_ns1_pid=$!
+	fi
+
 	local local_addr
 	if is_v6 "${connect_addr}"; then
 		local_addr="::"
@@ -748,6 +763,8 @@ do_transfer()
 	if [ $addr_nr_ns1 -gt 0 ]; then
 		local counter=2
 		local add_nr_ns1=${addr_nr_ns1}
+		local id=10
+		local tk
 		while [ $add_nr_ns1 -gt 0 ]; do
 			local addr
 			if is_v6 "${connect_addr}"; then
@@ -755,9 +772,18 @@ do_transfer()
 			else
 				addr="10.0.$counter.1"
 			fi
-			pm_nl_add_endpoint $ns1 $addr flags signal
+			if [ $userspace_pm -eq 0 ]; then
+				pm_nl_add_endpoint $ns1 $addr flags signal
+			else
+				tk=$(sed -n 's/.*\(token:\)\([[:digit:]]*\).*$/\2/p;q' "$evts_ns1")
+				ip netns exec ${listener_ns} ./pm_nl_ctl ann $addr token $tk id $id
+				sleep 1
+				ip netns exec ${listener_ns} ./pm_nl_ctl rem token $tk id $id
+			fi
+
 			counter=$((counter + 1))
 			add_nr_ns1=$((add_nr_ns1 - 1))
+			id=$((id + 1))
 		done
 	elif [ $addr_nr_ns1 -lt 0 ]; then
 		local rm_nr_ns1=$((-addr_nr_ns1))
@@ -890,6 +916,12 @@ do_transfer()
 	    kill $cappid
 	fi
 
+	if [ $userspace_pm -eq 1 ]; then
+		kill $evts_ns1_pid
+		wait $evts_ns1_pid 2>/dev/null
+		rm -rf $evts_ns1
+	fi
+
 	NSTAT_HISTORY=/tmp/${listener_ns}.nstat ip netns exec ${listener_ns} \
 		nstat | grep Tcp > /tmp/${listener_ns}.out
 	NSTAT_HISTORY=/tmp/${connector_ns}.nstat ip netns exec ${connector_ns} \
@@ -2810,6 +2842,17 @@ userspace_tests()
 		chk_join_nr 0 0 0
 		chk_rm_nr 0 0
 	fi
+
+	# userspace pm add & remove address
+	if reset "userspace pm add & remove address"; then
+		set_userspace_pm $ns1
+		pm_nl_set_limits $ns1 2 2
+		pm_nl_set_limits $ns2 2 2
+		run_tests $ns1 $ns2 10.0.1.1 0 userspace_1 0 slow
+		chk_join_nr 1 1 1
+		chk_add_nr 1 1
+		chk_rm_nr 1 1 invert
+	fi
 }
 
 endpoint_tests()
-- 
2.35.3

Re: [PATCH mptcp-next v2 2/5] selftests: mptcp: userspace pm address tests

[Mat Martineau]

On Sat, 11 Jun 2022, Geliang Tang wrote:

> This patch adds userspace pm tests support for mptcp_join.sh script. Add
> userpace pm add_addr and rm_addr test cases in userspace_tests().
>
> Signed-off-by: Geliang Tang <geliang.tang@suse.com>
> ---
> .../testing/selftests/net/mptcp/mptcp_join.sh | 45 ++++++++++++++++++-
> 1 file changed, 44 insertions(+), 1 deletion(-)
>
> diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh
> index a4406b7a8064..c3cea1d0d245 100755
> --- a/tools/testing/selftests/net/mptcp/mptcp_join.sh
> +++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh
> @@ -654,6 +654,9 @@ do_transfer()
>
> 	local port=$((10000 + TEST_COUNT - 1))
> 	local cappid
> +	local userspace_pm=0
> +	local evts_ns1
> +	local evts_ns1_pid
>
> 	:> "$cout"
> 	:> "$sout"
> @@ -690,12 +693,24 @@ do_transfer()
> 		extra_args="-r ${speed:6}"
> 	fi
>
> +	if [[ "${addr_nr_ns1}" = "userspace_"* ]]; then
> +		userspace_pm=1
> +		addr_nr_ns1=${addr_nr_ns1:10}
> +	fi
> +
> 	if [[ "${addr_nr_ns2}" = "fastclose_"* ]]; then
> 		# disconnect
> 		extra_args="$extra_args -I ${addr_nr_ns2:10}"
> 		addr_nr_ns2=0
> 	fi
>
> +	if [ $userspace_pm -eq 1 ]; then
> +		evts_ns1=$(mktemp)
> +		:> "$evts_ns1"
> +		ip netns exec ${listener_ns} ./pm_nl_ctl events >> "$evts_ns1" 2>&1 &
> +		evts_ns1_pid=$!
> +	fi
> +
> 	local local_addr
> 	if is_v6 "${connect_addr}"; then
> 		local_addr="::"
> @@ -748,6 +763,8 @@ do_transfer()
> 	if [ $addr_nr_ns1 -gt 0 ]; then
> 		local counter=2
> 		local add_nr_ns1=${addr_nr_ns1}
> +		local id=10
> +		local tk
> 		while [ $add_nr_ns1 -gt 0 ]; do
> 			local addr
> 			if is_v6 "${connect_addr}"; then
> @@ -755,9 +772,18 @@ do_transfer()
> 			else
> 				addr="10.0.$counter.1"
> 			fi
> -			pm_nl_add_endpoint $ns1 $addr flags signal
> +			if [ $userspace_pm -eq 0 ]; then
> +				pm_nl_add_endpoint $ns1 $addr flags signal
> +			else
> +				tk=$(sed -n 's/.*\(token:\)\([[:digit:]]*\).*$/\2/p;q' "$evts_ns1")
> +				ip netns exec ${listener_ns} ./pm_nl_ctl ann $addr token $tk id $id
> +				sleep 1
> +				ip netns exec ${listener_ns} ./pm_nl_ctl rem token $tk id $id
> +			fi
> +
> 			counter=$((counter + 1))
> 			add_nr_ns1=$((add_nr_ns1 - 1))
> +			id=$((id + 1))
> 		done
> 	elif [ $addr_nr_ns1 -lt 0 ]; then
> 		local rm_nr_ns1=$((-addr_nr_ns1))
> @@ -890,6 +916,12 @@ do_transfer()
> 	    kill $cappid
> 	fi
>
> +	if [ $userspace_pm -eq 1 ]; then
> +		kill $evts_ns1_pid
> +		wait $evts_ns1_pid 2>/dev/null
> +		rm -rf $evts_ns1
> +	fi
> +
> 	NSTAT_HISTORY=/tmp/${listener_ns}.nstat ip netns exec ${listener_ns} \
> 		nstat | grep Tcp > /tmp/${listener_ns}.out
> 	NSTAT_HISTORY=/tmp/${connector_ns}.nstat ip netns exec ${connector_ns} \
> @@ -2810,6 +2842,17 @@ userspace_tests()
> 		chk_join_nr 0 0 0
> 		chk_rm_nr 0 0
> 	fi
> +
> +	# userspace pm add & remove address
> +	if reset "userspace pm add & remove address"; then
> +		set_userspace_pm $ns1
> +		pm_nl_set_limits $ns1 2 2

These limits are ignored with the userspace PM, it should be ok to delete 
this line.

(comment also applies to the next patch in the series)

> +		pm_nl_set_limits $ns2 2 2
> +		run_tests $ns1 $ns2 10.0.1.1 0 userspace_1 0 slow
> +		chk_join_nr 1 1 1
> +		chk_add_nr 1 1
> +		chk_rm_nr 1 1 invert
> +	fi
> }
>
> endpoint_tests()
> -- 
> 2.35.3
>
>
>

--
Mat Martineau
Intel

[PATCH mptcp-next v2 3/5] selftests: mptcp: userspace pm subflow tests

[Geliang Tang]

This patch adds userspace pm subflow tests support for mptcp_join.sh
script. Add userpace pm create subflow and destroy test cases in
userspace_tests().

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 .../testing/selftests/net/mptcp/mptcp_join.sh | 41 ++++++++++++++++++-
 1 file changed, 39 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh
index c3cea1d0d245..6bbe6e8ef51a 100755
--- a/tools/testing/selftests/net/mptcp/mptcp_join.sh
+++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh
@@ -657,6 +657,8 @@ do_transfer()
 	local userspace_pm=0
 	local evts_ns1
 	local evts_ns1_pid
+	local evts_ns2
+	local evts_ns2_pid
 
 	:> "$cout"
 	:> "$sout"
@@ -702,13 +704,20 @@ do_transfer()
 		# disconnect
 		extra_args="$extra_args -I ${addr_nr_ns2:10}"
 		addr_nr_ns2=0
+	elif [[ "${addr_nr_ns2}" = "userspace_"* ]]; then
+		userspace_pm=1
+		addr_nr_ns2=${addr_nr_ns2:10}
 	fi
 
 	if [ $userspace_pm -eq 1 ]; then
 		evts_ns1=$(mktemp)
+		evts_ns2=$(mktemp)
 		:> "$evts_ns1"
+		:> "$evts_ns2"
 		ip netns exec ${listener_ns} ./pm_nl_ctl events >> "$evts_ns1" 2>&1 &
 		evts_ns1_pid=$!
+		ip netns exec ${connector_ns} ./pm_nl_ctl events >> "$evts_ns2" 2>&1 &
+		evts_ns2_pid=$!
 	fi
 
 	local local_addr
@@ -830,6 +839,8 @@ do_transfer()
 	if [ $addr_nr_ns2 -gt 0 ]; then
 		local add_nr_ns2=${addr_nr_ns2}
 		local counter=3
+		local id=20
+		local tk da dp sp
 		while [ $add_nr_ns2 -gt 0 ]; do
 			local addr
 			if is_v6 "${connect_addr}"; then
@@ -837,9 +848,23 @@ do_transfer()
 			else
 				addr="10.0.$counter.2"
 			fi
-			pm_nl_add_endpoint $ns2 $addr flags $flags
+			if [ $userspace_pm -eq 0 ]; then
+				pm_nl_add_endpoint $ns2 $addr flags $flags
+			else
+				tk=$(sed -n 's/.*\(token:\)\([[:digit:]]*\).*$/\2/p;q' "$evts_ns2")
+				da=$(sed -n 's/.*\(daddr4:\)\([0-9.]*\).*$/\2/p;q' "$evts_ns2")
+				dp=$(sed -n 's/.*\(dport:\)\([[:digit:]]*\).*$/\2/p;q' "$evts_ns2")
+				ip netns exec ${connector_ns} ./pm_nl_ctl csf lip $addr lid $id \
+									rip $da rport $dp token $tk
+				sleep 1
+				sp=$(grep "type:10" "$evts_ns2" |
+				     sed -n 's/.*\(sport:\)\([[:digit:]]*\).*$/\2/p;q')
+				ip netns exec ${connector_ns} ./pm_nl_ctl dsf lip $addr lport $sp \
+									rip $da rport $dp token $tk
+			fi
 			counter=$((counter + 1))
 			add_nr_ns2=$((add_nr_ns2 - 1))
+			id=$((id + 1))
 		done
 	elif [ $addr_nr_ns2 -lt 0 ]; then
 		local rm_nr_ns2=$((-addr_nr_ns2))
@@ -919,7 +944,9 @@ do_transfer()
 	if [ $userspace_pm -eq 1 ]; then
 		kill $evts_ns1_pid
 		wait $evts_ns1_pid 2>/dev/null
-		rm -rf $evts_ns1
+		kill $evts_ns2_pid
+		wait $evts_ns2_pid 2>/dev/null
+		rm -rf $evts_ns1 $evts_ns2
 	fi
 
 	NSTAT_HISTORY=/tmp/${listener_ns}.nstat ip netns exec ${listener_ns} \
@@ -2853,6 +2880,16 @@ userspace_tests()
 		chk_add_nr 1 1
 		chk_rm_nr 1 1 invert
 	fi
+
+	# userspace pm create destroy subflow
+	if reset "userspace pm create destroy subflow"; then
+		set_userspace_pm $ns2
+		pm_nl_set_limits $ns1 2 2
+		pm_nl_set_limits $ns2 2 2
+		run_tests $ns1 $ns2 10.0.1.1 0 0 userspace_1 slow
+		chk_join_nr 1 1 1
+		chk_rm_nr 0 1
+	fi
 }
 
 endpoint_tests()
-- 
2.35.3

[PATCH mptcp-next v2 4/5] selftests: mptcp: avoid Terminated messages in userspace_pm

[Geliang Tang]

There're some 'Terminated' messages in the output of userspace pm tests
script after killing './pm_nl_ctl events' processes:

Created network namespaces ns1, ns2         			[OK]
./userspace_pm.sh: line 166: 13735 Terminated              ip netns exec "$ns2" ./pm_nl_ctl events >> "$client_evts" 2>&1
./userspace_pm.sh: line 172: 13737 Terminated              ip netns exec "$ns1" ./pm_nl_ctl events >> "$server_evts" 2>&1
Established IPv4 MPTCP Connection ns2 => ns1    		[OK]
./userspace_pm.sh: line 166: 13753 Terminated              ip netns exec "$ns2" ./pm_nl_ctl events >> "$client_evts" 2>&1
./userspace_pm.sh: line 172: 13755 Terminated              ip netns exec "$ns1" ./pm_nl_ctl events >> "$server_evts" 2>&1
Established IPv6 MPTCP Connection ns2 => ns1    		[OK]
ADD_ADDR 10.0.2.2 (ns2) => ns1, invalid token    		[OK]

This patch adds a helper kill_wait(), in it using 'wait $pid 2>/dev/null'
commands after 'kill $pid' to avoid printing out these Terminated messages.
Use this helper instead of using 'kill $pid'.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 .../selftests/net/mptcp/userspace_pm.sh       | 40 +++++++++++--------
 1 file changed, 23 insertions(+), 17 deletions(-)

diff --git a/tools/testing/selftests/net/mptcp/userspace_pm.sh b/tools/testing/selftests/net/mptcp/userspace_pm.sh
index 78d0bb640b11..d586bc5ffe01 100755
--- a/tools/testing/selftests/net/mptcp/userspace_pm.sh
+++ b/tools/testing/selftests/net/mptcp/userspace_pm.sh
@@ -37,6 +37,12 @@ rndh=$(stdbuf -o0 -e0 printf %x "$sec")-$(mktemp -u XXXXXX)
 ns1="ns1-$rndh"
 ns2="ns2-$rndh"
 
+kill_wait()
+{
+	kill $1 > /dev/null 2>&1
+	wait $1 2>/dev/null
+}
+
 cleanup()
 {
 	echo "cleanup"
@@ -48,16 +54,16 @@ cleanup()
 		kill -SIGUSR1 $client4_pid > /dev/null 2>&1
 	fi
 	if [ $server4_pid -ne 0 ]; then
-		kill $server4_pid > /dev/null 2>&1
+		kill_wait $server4_pid
 	fi
 	if [ $client6_pid -ne 0 ]; then
 		kill -SIGUSR1 $client6_pid > /dev/null 2>&1
 	fi
 	if [ $server6_pid -ne 0 ]; then
-		kill $server6_pid > /dev/null 2>&1
+		kill_wait $server6_pid
 	fi
 	if [ $evts_pid -ne 0 ]; then
-		kill $evts_pid > /dev/null 2>&1
+		kill_wait $evts_pid
 	fi
 	local netns
 	for netns in "$ns1" "$ns2" ;do
@@ -153,7 +159,7 @@ make_connection()
 	sleep 1
 
 	# Capture client/server attributes from MPTCP connection netlink events
-	kill $client_evts_pid
+	kill_wait $client_evts_pid
 
 	local client_token
 	local client_port
@@ -165,7 +171,7 @@ make_connection()
 	client_port=$(sed --unbuffered -n 's/.*\(sport:\)\([[:digit:]]*\).*$/\2/p;q' "$client_evts")
 	client_serverside=$(sed --unbuffered -n 's/.*\(server_side:\)\([[:digit:]]*\).*$/\2/p;q'\
 				      "$client_evts")
-	kill $server_evts_pid
+	kill_wait $server_evts_pid
 	server_token=$(sed --unbuffered -n 's/.*\(token:\)\([[:digit:]]*\).*$/\2/p;q' "$server_evts")
 	server_serverside=$(sed --unbuffered -n 's/.*\(server_side:\)\([[:digit:]]*\).*$/\2/p;q'\
 				      "$server_evts")
@@ -286,7 +292,7 @@ test_announce()
 	verify_announce_event "$evts" "$ANNOUNCED" "$server4_token" "10.0.2.2"\
 			      "$client_addr_id" "$new4_port"
 
-	kill $evts_pid
+	kill_wait $evts_pid
 
 	# Capture events on the network namespace running the client
 	:>"$evts"
@@ -321,7 +327,7 @@ test_announce()
 	verify_announce_event "$evts" "$ANNOUNCED" "$client4_token" "10.0.2.1"\
 			      "$server_addr_id" "$new4_port"
 
-	kill $evts_pid
+	kill_wait $evts_pid
 	rm -f "$evts"
 }
 
@@ -416,7 +422,7 @@ test_remove()
 	sleep 0.5
 	verify_remove_event "$evts" "$REMOVED" "$server6_token" "$client_addr_id"
 
-	kill $evts_pid
+	kill_wait $evts_pid
 
 	# Capture events on the network namespace running the client
 	:>"$evts"
@@ -449,7 +455,7 @@ test_remove()
 	sleep 0.5
 	verify_remove_event "$evts" "$REMOVED" "$client6_token" "$server_addr_id"
 
-	kill $evts_pid
+	kill_wait $evts_pid
 	rm -f "$evts"
 }
 
@@ -553,7 +559,7 @@ test_subflows()
 			      "10.0.2.2" "$client4_port" "23" "$client_addr_id" "ns1" "ns2"
 
 	# Delete the listener from the client ns, if one was created
-	kill $listener_pid > /dev/null 2>&1
+	kill_wait $listener_pid
 
 	local sport
 	sport=$(sed --unbuffered -n 's/.*\(sport:\)\([[:digit:]]*\).*$/\2/p;q' "$evts")
@@ -592,7 +598,7 @@ test_subflows()
 			      "$client_addr_id" "ns1" "ns2"
 
 	# Delete the listener from the client ns, if one was created
-	kill $listener_pid > /dev/null 2>&1
+	kill_wait $listener_pid
 
 	sport=$(sed --unbuffered -n 's/.*\(sport:\)\([[:digit:]]*\).*$/\2/p;q' "$evts")
 
@@ -631,7 +637,7 @@ test_subflows()
 			      "$client_addr_id" "ns1" "ns2"
 
 	# Delete the listener from the client ns, if one was created
-	kill $listener_pid > /dev/null 2>&1
+	kill_wait $listener_pid
 
 	sport=$(sed --unbuffered -n 's/.*\(sport:\)\([[:digit:]]*\).*$/\2/p;q' "$evts")
 
@@ -647,7 +653,7 @@ test_subflows()
 	ip netns exec "$ns2" ./pm_nl_ctl rem id $client_addr_id token\
 	   "$client4_token" > /dev/null 2>&1
 
-	kill $evts_pid
+	kill_wait $evts_pid
 
 	# Capture events on the network namespace running the client
 	:>"$evts"
@@ -674,7 +680,7 @@ test_subflows()
 			      "10.0.2.1" "$app4_port" "23" "$server_addr_id" "ns2" "ns1"
 
 	# Delete the listener from the server ns, if one was created
-	kill $listener_pid> /dev/null 2>&1
+	kill_wait $listener_pid
 
 	sport=$(sed --unbuffered -n 's/.*\(sport:\)\([[:digit:]]*\).*$/\2/p;q' "$evts")
 
@@ -713,7 +719,7 @@ test_subflows()
 			      "$server_addr_id" "ns2" "ns1"
 
 	# Delete the listener from the server ns, if one was created
-	kill $listener_pid > /dev/null 2>&1
+	kill_wait $listener_pid
 
 	sport=$(sed --unbuffered -n 's/.*\(sport:\)\([[:digit:]]*\).*$/\2/p;q' "$evts")
 
@@ -750,7 +756,7 @@ test_subflows()
 			      "10.0.2.2" "10.0.2.1" "$new4_port" "23" "$server_addr_id" "ns2" "ns1"
 
 	# Delete the listener from the server ns, if one was created
-	kill $listener_pid > /dev/null 2>&1
+	kill_wait $listener_pid
 
 	sport=$(sed --unbuffered -n 's/.*\(sport:\)\([[:digit:]]*\).*$/\2/p;q' "$evts")
 
@@ -766,7 +772,7 @@ test_subflows()
 	ip netns exec "$ns1" ./pm_nl_ctl rem id $server_addr_id token\
 	   "$server4_token" > /dev/null 2>&1
 
-	kill $evts_pid
+	kill_wait $evts_pid
 	rm -f "$evts"
 }
 
-- 
2.35.3

[PATCH mptcp-next v2 5/5] selftests: mptcp: update pm_nl_ctl usage header

[Geliang Tang]

The usage header of pm_nl_ctl command doesn't match with the context. So
this patch adds the missing userspace PM keywords 'ann', 'rem', 'csf',
'dsf', 'events' and 'listen' in it.

Reviewed-by: Mat Martineau <mathew.j.martineau@linux.intel.com>
Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 tools/testing/selftests/net/mptcp/pm_nl_ctl.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/net/mptcp/pm_nl_ctl.c b/tools/testing/selftests/net/mptcp/pm_nl_ctl.c
index 6a2f4b981e1d..4dd87bb9ee91 100644
--- a/tools/testing/selftests/net/mptcp/pm_nl_ctl.c
+++ b/tools/testing/selftests/net/mptcp/pm_nl_ctl.c
@@ -31,7 +31,7 @@
 
 static void syntax(char *argv[])
 {
-	fprintf(stderr, "%s add|get|set|del|flush|dump|accept [<args>]\n", argv[0]);
+	fprintf(stderr, "%s add|ann|rem|csf|dsf|get|set|del|flush|dump|events|listen|accept [<args>]\n", argv[0]);
 	fprintf(stderr, "\tadd [flags signal|subflow|backup|fullmesh] [id <nr>] [dev <name>] <ip>\n");
 	fprintf(stderr, "\tann <local-ip> id <local-id> token <token> [port <local-port>] [dev <name>]\n");
 	fprintf(stderr, "\trem id <local-id> token <token>\n");
-- 
2.35.3

Re: selftests: mptcp: update pm_nl_ctl usage header: Tests Results

[MPTCP CI]

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal:
  - Critical: 7 Call Trace(s) ❌:
  - Task: https://cirrus-ci.com/task/5443636765655040
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5443636765655040/summary/summary.txt

- {"code":404,"message":
  - "Can't find artifacts containing file conclusion.txt"}:
  - Task: https://cirrus-ci.com/task/6569536672497664
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/6569536672497664/summary/summary.txt

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/663169e99df0


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

[PATCH mptcp-next v3 5/5] selftests: mptcp: update pm_nl_ctl usage header

[Geliang Tang]

The usage header of pm_nl_ctl command doesn't match with the context. So
this patch adds the missing userspace PM keywords 'ann', 'rem', 'csf',
'dsf', 'events' and 'listen' in it.

Reviewed-by: Mat Martineau <mathew.j.martineau@linux.intel.com>
Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 tools/testing/selftests/net/mptcp/pm_nl_ctl.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/net/mptcp/pm_nl_ctl.c b/tools/testing/selftests/net/mptcp/pm_nl_ctl.c
index 6a2f4b981e1d..4dd87bb9ee91 100644
--- a/tools/testing/selftests/net/mptcp/pm_nl_ctl.c
+++ b/tools/testing/selftests/net/mptcp/pm_nl_ctl.c
@@ -31,7 +31,7 @@
 
 static void syntax(char *argv[])
 {
-	fprintf(stderr, "%s add|get|set|del|flush|dump|accept [<args>]\n", argv[0]);
+	fprintf(stderr, "%s add|ann|rem|csf|dsf|get|set|del|flush|dump|events|listen|accept [<args>]\n", argv[0]);
 	fprintf(stderr, "\tadd [flags signal|subflow|backup|fullmesh] [id <nr>] [dev <name>] <ip>\n");
 	fprintf(stderr, "\tann <local-ip> id <local-id> token <token> [port <local-port>] [dev <name>]\n");
 	fprintf(stderr, "\trem id <local-id> token <token>\n");
-- 
2.35.3

Re: selftests: mptcp: update pm_nl_ctl usage header: Tests Results

[MPTCP CI]

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal:
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/6339386420756480
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/6339386420756480/summary/summary.txt

- KVM Validation: debug:
  - Unstable: 2 failed test(s): packetdrill_sockopts selftest_diag 🔴:
  - Task: https://cirrus-ci.com/task/4932011537203200
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/4932011537203200/summary/summary.txt

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/d4a1e2c951f3


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

[PATCH mptcp-next] selftests: mptcp: update pm_nl_ctl usage header

[Geliang Tang]

The usage header of pm_nl_ctl command doesn't match with the context. So
this patch adds the missing userspace PM keywords 'ann', 'rem', 'csf',
'dsf', 'events' and 'listen' in it.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 tools/testing/selftests/net/mptcp/pm_nl_ctl.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/net/mptcp/pm_nl_ctl.c b/tools/testing/selftests/net/mptcp/pm_nl_ctl.c
index 6a2f4b981e1d..4dd87bb9ee91 100644
--- a/tools/testing/selftests/net/mptcp/pm_nl_ctl.c
+++ b/tools/testing/selftests/net/mptcp/pm_nl_ctl.c
@@ -31,7 +31,7 @@
 
 static void syntax(char *argv[])
 {
-	fprintf(stderr, "%s add|get|set|del|flush|dump|accept [<args>]\n", argv[0]);
+	fprintf(stderr, "%s add|ann|rem|csf|dsf|get|set|del|flush|dump|events|listen|accept [<args>]\n", argv[0]);
 	fprintf(stderr, "\tadd [flags signal|subflow|backup|fullmesh] [id <nr>] [dev <name>] <ip>\n");
 	fprintf(stderr, "\tann <local-ip> id <local-id> token <token> [port <local-port>] [dev <name>]\n");
 	fprintf(stderr, "\trem id <local-id> token <token>\n");
-- 
2.35.3

Re: selftests: mptcp: update pm_nl_ctl usage header: Tests Results

[MPTCP CI]

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal:
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/6017698135015424
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/6017698135015424/summary/summary.txt

- KVM Validation: debug:
  - Unstable: 2 failed test(s): selftest_diag selftest_simult_flows 🔴:
  - Task: https://cirrus-ci.com/task/6246459333935104
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/6246459333935104/summary/summary.txt

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/c86f7081c919


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)