[PATCH mptcp-next 0/2] Add MPTCP support for NBD

[PATCH mptcp-next 0/2] Add MPTCP support for NBD

[Geliang Tang]

From: Geliang Tang <tanggeliang@kylinos.cn>

This series adds MPTCP (Multipath TCP) support to NBD (Network Block
Device) and includes comprehensive tests.

MPTCP can aggregate bandwidth from multiple network interfaces, providing
significant performance improvements in multi-NIC scenarios. Our tests
with 4 virtual NICs (each limited to 1000 Mbit/s / 125 MB/s) show ~4x
bandwidth improvement: TCP achieves ~110 MiB/s while MPTCP achieves
~400 MiB/s for both read and write operations.

The first patch adds MPTCP socket support to the kernel NBD driver by
allowing MPTCP sockets in nbd_get_socket().

The second patch adds a comprehensive test script (mptcp_nbd.sh) that
validates NBD over both TCP and MPTCP.

Geliang Tang (2):
  nbd: allow MPTCP sockets
  selftests: mptcp: add mptcp nbd tests

 drivers/block/nbd.c                           |   4 +-
 tools/testing/selftests/net/mptcp/Makefile    |   1 +
 tools/testing/selftests/net/mptcp/config      |   1 +
 .../testing/selftests/net/mptcp/mptcp_lib.sh  |  13 +
 .../testing/selftests/net/mptcp/mptcp_nbd.sh  | 309 ++++++++++++++++++
 5 files changed, 327 insertions(+), 1 deletion(-)
 create mode 100755 tools/testing/selftests/net/mptcp/mptcp_nbd.sh

-- 
2.55.0

[PATCH mptcp-next 1/2] nbd: allow MPTCP sockets

[Geliang Tang]

From: Geliang Tang <tanggeliang@kylinos.cn>

MPTCP (Multipath TCP, RFC 8684) is an extension to TCP introduced in
Linux 5.6 that can combine bandwidth from multiple network interfaces.

For NBD (Network Block Device), MPTCP provides significant bandwidth
aggregation in multi-NIC scenarios. Our tests with 4 virtual NICs
(each limited to 1000 Mbit/s / 125 MB/s) show ~4x improvement,
TCP achieves ~110 MiB/s while MPTCP achieves ~400 MiB/s for both
read and write:

 # ./mptcp_nbd.sh mptcp
 INFO: nbd-server lacks native MPTCP, using mptcpize
 INFO: nbd-client lacks native MPTCP, using mptcpize
 Negotiation: ..size = 256MB
 Connected /dev/nbd0
 connected

 fio-3.41
 ... ...
 Run status group 0 (all jobs):
    READ: bw=406MiB/s (426MB/s), 406MiB/s-406MiB/s (426MB/s-426MB/s), \
		io=4063MiB (4261MB), run=10001-10001msec
 ... ...
 Run status group 0 (all jobs):
   WRITE: bw=376MiB/s (394MB/s), 376MiB/s-376MiB/s (394MB/s-394MB/s), \
		io=3758MiB (3940MB), run=10001-10001msec

MPTCP sockets can be created with:

  fd = socket(AF_INET, SOCK_STREAM, IPPROTO_MPTCP);

Or enabled transparently using the mptcpize tool without modifying
applications (mptcpize run nbd-server, mptcpize run nbd-client).
Future patches will add -M (--mptcp) options to nbd-server and
nbd-client for native MPTCP support.

Currently, the kernel NBD driver only accepts TCP and UNIX sockets.
This patch adds MPTCP socket support by adding msk_is_mptcp() check
in nbd_get_socket().

Signed-off-by: Geliang Tang <tanggeliang@kylinos.cn>
---
 drivers/block/nbd.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index ffce519bf008..2aeb22b1ad40 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -33,6 +33,7 @@
 #include <linux/kernel.h>
 #include <linux/slab.h>
 #include <net/sock.h>
+#include <net/mptcp.h>
 #include <linux/net.h>
 #include <linux/kthread.h>
 #include <linux/types.h>
@@ -1221,8 +1222,9 @@ static struct socket *nbd_get_socket(struct nbd_device *nbd, unsigned long fd,
 		return NULL;
 
 	if (!sk_is_tcp(sock->sk) &&
+	    !msk_is_mptcp(sock->sk) &&
 	    !sk_is_stream_unix(sock->sk)) {
-		dev_err(disk_to_dev(nbd->disk), "Unsupported socket: should be TCP or UNIX.\n");
+		dev_err(disk_to_dev(nbd->disk), "Unsupported socket: should be TCP, MPTCP or UNIX.\n");
 		*err = -EINVAL;
 		sockfd_put(sock);
 		return NULL;
-- 
2.55.0

[PATCH mptcp-next 2/2] selftests: mptcp: add mptcp nbd tests

[Geliang Tang]

From: Geliang Tang <tanggeliang@kylinos.cn>

Add mptcp_nbd.sh to verify NBD (Network Block Device) works correctly
over both TCP and MPTCP.

Both ns1 and ns2 have 4 virtual network interfaces, each limited to
1000 Mbit/s (125 MB/s). When running NBD over TCP, fio tests show
read/write bandwidth of about 110 MiB/s:

 # ./mptcp_nbd.sh tcp
 Negotiation: ..size = 256MB
 Connected /dev/nbd0
 connected

 fio-3.41
 ... ...
 Run status group 0 (all jobs):
    READ: bw=112MiB/s (117MB/s), 112MiB/s-112MiB/s (117MB/s-117MB/s), \
		io=1119MiB (1173MB), run=10002-10002msec
 ... ...
 Run status group 0 (all jobs):
   WRITE: bw=113MiB/s (118MB/s), 113MiB/s-113MiB/s (118MB/s-118MB/s), \
		io=1129MiB (1184MB), run=10002-10002msec

With simple MPTCP configuration and running MPTCP NBD, we can achieve
about 400 MiB/s read/write bandwidth:

 # ./mptcp_nbd.sh mptcp
 INFO: nbd-server lacks native MPTCP, using mptcpize
 INFO: nbd-client lacks native MPTCP, using mptcpize
 Negotiation: ..size = 256MB
 Connected /dev/nbd0
 connected

 fio-3.41
 ... ...
 Run status group 0 (all jobs):
    READ: bw=406MiB/s (426MB/s), 406MiB/s-406MiB/s (426MB/s-426MB/s), \
		io=4063MiB (4261MB), run=10001-10001msec
 ... ...
 Run status group 0 (all jobs):
   WRITE: bw=376MiB/s (394MB/s), 376MiB/s-376MiB/s (394MB/s-394MB/s), \
		io=3758MiB (3940MB), run=10001-10001msec

Signed-off-by: Geliang Tang <tanggeliang@kylinos.cn>
---
 tools/testing/selftests/net/mptcp/Makefile    |   1 +
 tools/testing/selftests/net/mptcp/config      |   1 +
 .../testing/selftests/net/mptcp/mptcp_lib.sh  |  13 +
 .../testing/selftests/net/mptcp/mptcp_nbd.sh  | 309 ++++++++++++++++++
 4 files changed, 324 insertions(+)
 create mode 100755 tools/testing/selftests/net/mptcp/mptcp_nbd.sh

diff --git a/tools/testing/selftests/net/mptcp/Makefile b/tools/testing/selftests/net/mptcp/Makefile
index 22ba0da2adb8..2c8b12866f81 100644
--- a/tools/testing/selftests/net/mptcp/Makefile
+++ b/tools/testing/selftests/net/mptcp/Makefile
@@ -13,6 +13,7 @@ TEST_PROGS := \
 	mptcp_connect_sendfile.sh \
 	mptcp_connect_splice.sh \
 	mptcp_join.sh \
+	mptcp_nbd.sh \
 	mptcp_sockopt.sh \
 	pm_netlink.sh \
 	simult_flows.sh \
diff --git a/tools/testing/selftests/net/mptcp/config b/tools/testing/selftests/net/mptcp/config
index 6a96cfbbc461..f6f266211bee 100644
--- a/tools/testing/selftests/net/mptcp/config
+++ b/tools/testing/selftests/net/mptcp/config
@@ -1,3 +1,4 @@
+CONFIG_BLK_DEV_NBD=y
 CONFIG_INET_DIAG=m
 CONFIG_INET_MPTCP_DIAG=m
 CONFIG_IP_ADVANCED_ROUTER=y
diff --git a/tools/testing/selftests/net/mptcp/mptcp_lib.sh b/tools/testing/selftests/net/mptcp/mptcp_lib.sh
index e1c8c6fd8820..aee7a73d9772 100644
--- a/tools/testing/selftests/net/mptcp/mptcp_lib.sh
+++ b/tools/testing/selftests/net/mptcp/mptcp_lib.sh
@@ -541,6 +541,19 @@ mptcp_lib_check_tools() {
 				exit ${KSFT_SKIP}
 			fi
 			;;
+		"nbd")
+			if ! command -v nbd-client &> /dev/null || \
+			   ! command -v nbd-server &> /dev/null; then
+				mptcp_lib_pr_skip "nbd tools not found"
+				exit ${KSFT_SKIP}
+			fi
+			;;
+		"fio")
+			if ! fio -h &> /dev/null; then
+				mptcp_lib_pr_skip "fio tool not found"
+				exit ${KSFT_SKIP}
+			fi
+			;;
 		*)
 			mptcp_lib_pr_fail "Internal error: unsupported tool: ${tool}"
 			exit ${KSFT_FAIL}
diff --git a/tools/testing/selftests/net/mptcp/mptcp_nbd.sh b/tools/testing/selftests/net/mptcp/mptcp_nbd.sh
new file mode 100755
index 000000000000..935dea5738aa
--- /dev/null
+++ b/tools/testing/selftests/net/mptcp/mptcp_nbd.sh
@@ -0,0 +1,309 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+. "$(dirname "$0")/mptcp_lib.sh"
+
+ret=0
+trtype="${1:-mptcp}"
+ns1=""
+ns2=""
+port=$((RANDOM % 10000 + 20000))
+temp_file=""
+server_pid=""
+
+export trtype port
+export temp_file server_pid
+
+usage()
+{
+	cat << EOF
+
+Usage:
+
+	$(basename "$0") [trtype]
+
+	trtype   Transport type (tcp|mptcp) - default: mptcp
+
+EOF
+exit ${KSFT_FAIL}
+}
+
+validate_params()
+{
+	if [[ ! "${trtype}" =~ ^(tcp|mptcp)$ ]]; then
+		echo "Invalid trtype ${trtype}. Must be tcp or mptcp"
+		usage
+	fi
+}
+
+# This function is used in the cleanup trap
+#shellcheck disable=SC2317,SC2329
+cleanup()
+{
+	# Disconnect nbd device in ns2
+	ip netns exec "$ns2" nbd-client -d /dev/nbd0 2>/dev/null || true
+
+	sleep 1
+
+	# Kill nbd-server in ns1
+	if [ -n "${server_pid}" ]; then
+		ip netns exec "$ns1" kill "${server_pid}" 2>/dev/null || true
+	fi
+	ip netns exec "$ns1" pkill -9 -f nbd-server 2>/dev/null || true
+
+	# Clean up temp files
+	if [ -n "${temp_file}" ]; then
+		ip netns exec "$ns1" rm -f "${temp_file}" 2>/dev/null || true
+	fi
+
+	mptcp_lib_ns_exit "$ns1" "$ns2"
+
+	unset -v trtype port
+	unset -v temp_file server_pid
+}
+
+init()
+{
+	mptcp_lib_ns_init ns1 ns2
+
+	ip netns exec "$ns1" sysctl -qw net.ipv4.tcp_reflect_tos=1
+
+	# ns1		ns2
+	# 10.1.1.1	10.1.1.2
+	# 10.1.2.1	10.1.2.2
+	# 10.1.3.1	10.1.3.2
+	# 10.1.4.1	10.1.4.2
+	for i in {1..4}; do
+		ip link add ns1eth"$i" netns "$ns1" type veth peer \
+			name ns2eth"$i" netns "$ns2"
+		ip -net "$ns1" addr add 10.1."$i".1/24 dev ns1eth"$i"
+		ip -net "$ns1" link set ns1eth"$i" up
+		ip -net "$ns2" addr add 10.1."$i".2/24 dev ns2eth"$i"
+		ip -net "$ns2" link set ns2eth"$i" up
+		ip -net "$ns2" route add default via 10.1."$i".1 \
+			dev ns2eth"$i" metric 10"$i"
+
+		# Rate limit to 1000mbit
+		tc -n "$ns1" qdisc add dev ns1eth"$i" root netem rate 1000mbit
+		tc -n "$ns2" qdisc add dev ns2eth"$i" root netem rate 1000mbit
+	done
+
+	mptcp_lib_pm_nl_set_limits "${ns1}" 8 8
+	mptcp_lib_pm_nl_add_endpoint "$ns1" 10.1.1.1 flags signal
+	mptcp_lib_pm_nl_add_endpoint "$ns1" 10.1.2.1 flags signal
+	mptcp_lib_pm_nl_add_endpoint "$ns1" 10.1.3.1 flags signal
+	mptcp_lib_pm_nl_add_endpoint "$ns1" 10.1.4.1 flags signal
+
+	mptcp_lib_pm_nl_set_limits "${ns2}" 8 8
+	mptcp_lib_pm_nl_add_endpoint "$ns2" 10.1.1.2 flags subflow
+	mptcp_lib_pm_nl_add_endpoint "$ns2" 10.1.2.2 flags subflow
+	mptcp_lib_pm_nl_add_endpoint "$ns2" 10.1.3.2 flags subflow
+	mptcp_lib_pm_nl_add_endpoint "$ns2" 10.1.4.2 flags subflow
+}
+
+# This function is invoked indirectly
+#shellcheck disable=SC2317,SC2329
+run_server()
+{
+	local conf_file="/tmp/nbd-server.conf"
+
+	# Create backing file
+	temp_file="/tmp/nbd_backing.raw"
+	dd if=/dev/zero of="${temp_file}" bs=1M count=0 seek=256 2>/dev/null
+
+	# Detect MPTCP support: native config or mptcpize fallback
+	local use_mptcpize=""
+	local mptcp_opt=""
+	if [ "${trtype}" = "mptcp" ]; then
+		if ! grep -q '"mptcp"' /usr/bin/nbd-server 2>/dev/null && \
+		   ! nbd-server --help 2>&1 | grep -qi mptcp; then
+			if command -v mptcpize &> /dev/null; then
+				use_mptcpize="mptcpize run -d"
+				echo "INFO: nbd-server lacks native MPTCP," \
+					"using mptcpize"
+			else
+				echo "ERROR: no MPTCP support in nbd-server"
+				return 1
+			fi
+		else
+			mptcp_opt="mptcp = true"
+			echo "INFO: nbd-server has native MPTCP support"
+		fi
+	fi
+
+	cat > "${conf_file}" << CONF
+[generic]
+	port = ${port}
+	listenaddr = 0.0.0.0
+	${mptcp_opt}
+
+[test]
+	exportname = ${temp_file}
+CONF
+
+	# Use -n (nodaemon) and background manually for better control
+	${use_mptcpize} nbd-server -C "${conf_file}" -n > /dev/null 2>&1 &
+	server_pid=$!
+
+	sleep 2
+
+	# Verify server is running
+	if ! kill -0 "${server_pid}" 2>/dev/null; then
+		echo "nbd-server failed to start"
+		return 1
+	fi
+}
+
+# This function is invoked indirectly
+#shellcheck disable=SC2317,SC2329
+run_client()
+{
+	local extra_args=()
+	local nbd_dev="/dev/nbd0"
+	local server_addr="10.1.1.1"
+
+	local use_mptcpize=""
+	if [ "${trtype}" = "mptcp" ]; then
+		if nbd-client 2>&1 | grep -qE "\-M|\-\-mptcp"; then
+			extra_args+=("-M")
+			echo "INFO: nbd-client has native MPTCP support (-M)"
+		elif command -v mptcpize &> /dev/null; then
+			use_mptcpize="mptcpize run -d"
+			echo "INFO: nbd-client lacks native MPTCP," \
+				"using mptcpize"
+		else
+			echo "ERROR: no MPTCP support in nbd-client"
+			return 1
+		fi
+	fi
+
+	# Options before positional args: -N name host port device
+	if ! ${use_mptcpize} nbd-client -N test "${extra_args[@]}" \
+			"${server_addr}" "${port}" "${nbd_dev}"; then
+		echo "Failed to connect ${nbd_dev} to ${server_addr}"
+		return 1
+	fi
+
+	sleep 1
+
+	# Verify connection
+	if ! nbd-client -c /dev/nbd0; then
+		echo "nbd-client check failed"
+		return 1
+	fi
+
+	return 0
+}
+
+# This function is invoked indirectly
+#shellcheck disable=SC2317,SC2329
+run_io_test()
+{
+	local nbd_dev="/dev/nbd0"
+
+	# dd write test
+	if ! dd if=/dev/urandom of="${nbd_dev}" bs=4k count=1000 \
+			conv=fdatasync 2>/dev/null; then
+		echo "dd write failed"
+		return 1
+	fi
+
+	# dd read test
+	if ! dd if="${nbd_dev}" of=/dev/null bs=4k count=1000 2>/dev/null; then
+		echo "dd read failed"
+		return 1
+	fi
+
+	# fio random read test
+	if ! fio --name=randread --direct=1 --ioengine=libaio \
+		 --blocksize=4k --runtime=10 --time_based \
+		 --rw=randread --numjobs=1 --iodepth=32 \
+		 --group_reporting --size=128M \
+		 --filename="${nbd_dev}"; then
+		echo "fio randread failed"
+		return 1
+	fi
+
+	# fio random write test
+	if ! fio --name=randwrite --direct=1 --ioengine=libaio \
+		 --blocksize=4k --runtime=10 --time_based \
+		 --rw=randwrite --numjobs=1 --iodepth=32 \
+		 --group_reporting --size=128M \
+		 --filename="${nbd_dev}"; then
+		echo "fio randwrite failed"
+		return 1
+	fi
+
+	# Verify MPTCP subflows if trtype is mptcp
+	if [ "${trtype}" = "mptcp" ]; then
+		local sf_total sf_max tcp_conns is_fallback
+		sf_total=$(ss -Mia | grep -oP "subflows_total:\K[0-9]+" | \
+			head -1)
+		sf_max=$(ss -Mia | grep -oP "subflows_max:\K[0-9]+" | head -1)
+		tcp_conns=$(ss -tia | grep -c "tcp-ulp-mptcp" || true)
+		is_fallback=$(ss -Mia | grep -c "fallback" || true)
+		if [ "${is_fallback}" -gt 0 ]; then
+			echo "FAIL: MPTCP fell back to TCP!" \
+				"subflows_total:${sf_total}" \
+				"tcp_subflows:${tcp_conns}"
+		elif [ -n "${sf_total}" ]; then
+			echo "PASS: MPTCP subflows_total:${sf_total}" \
+				"subflows_max:${sf_max}" \
+				"tcp_subflows:${tcp_conns}"
+		else
+			echo "WARNING: No MPTCP subflows info found"
+		fi
+	fi
+
+	return 0
+}
+
+mptcp_lib_check_tools nbd fio
+validate_params
+
+trap cleanup EXIT
+
+init
+
+run_test()
+{
+	if ! ip netns exec "$ns1" bash <<- EOF
+		$(declare -f run_server)
+		export trtype port
+		run_server
+		exit \$?
+	EOF
+	then
+		ret="${KSFT_FAIL}"
+		return
+	fi
+
+	# Get server pid for cleanup
+	server_pid=$(ip netns exec "$ns1" pgrep -f "nbd-server" | head -1)
+
+	sleep 1
+
+	if ! ip netns exec "$ns2" bash <<- EOF
+		$(declare -f run_client run_io_test)
+		export trtype port
+		run_client || exit 1
+		run_io_test || exit 1
+		exit \$?
+	EOF
+	then
+		ret="${KSFT_FAIL}"
+	fi
+
+	sleep 1
+}
+
+run_test "$@"
+
+if [ "${ret}" -eq 0 ]; then
+	mptcp_lib_result_pass "nbd over ${trtype} test"
+else
+	mptcp_lib_result_fail "nbd over ${trtype} test"
+fi
+
+mptcp_lib_result_print_all_tap
+exit "$ret"
-- 
2.55.0

Re: [PATCH mptcp-next 2/2] selftests: mptcp: add mptcp nbd tests

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
- [Medium] selftests: mptcp: run_io_test() returns 0 on fallback or missing subflows
- [Medium] selftests: mptcp: missing prerequisite checks cause failure instead of skip
- [Medium] selftests: mptcp: temp files leaked in cleanup() due to subshell assignment
- [Medium] selftests: mptcp: host-wide pkill/pgrep and /dev/nbd0 clobber in cleanup()
--

commit 3361c31ea4cfbf255e2c3ce5c5e70d2193d46dd4
Author: Geliang Tang <geliang@kernel.org>

selftests: mptcp: add mptcp nbd tests

This commit adds the mptcp_nbd.sh selftest to verify Network Block Device
(NBD) functionality and throughput over both TCP and MPTCP across multiple
virtual network interfaces.

> diff --git a/tools/testing/selftests/net/mptcp/mptcp_nbd.sh b/tools/testing/selftests/net/mptcp/mptcp_nbd.sh
> new file mode 100755
> index 0000000000000..935dea5738aa1
> --- /dev/null
> +++ b/tools/testing/selftests/net/mptcp/mptcp_nbd.sh
> @@ -0,0 +1,309 @@
[ ... ]
> +cleanup()
> +{
> +	# Disconnect nbd device in ns2
> +	ip netns exec "$ns2" nbd-client -d /dev/nbd0 2>/dev/null || true
> +
> +	sleep 1
> +
> +	# Kill nbd-server in ns1
> +	if [ -n "${server_pid}" ]; then
> +		ip netns exec "$ns1" kill "${server_pid}" 2>/dev/null || true
> +	fi
> +	ip netns exec "$ns1" pkill -9 -f nbd-server 2>/dev/null || true

[Severity: Medium]
Since ip netns exec only switches the network namespace while sharing the
host PID namespace, won't running pkill -9 -f nbd-server in cleanup() kill
all nbd-server processes across the entire host system, and won't pgrep -f
"nbd-server" in run_test() match unrelated host processes as well?

Also, /dev/nbd0 is a host-global block device that is not isolated by
network namespaces. Could run_client(), run_io_test(), and cleanup()
clobber or disconnect /dev/nbd0 if it is already in use on the host?

> +
> +	# Clean up temp files
> +	if [ -n "${temp_file}" ]; then
> +		ip netns exec "$ns1" rm -f "${temp_file}" 2>/dev/null || true
> +	fi

[Severity: Medium]
Does cleanup() leak /tmp/nbd_backing.raw and /tmp/nbd-server.conf on every
run?

In run_test(), run_server() is executed inside a child bash process:

	if ! ip netns exec "$ns1" bash <<- EOF
		$(declare -f run_server)
		export trtype port
		run_server
		exit \$?
	EOF

Because temp_file="/tmp/nbd_backing.raw" is assigned inside that child
process, temp_file in the parent shell remains empty when the EXIT trap
invokes cleanup(). In addition, conf_file="/tmp/nbd-server.conf" is a local
variable in run_server() with no removal in cleanup(), and both use
hardcoded /tmp paths instead of mktemp.

[ ... ]
> +run_server()
> +{
> +	local conf_file="/tmp/nbd-server.conf"
> +
> +	# Create backing file
> +	temp_file="/tmp/nbd_backing.raw"
> +	dd if=/dev/zero of="${temp_file}" bs=1M count=0 seek=256 2>/dev/null
> +
> +	# Detect MPTCP support: native config or mptcpize fallback
> +	local use_mptcpize=""
> +	local mptcp_opt=""
> +	if [ "${trtype}" = "mptcp" ]; then
> +		if ! grep -q '"mptcp"' /usr/bin/nbd-server 2>/dev/null && \
> +		   ! nbd-server --help 2>&1 | grep -qi mptcp; then
> +			if command -v mptcpize &> /dev/null; then
> +				use_mptcpize="mptcpize run -d"
> +				echo "INFO: nbd-server lacks native MPTCP," \
> +					"using mptcpize"
> +			else
> +				echo "ERROR: no MPTCP support in nbd-server"
> +				return 1
> +			fi

[Severity: Medium]
Should the test skip with KSFT_SKIP instead of failing with KSFT_FAIL when
optional prerequisites are unavailable?

In run_server() and run_client(), if trtype is mptcp and neither native
MPTCP support in nbd-server/nbd-client nor mptcpize is installed, returning
1 causes run_test() to set ret=${KSFT_FAIL}. Also, run_server() hardcodes
/usr/bin/nbd-server rather than using command -v nbd-server.

Similarly, at the top level of mptcp_nbd.sh:

mptcp_lib_check_tools nbd fio
validate_params

if MPTCP is not supported by the kernel (checked via mptcp_lib_check_mptcp
in other mptcp selftests), ip/tc/ss are missing, or /dev/nbd0 is not
present (for example when CONFIG_BLK_DEV_NBD is disabled or the nbd module
is not loaded), the script will fail instead of skipping.

[ ... ]
> +	# Verify MPTCP subflows if trtype is mptcp
> +	if [ "${trtype}" = "mptcp" ]; then
> +		local sf_total sf_max tcp_conns is_fallback
> +		sf_total=$(ss -Mia | grep -oP "subflows_total:\K[0-9]+" | \
> +			head -1)
> +		sf_max=$(ss -Mia | grep -oP "subflows_max:\K[0-9]+" | head -1)
> +		tcp_conns=$(ss -tia | grep -c "tcp-ulp-mptcp" || true)
> +		is_fallback=$(ss -Mia | grep -c "fallback" || true)
> +		if [ "${is_fallback}" -gt 0 ]; then
> +			echo "FAIL: MPTCP fell back to TCP!" \
> +				"subflows_total:${sf_total}" \
> +				"tcp_subflows:${tcp_conns}"
> +		elif [ -n "${sf_total}" ]; then
> +			echo "PASS: MPTCP subflows_total:${sf_total}" \
> +				"subflows_max:${sf_max}" \
> +				"tcp_subflows:${tcp_conns}"
> +		else
> +			echo "WARNING: No MPTCP subflows info found"
> +		fi
> +	fi
> +
> +	return 0
> +}

[Severity: Medium]
In run_io_test(), when is_fallback is greater than 0 or sf_total is empty,
the function prints a FAIL or WARNING message but still falls through to
return 0, and the elif branch only checks that sf_total is non-empty rather
than verifying that multiple subflows were created. Doesn't this cause
mptcp_nbd.sh to report KSFT_PASS even when MPTCP falls back to TCP or fails
to establish subflows?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1792746800.git.tanggeliang@kylinos.cn?part=2

Re: [PATCH mptcp-next 1/2] nbd: allow MPTCP sockets

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] nbd: missing SOCK_MEMALLOC and sk_allocation propagation to MPTCP subflows
- [High] nbd: lockdep circular dependency and reclaim deadlock with MPTCP sockets
--

commit dbca698e3d4256ddeebe565fd33ea333c059d349
Author: Geliang Tang <geliang@kernel.org>

nbd: allow MPTCP sockets

This commit allows MPTCP sockets to be used with NBD devices by checking
msk_is_mptcp(sock->sk) in nbd_get_socket().

> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index ffce519bf0087..2aeb22b1ad40e 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c

[ ... ]

> @@ -1221,8 +1222,9 @@ static struct socket *nbd_get_socket(struct nbd_device *nbd, unsigned long fd,
>  		return NULL;
>  
>  	if (!sk_is_tcp(sock->sk) &&
> +	    !msk_is_mptcp(sock->sk) &&
>  	    !sk_is_stream_unix(sock->sk)) {

[Severity: High]
When an MPTCP socket is attached to NBD, does SOCK_MEMALLOC and
sk_allocation get propagated to the underlying TCP subflow sockets (ssk)?

In drivers/block/nbd.c, nbd_start_device() and
nbd_genl_reconnect_sock_cb() call sk_set_memalloc(sock->sk), and
__sock_xmit() configures sock->sk->sk_allocation:

drivers/block/nbd.c:__sock_xmit()
    sock->sk->sk_allocation = GFP_NOIO | __GFP_MEMALLOC;
    sock->sk->sk_use_task_frag = false;

For an MPTCP socket, sock->sk is the master struct mptcp_sock (msk), while
actual wire segments are sent and received on the underlying TCP subflow
sockets (ssk). Calling sk_set_memalloc() on msk enables the global
memalloc_socks_key static key, but neither NBD nor MPTCP (such as
sync_socket_options() in net/mptcp/sockopt.c or
mptcp_subflow_create_socket() in net/mptcp/subflow.c) propagates
SOCK_MEMALLOC or sk_allocation to existing or newly created subflows.

Under memory pressure (for example, swap or writeback over NBD), incoming
packets allocated from emergency reserves have skb_pfmemalloc(skb) set.
When tcp_v4_rcv() or tcp_v6_rcv() looks up the subflow socket ssk, both
sk_filter_trim_cap() in net/core/filter.c:

    if (skb_pfmemalloc(skb) && !sock_flag(sk, SOCK_MEMALLOC)) {
        NET_INC_STATS(sock_net(sk), LINUX_MIB_PFMEMALLOCDROP);
        return SKB_DROP_REASON_PFMEMALLOC;
    }

and sk_add_backlog() in include/net/sock.h:

    if (skb_pfmemalloc(skb) && !sock_flag(sk, SOCK_MEMALLOC))
        return -ENOMEM;

check ssk rather than msk, dropping all incoming pfmemalloc packets. In
addition, subflow TX allocations in __tcp_push_pending_frames() and
__tcp_send_ack() use ssk->sk_allocation (which remains GFP_KERNEL without
__GFP_MEMALLOC). Won't this stall NBD I/O completion and cause a memory
reclaim deadlock under memory pressure?

[Severity: High]
Could attaching an MPTCP socket to NBD also trigger a lockdep circular
locking dependency splat and a reclaim deadlock via subflow socket locks
and mptcp_worker()?

When nbd_add_socket() attaches a socket, nbd_reclassify_socket() in
drivers/block/nbd.c only reclassifies the top-level sock->sk (msk) to
sk_lock-AF_INET-NBD or sk_lock-AF_INET6-NBD. The underlying TCP subflow
sockets created in mptcp_subflow_create_socket() via sock_create_kern()
retain the generic k-sk_lock-AF_INET or k-sk_lock-AF_INET6 lock class,
under which GFP_KERNEL allocations are performed:

net/mptcp/subflow.c:mptcp_subflow_create_socket()
    err = sock_create_kern(net, family, SOCK_STREAM, IPPROTO_TCP, &sf);
    if (err)
        return err;

    lock_sock_nested(sf->sk, SINGLE_DEPTH_NESTING);
    ...
    sk_net_refcnt_upgrade(sf->sk);
    err = tcp_set_ulp(sf->sk, "mptcp");

This establishes k-sk_lock-AF_INET -> fs_reclaim. Meanwhile, on the NBD I/O
dispatch path, nbd_queue_rq() -> nbd_handle_cmd() -> nbd_send_cmd() ->
sock_xmit() -> __sock_xmit() -> sock_sendmsg() -> mptcp_sendmsg() ->
__mptcp_push_pending() runs under set->srcu (which is chained after
fs_reclaim -> &q->q_usage_counter(io)) and acquires lock_sock(ssk) in
net/mptcp/protocol.c:

net/mptcp/protocol.c:__mptcp_push_pending()
    lock_sock(ssk);

completing a circular dependency chain between k-sk_lock-AF_INET and
fs_reclaim.

Furthermore, mptcp_worker() in net/mptcp/protocol.c runs in workqueue
context without GFP_NOIO / PF_MEMALLOC, acquires lock_sock(sk) on msk
(sk_lock-AF_INET-NBD), and calls mptcp_pm_worker() and
__mptcp_close_subflow(), which perform GFP_KERNEL allocations while holding
msk->sk_lock. Can this cause a lockdep splat and deadlocks if
mptcp_worker() enters direct reclaim waiting on NBD writeback while
nbd_queue_rq() or recv_work() waits for lock_sock(msk)?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1792746800.git.tanggeliang@kylinos.cn?part=1

Re: [PATCH mptcp-next 0/2] Add MPTCP support for NBD

[MPTCP CI]

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Unstable: 2 failed test(s): selftest_mptcp_nbd selftest_simult_flows ⚠️ 
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Unstable: 1 failed test(s): selftest_mptcp_nbd ⚠️ 
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Perf: Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/39933748069

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/04bb6f5fada2
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1191964


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)

Re: [PATCH mptcp-next 0/2] Add MPTCP support for NBD

[MPTCP CI]

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join): Success! ✅
- KVM Validation: normal (only selftest_mptcp_join): Success! ✅
- KVM Validation: debug (except selftest_mptcp_join): Success! ✅
- KVM Validation: debug (only selftest_mptcp_join): Success! ✅
- KVM Validation: btf-normal (only bpftest_all): Success! ✅
- KVM Validation: btf-debug (only bpftest_all): Success! ✅
- Perf: Success! ✅
- Task: https://github.com/multipath-tcp/mptcp_net-next/actions/runs/39933748069

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/04bb6f5fada2
Patchwork: https://patchwork.kernel.org/project/mptcp/list/?series=1191964


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-normal

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (NGI0 Core)

Re: [PATCH mptcp-next 1/2] nbd: allow MPTCP sockets

[Matthieu Baerts]

Hi Geliang,

On 23/10/2026 11:19, Geliang Tang wrote:
> From: Geliang Tang <tanggeliang@kylinos.cn>
> 
> MPTCP (Multipath TCP, RFC 8684) is an extension to TCP introduced in
> Linux 5.6 that can combine bandwidth from multiple network interfaces.
> 
> For NBD (Network Block Device), MPTCP provides significant bandwidth
> aggregation in multi-NIC scenarios. Our tests with 4 virtual NICs
> (each limited to 1000 Mbit/s / 125 MB/s) show ~4x improvement,
> TCP achieves ~110 MiB/s while MPTCP achieves ~400 MiB/s for both
> read and write:
> 
>  # ./mptcp_nbd.sh mptcp
>  INFO: nbd-server lacks native MPTCP, using mptcpize
>  INFO: nbd-client lacks native MPTCP, using mptcpize
>  Negotiation: ..size = 256MB
>  Connected /dev/nbd0
>  connected
> 
>  fio-3.41
>  ... ...
>  Run status group 0 (all jobs):
>     READ: bw=406MiB/s (426MB/s), 406MiB/s-406MiB/s (426MB/s-426MB/s), \
> 		io=4063MiB (4261MB), run=10001-10001msec
>  ... ...
>  Run status group 0 (all jobs):
>    WRITE: bw=376MiB/s (394MB/s), 376MiB/s-376MiB/s (394MB/s-394MB/s), \
> 		io=3758MiB (3940MB), run=10001-10001msec
> 
> MPTCP sockets can be created with:
> 
>   fd = socket(AF_INET, SOCK_STREAM, IPPROTO_MPTCP);
> 
> Or enabled transparently using the mptcpize tool without modifying
> applications (mptcpize run nbd-server, mptcpize run nbd-client).
> Future patches will add -M (--mptcp) options to nbd-server and
> nbd-client for native MPTCP support.
> 
> Currently, the kernel NBD driver only accepts TCP and UNIX sockets.
> This patch adds MPTCP socket support by adding msk_is_mptcp() check
> in nbd_get_socket().

Just to be sure: are you sure it is safe with just that? We got a bunch
of Syzbot/Syzkaller reports back then before commit:

  9f7c02e03157 ("nbd: restrict sockets to TCP and UDP")

https://github.com/multipath-tcp/mptcp_net-next/issues/599

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.

Re: [PATCH mptcp-next 1/2] nbd: allow MPTCP sockets

[netdev-bot+sashiko]

Thank you for your contribution! Sashiko AI review found 3 potential
issue(s) to consider.

Critical: 0 · High: 2 · Medium: 1 · Low: 0

If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).

- [High] NBD's memory-reserve setup for its socket does not reach the TCP
  subflows of an MPTCP socket.
- [High] Accepting MPTCP sockets lets an in-kernel, process-context
  worker hold the NBD msk socket lock while making reclaim-capable…
- [Medium] nbd_reclassify_socket() (drivers/block/nbd.c:1247-1274) moves
  only sock->sk into the NBD lock classes.

--

nbd: allow MPTCP sockets

This patch adds an msk_is_mptcp() check to nbd_get_socket(), so NBD
accepts MPTCP sockets as well as TCP and stream UNIX sockets. The
stated goal is to aggregate bandwidth across several NICs.

> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index ffce519bf0087..2aeb22b1ad40e 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c

[ ... ]

> @@ -1221,8 +1222,9 @@ static struct socket *nbd_get_socket(struct nbd_device *nbd, unsigned long fd,
>  		return NULL;
>  
>  	if (!sk_is_tcp(sock->sk) &&
> +	    !msk_is_mptcp(sock->sk) &&

[Severity: High]
Once an MPTCP socket is accepted here, does NBD's memalloc setup reach
the sockets that actually carry the traffic?

NBD marks sock->sk in nbd_start_device() and
nbd_genl_reconnect_sock_cb():

    sk_set_memalloc(config->socks[i]->sock->sk);

and __sock_xmit() sets:

    sock->sk->sk_allocation = GFP_NOIO | __GFP_MEMALLOC;
    sock->sk->sk_use_task_frag = false;

For MPTCP, sock->sk is the msk. The data travels on TCP subflows created
by mptcp_subflow_create_socket()->sock_create_kern().

sync_socket_options() in net/mptcp/sockopt.c copies SOCK_KEEPOPEN,
SOCK_LINGER, SOCK_DBG, priority, mark and similar settings to the
subflow. It does not copy SOCK_MEMALLOC or sk_allocation, and nothing
under net/mptcp/ seems to refer to SOCK_MEMALLOC.

On receive, tcp_v4_rcv()->tcp_filter()->sk_filter_trim_cap() checks
the subflow, not the msk:

net/core/filter.c:sk_filter_trim_cap() {
    ...
	if (skb_pfmemalloc(skb) && !sock_flag(sk, SOCK_MEMALLOC)) {
		NET_INC_STATS(sock_net(sk), LINUX_MIB_PFMEMALLOCDROP);
		return SKB_DROP_REASON_PFMEMALLOC;
	}
    ...
}

Because the msk is marked, sk_memalloc_socks() is enabled, and drivers
can pass up pfmemalloc RX skbs under memory pressure. Wouldn't every
NBD reply, and every retransmission, that arrives in such an skb be
dropped at the subflow?

Transmit allocations on the subflow (ACKs, SYNs, retransmit clones via
sk_gfp_mask(ssk, ...)) also keep the default GFP_KERNEL. Subflows that
the path manager adds later through __mptcp_subflow_connect() start
with the same defaults.

With swap or dirty writeback on an NBD-over-MPTCP device, can this
stall or livelock reclaim? Preventing that for TCP is the reason
sk_set_memalloc() is called. The commit message does not mention this
limitation.

[Severity: Medium]
Does nbd_reclassify_socket() still cover the socket locks NBD takes
when the socket is MPTCP?

nbd_reclassify_socket() only reclassifies sock->sk:

drivers/block/nbd.c:nbd_reclassify_socket() {
	struct sock *sk = sock->sk;
    ...
	case AF_INET:
		sock_lock_init_class_and_name(sk, "slock-AF_INET-NBD",
    ...
}

This helper came from commit d532cddb6c60 ("nbd: Reclassify sockets to
avoid lockdep circular dependency"). For MPTCP, only the msk lock moves
into the NBD class.

Every NBD send also takes subflow locks through
mptcp_sendmsg()->__mptcp_push_pending()->lock_sock(ssk). Subflows are
kernel sockets from sock_create_kern(), so they share the
k-sk_lock-AF_INET/AF_INET6 class with every other MPTCP subflow on the
system.

Ordinary MPTCP user sockets allocate with GFP_KERNEL while holding a
lock of that class. The path is
__subflow_push_pending()->mptcp_sendmsg_frag()->mptcp_alloc_tx_skb():

    gfp_t gfp = data_lock_held ? GFP_ATOMIC : sk->sk_allocation;

That records a k-sk_lock -> fs_reclaim dependency. Meanwhile, NBD I/O
issued under the block layer's reclaim ordering takes the same class.

Could this bring back the fs_reclaim/sk_lock circular dependency report
that d532cddb6c60 suppressed, this time through the subflow lock class?
Nothing seems to reclassify subflows, including the ones the path
manager creates later.

[Severity: High]
Can accepting MPTCP sockets lead to a deadlock between reclaim and the
msk socket lock?

mptcp_worker() runs on the system workqueue and holds the msk lock
while it runs the path manager and closes subflows:

net/mptcp/protocol.c:mptcp_worker() {
    ...
	lock_sock(sk);
    ...
	mptcp_pm_worker(msk);
    ...
	if (test_and_clear_bit(MPTCP_WORK_CLOSE_SUBFLOW, &msk->flags))
		__mptcp_close_subflow(sk);
    ...
}

>From there, __mptcp_subflow_connect() calls
mptcp_subflow_create_socket()->sock_create_kern() and
subflow_ulp_init()->subflow_create_ctx(sk, GFP_KERNEL). The subflow
close path also calls mptcp_close_ssk()->mptcp_event(..., GFP_KERNEL).
None of this runs under a memalloc_noio scope.

If direct reclaim in that worker waits on swap or writeback to the NBD
device, the NBD I/O needs the same msk lock:

    nbd_queue_rq()->sock_xmit()->mptcp_sendmsg()->lock_sock(msk)
    recv_work()->sock_xmit()->mptcp_recvmsg()->lock_sock(msk)

Wouldn't the worker and NBD then wait on each other?

There is also a self-deadlock case. If the worker's own reclaim flushes
a plug into the BLK_MQ_F_BLOCKING nbd queue from the worker's own
context, lock_sock() would wait on an msk that the same task already
holds.

The memalloc_noreclaim_save() call and the sk_allocation setting in
__sock_xmit() only cover the NBD task and allocations that use
sk_allocation. They do not cover explicit GFP_KERNEL allocations in the
MPTCP worker. With plain TCP, no in-kernel process context sleeps in
reclaim while holding the NBD socket lock.

-- 
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/cover.1792746800.git.tanggeliang%40kylinos.cn