[PATCH mptcp-next v19 0/9] BPF redundant scheduler, part 1

[PATCH mptcp-next v19 0/9] BPF redundant scheduler, part 1

[Geliang Tang]

v19:
 - update patch 1.
 - split "BPF redundant scheduler" into two parts.
 - rebased on "export/20221111T055536".

Geliang Tang (9):
  mptcp: refactor push_pending logic
  mptcp: drop last_snd and MPTCP_RESET_SCHEDULER
  mptcp: add sched_data_set_contexts helper
  Squash to "mptcp: add struct mptcp_sched_ops"
  Squash to "bpf: Add bpf_mptcp_sched_ops"
  Squash to "bpf: Add bpf_mptcp_sched_kfunc_set"
  Squash to "selftests/bpf: Add bpf_first scheduler"
  Squash to "selftests/bpf: Add bpf_bkup scheduler"
  Squash to "selftests/bpf: Add bpf_rr scheduler"

 include/net/mptcp.h                           |   6 +-
 net/mptcp/bpf.c                               |   1 +
 net/mptcp/pm.c                                |   9 +-
 net/mptcp/pm_netlink.c                        |   3 -
 net/mptcp/protocol.c                          | 161 +++++++++---------
 net/mptcp/protocol.h                          |   1 -
 net/mptcp/sched.c                             |  21 ++-
 tools/testing/selftests/bpf/bpf_tcp_helpers.h |   8 +-
 .../selftests/bpf/progs/mptcp_bpf_bkup.c      |  10 +-
 .../selftests/bpf/progs/mptcp_bpf_first.c     |  10 +-
 .../selftests/bpf/progs/mptcp_bpf_rr.c        |  10 +-
 11 files changed, 143 insertions(+), 97 deletions(-)

-- 
2.35.3

[PATCH mptcp-next v19 1/9] mptcp: refactor push_pending logic

[Geliang Tang]

To support redundant package schedulers more easily, this patch refactors
__mptcp_push_pending() logic from:

For each dfrag:
	While sends succeed:
		Call the scheduler (selects subflow and msk->snd_burst)
		Update subflow locks (push/release/acquire as needed)
		Send the dfrag data with mptcp_sendmsg_frag()
		Update already_sent, snd_nxt, snd_burst
	Update msk->first_pending
Push/release on final subflow

->

While first_pending isn't empty:
	Call the scheduler (selects subflow and msk->snd_burst)
	Update subflow locks (push/release/acquire as needed)
	For each pending dfrag:
		While sends succeed:
			Send the dfrag data with mptcp_sendmsg_frag()
			Update already_sent, snd_nxt, snd_burst
		Update msk->first_pending
		Break if required by msk->snd_burst / etc
	Push/release on final subflow

Refactors __mptcp_subflow_push_pending logic from:

For each dfrag:
	While sends succeed:
		Call the scheduler (selects subflow and msk->snd_burst)
		Send the dfrag data with mptcp_subflow_delegate(), break
		Send the dfrag data with mptcp_sendmsg_frag()
		Update dfrag->already_sent, msk->snd_nxt, msk->snd_burst
	Update msk->first_pending

->

While first_pending isn't empty:
	Call the scheduler (selects subflow and msk->snd_burst)
	Send the dfrag data with mptcp_subflow_delegate(), break
	Send the dfrag data with mptcp_sendmsg_frag()
	For each pending dfrag:
		While sends succeed:
			Send the dfrag data with mptcp_sendmsg_frag()
			Update already_sent, snd_nxt, snd_burst
		Update msk->first_pending
		Break if required by msk->snd_burst / etc

Move the duplicate code from __mptcp_push_pending() and
__mptcp_subflow_push_pending() into a new helper function, named
__subflow_push_pending(). Simplify __mptcp_push_pending() and
__mptcp_subflow_push_pending() by invoking this helper.

Also move the burst check conditions out of the function
mptcp_subflow_get_send(), check them in __subflow_push_pending() in
the inner "for each pending dfrag" loop.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 net/mptcp/protocol.c | 154 +++++++++++++++++++++++--------------------
 1 file changed, 83 insertions(+), 71 deletions(-)

diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 1edb8a03d7fa..99faf7c1a548 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -1417,14 +1417,6 @@ static struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 		       sk_stream_memory_free(msk->first) ? msk->first : NULL;
 	}
 
-	/* re-use last subflow, if the burst allow that */
-	if (msk->last_snd && msk->snd_burst > 0 &&
-	    sk_stream_memory_free(msk->last_snd) &&
-	    mptcp_subflow_active(mptcp_subflow_ctx(msk->last_snd))) {
-		mptcp_set_timeout(sk);
-		return msk->last_snd;
-	}
-
 	/* pick the subflow with the lower wmem/wspace ratio */
 	for (i = 0; i < SSK_MODE_MAX; ++i) {
 		send_info[i].ssk = NULL;
@@ -1528,57 +1520,86 @@ void mptcp_check_and_set_pending(struct sock *sk)
 		mptcp_sk(sk)->push_pending |= BIT(MPTCP_PUSH_PENDING);
 }
 
-void __mptcp_push_pending(struct sock *sk, unsigned int flags)
+static int __subflow_push_pending(struct sock *sk, struct sock *ssk,
+				  struct mptcp_sendmsg_info *info)
 {
-	struct sock *prev_ssk = NULL, *ssk = NULL;
 	struct mptcp_sock *msk = mptcp_sk(sk);
-	struct mptcp_sendmsg_info info = {
-				.flags = flags,
-	};
-	bool do_check_data_fin = false;
 	struct mptcp_data_frag *dfrag;
-	int len;
+	int len, copied = 0, err = 0;
 
 	while ((dfrag = mptcp_send_head(sk))) {
-		info.sent = dfrag->already_sent;
-		info.limit = dfrag->data_len;
+		info->sent = dfrag->already_sent;
+		info->limit = dfrag->data_len;
 		len = dfrag->data_len - dfrag->already_sent;
 		while (len > 0) {
 			int ret = 0;
 
-			prev_ssk = ssk;
-			ssk = mptcp_subflow_get_send(msk);
-
-			/* First check. If the ssk has changed since
-			 * the last round, release prev_ssk
-			 */
-			if (ssk != prev_ssk && prev_ssk)
-				mptcp_push_release(prev_ssk, &info);
-			if (!ssk)
-				goto out;
-
-			/* Need to lock the new subflow only if different
-			 * from the previous one, otherwise we are still
-			 * helding the relevant lock
-			 */
-			if (ssk != prev_ssk)
-				lock_sock(ssk);
-
-			ret = mptcp_sendmsg_frag(sk, ssk, dfrag, &info);
+			ret = mptcp_sendmsg_frag(sk, ssk, dfrag, info);
 			if (ret <= 0) {
-				if (ret == -EAGAIN)
-					continue;
-				mptcp_push_release(ssk, &info);
+				err = copied ? : ret;
 				goto out;
 			}
 
-			do_check_data_fin = true;
-			info.sent += ret;
+			info->sent += ret;
+			copied += ret;
 			len -= ret;
 
 			mptcp_update_post_push(msk, dfrag, ret);
 		}
 		WRITE_ONCE(msk->first_pending, mptcp_send_next(sk));
+
+		if (msk->snd_burst <= 0 ||
+		    !sk_stream_memory_free(ssk) ||
+		    !mptcp_subflow_active(mptcp_subflow_ctx(ssk))) {
+			err = copied ? : -EAGAIN;
+			goto out;
+		}
+		mptcp_set_timeout(sk);
+	}
+	err = copied;
+
+out:
+	return err;
+}
+
+void __mptcp_push_pending(struct sock *sk, unsigned int flags)
+{
+	struct sock *prev_ssk = NULL, *ssk = NULL;
+	struct mptcp_sock *msk = mptcp_sk(sk);
+	struct mptcp_sendmsg_info info = {
+				.flags = flags,
+	};
+	bool do_check_data_fin = false;
+
+	while (mptcp_send_head(sk)) {
+		int ret = 0;
+
+		prev_ssk = ssk;
+		ssk = mptcp_subflow_get_send(msk);
+
+		/* First check. If the ssk has changed since
+		 * the last round, release prev_ssk
+		 */
+		if (ssk != prev_ssk && prev_ssk)
+			mptcp_push_release(prev_ssk, &info);
+		if (!ssk)
+			goto out;
+
+		/* Need to lock the new subflow only if different
+		 * from the previous one, otherwise we are still
+		 * helding the relevant lock
+		 */
+		if (ssk != prev_ssk)
+			lock_sock(ssk);
+
+		ret = __subflow_push_pending(sk, ssk, &info);
+		if (ret <= 0) {
+			if (ret == -EAGAIN)
+				continue;
+			mptcp_push_release(ssk, &info);
+			goto out;
+		}
+		do_check_data_fin = true;
 	}
 
 	/* at this point we held the socket lock for the last subflow we used */
@@ -1599,42 +1620,33 @@ static void __mptcp_subflow_push_pending(struct sock *sk, struct sock *ssk, bool
 	struct mptcp_sendmsg_info info = {
 		.data_lock_held = true,
 	};
-	struct mptcp_data_frag *dfrag;
 	struct sock *xmit_ssk;
-	int len, copied = 0;
+	int copied = 0;
 
 	info.flags = 0;
-	while ((dfrag = mptcp_send_head(sk))) {
-		info.sent = dfrag->already_sent;
-		info.limit = dfrag->data_len;
-		len = dfrag->data_len - dfrag->already_sent;
-		while (len > 0) {
-			int ret = 0;
+	while (mptcp_send_head(sk)) {
+		int ret = 0;
 
-			/* check for a different subflow usage only after
-			 * spooling the first chunk of data
-			 */
-			xmit_ssk = first ? ssk : mptcp_subflow_get_send(msk);
-			if (!xmit_ssk)
-				goto out;
-			if (xmit_ssk != ssk) {
-				mptcp_subflow_delegate(mptcp_subflow_ctx(xmit_ssk),
-						       MPTCP_DELEGATE_SEND);
-				goto out;
-			}
-
-			ret = mptcp_sendmsg_frag(sk, ssk, dfrag, &info);
-			if (ret <= 0)
-				goto out;
-
-			info.sent += ret;
-			copied += ret;
-			len -= ret;
-			first = false;
+		/* check for a different subflow usage only after
+		 * spooling the first chunk of data
+		 */
+		xmit_ssk = first ? ssk : mptcp_subflow_get_send(msk);
+		if (!xmit_ssk)
+			goto out;
+		if (xmit_ssk != ssk) {
+			mptcp_subflow_delegate(mptcp_subflow_ctx(xmit_ssk),
+					       MPTCP_DELEGATE_SEND);
+			goto out;
+		}
 
-			mptcp_update_post_push(msk, dfrag, ret);
+		ret = __subflow_push_pending(sk, ssk, &info);
+		first = false;
+		if (ret <= 0) {
+			if (ret == -EAGAIN)
+				continue;
+			break;
 		}
-		WRITE_ONCE(msk->first_pending, mptcp_send_next(sk));
+		copied += ret;
 	}
 
 out:
-- 
2.35.3

[PATCH mptcp-next v19 2/9] mptcp: drop last_snd and MPTCP_RESET_SCHEDULER

[Geliang Tang]

Since the burst check conditions have moved out of the function
mptcp_subflow_get_send(), it makes some msk->last_snd useless.
This patch drops them as well as the macro MPTCP_RESET_SCHEDULER.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 net/mptcp/pm.c         | 9 +--------
 net/mptcp/pm_netlink.c | 3 ---
 net/mptcp/protocol.c   | 7 +------
 net/mptcp/protocol.h   | 1 -
 4 files changed, 2 insertions(+), 18 deletions(-)

diff --git a/net/mptcp/pm.c b/net/mptcp/pm.c
index 45e2a48397b9..cdeb7280ac76 100644
--- a/net/mptcp/pm.c
+++ b/net/mptcp/pm.c
@@ -282,15 +282,8 @@ void mptcp_pm_mp_prio_received(struct sock *ssk, u8 bkup)
 
 	pr_debug("subflow->backup=%d, bkup=%d\n", subflow->backup, bkup);
 	msk = mptcp_sk(sk);
-	if (subflow->backup != bkup) {
+	if (subflow->backup != bkup)
 		subflow->backup = bkup;
-		mptcp_data_lock(sk);
-		if (!sock_owned_by_user(sk))
-			msk->last_snd = NULL;
-		else
-			__set_bit(MPTCP_RESET_SCHEDULER,  &msk->cb_flags);
-		mptcp_data_unlock(sk);
-	}
 
 	mptcp_event(MPTCP_EVENT_SUB_PRIORITY, msk, ssk, GFP_ATOMIC);
 }
diff --git a/net/mptcp/pm_netlink.c b/net/mptcp/pm_netlink.c
index d66fbd558263..08806f97c8fb 100644
--- a/net/mptcp/pm_netlink.c
+++ b/net/mptcp/pm_netlink.c
@@ -475,9 +475,6 @@ static void __mptcp_pm_send_ack(struct mptcp_sock *msk, struct mptcp_subflow_con
 
 	slow = lock_sock_fast(ssk);
 	if (prio) {
-		if (subflow->backup != backup)
-			msk->last_snd = NULL;
-
 		subflow->send_mp_prio = 1;
 		subflow->backup = backup;
 		subflow->request_bkup = backup;
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 99faf7c1a548..9e5b3cf27a45 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -1469,16 +1469,13 @@ static struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk)
 
 	burst = min_t(int, MPTCP_SEND_BURST_SIZE, mptcp_wnd_end(msk) - msk->snd_nxt);
 	wmem = READ_ONCE(ssk->sk_wmem_queued);
-	if (!burst) {
-		msk->last_snd = NULL;
+	if (!burst)
 		return ssk;
-	}
 
 	subflow = mptcp_subflow_ctx(ssk);
 	subflow->avg_pacing_rate = div_u64((u64)subflow->avg_pacing_rate * wmem +
 					   READ_ONCE(ssk->sk_pacing_rate) * burst,
 					   burst + wmem);
-	msk->last_snd = ssk;
 	msk->snd_burst = burst;
 	return ssk;
 }
@@ -3284,8 +3281,6 @@ static void mptcp_release_cb(struct sock *sk)
 			__mptcp_set_connected(sk);
 		if (__test_and_clear_bit(MPTCP_ERROR_REPORT, &msk->cb_flags))
 			__mptcp_error_report(sk);
-		if (__test_and_clear_bit(MPTCP_RESET_SCHEDULER, &msk->cb_flags))
-			msk->last_snd = NULL;
 	}
 
 	__mptcp_update_rmem(sk);
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index f805036e3c93..4780fb5a8087 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -124,7 +124,6 @@
 #define MPTCP_RETRANSMIT	4
 #define MPTCP_FLUSH_JOIN_LIST	5
 #define MPTCP_CONNECTED		6
-#define MPTCP_RESET_SCHEDULER	7
 
 struct mptcp_skb_cb {
 	u64 map_seq;
-- 
2.35.3

[PATCH mptcp-next v19 3/9] mptcp: add sched_data_set_contexts helper

[Geliang Tang]

Add a new helper mptcp_sched_data_set_contexts() to set the subflow
pointers array in struct mptcp_sched_data. It will be invoked by the
BPF schedulers to export the subflow pointers to the BPF contexts.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 net/mptcp/sched.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/net/mptcp/sched.c b/net/mptcp/sched.c
index d295b92a5789..5a910da1452b 100644
--- a/net/mptcp/sched.c
+++ b/net/mptcp/sched.c
@@ -93,3 +93,22 @@ void mptcp_subflow_set_scheduled(struct mptcp_subflow_context *subflow,
 {
 	WRITE_ONCE(subflow->scheduled, scheduled);
 }
+
+void mptcp_sched_data_set_contexts(const struct mptcp_sock *msk,
+				   struct mptcp_sched_data *data)
+{
+	struct mptcp_subflow_context *subflow;
+	int i = 0;
+
+	mptcp_for_each_subflow(msk, subflow) {
+		if (i == MPTCP_SUBFLOWS_MAX) {
+			pr_warn_once("too many subflows");
+			break;
+		}
+		mptcp_subflow_set_scheduled(subflow, false);
+		data->contexts[i++] = subflow;
+	}
+
+	for (; i < MPTCP_SUBFLOWS_MAX; i++)
+		data->contexts[i] = NULL;
+}
-- 
2.35.3

[PATCH mptcp-next v19 4/9] Squash to "mptcp: add struct mptcp_sched_ops"

[Geliang Tang]

New api:
 - add new data_init
 - add an int return value for get_subflow

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 include/net/mptcp.h | 6 ++++--
 net/mptcp/sched.c   | 2 +-
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/include/net/mptcp.h b/include/net/mptcp.h
index c25939b2af68..0f386d805957 100644
--- a/include/net/mptcp.h
+++ b/include/net/mptcp.h
@@ -105,8 +105,10 @@ struct mptcp_sched_data {
 };
 
 struct mptcp_sched_ops {
-	void (*get_subflow)(const struct mptcp_sock *msk,
-			    struct mptcp_sched_data *data);
+	void (*data_init)(const struct mptcp_sock *msk,
+			  struct mptcp_sched_data *data);
+	int (*get_subflow)(const struct mptcp_sock *msk,
+			   struct mptcp_sched_data *data);
 
 	char			name[MPTCP_SCHED_NAME_MAX];
 	struct module		*owner;
diff --git a/net/mptcp/sched.c b/net/mptcp/sched.c
index 5a910da1452b..0d7c73e9562e 100644
--- a/net/mptcp/sched.c
+++ b/net/mptcp/sched.c
@@ -33,7 +33,7 @@ struct mptcp_sched_ops *mptcp_sched_find(const char *name)
 
 int mptcp_register_scheduler(struct mptcp_sched_ops *sched)
 {
-	if (!sched->get_subflow)
+	if (!sched->data_init || !sched->get_subflow)
 		return -EINVAL;
 
 	spin_lock(&mptcp_sched_list_lock);
-- 
2.35.3

[PATCH mptcp-next v19 5/9] Squash to "bpf: Add bpf_mptcp_sched_ops"

[Geliang Tang]

Use new API.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 tools/testing/selftests/bpf/bpf_tcp_helpers.h | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/bpf/bpf_tcp_helpers.h b/tools/testing/selftests/bpf/bpf_tcp_helpers.h
index c7d4a9a69cfc..72c618037386 100644
--- a/tools/testing/selftests/bpf/bpf_tcp_helpers.h
+++ b/tools/testing/selftests/bpf/bpf_tcp_helpers.h
@@ -249,8 +249,10 @@ struct mptcp_sched_ops {
 	void (*init)(const struct mptcp_sock *msk);
 	void (*release)(const struct mptcp_sock *msk);
 
-	void (*get_subflow)(const struct mptcp_sock *msk,
-			    struct mptcp_sched_data *data);
+	void (*data_init)(const struct mptcp_sock *msk,
+			  struct mptcp_sched_data *data);
+	int (*get_subflow)(const struct mptcp_sock *msk,
+			   struct mptcp_sched_data *data);
 	void *owner;
 };
 
@@ -265,5 +267,7 @@ struct mptcp_sock {
 
 extern void mptcp_subflow_set_scheduled(struct mptcp_subflow_context *subflow,
 					bool scheduled) __ksym;
+extern void mptcp_sched_data_set_contexts(const struct mptcp_sock *msk,
+					  struct mptcp_sched_data *data) __ksym;
 
 #endif
-- 
2.35.3

[PATCH mptcp-next v19 6/9] Squash to "bpf: Add bpf_mptcp_sched_kfunc_set"

[Geliang Tang]

Add mptcp_sched_data_set_contexts in kfunc_set.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 net/mptcp/bpf.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/net/mptcp/bpf.c b/net/mptcp/bpf.c
index 0a768898990f..03decb05755f 100644
--- a/net/mptcp/bpf.c
+++ b/net/mptcp/bpf.c
@@ -164,6 +164,7 @@ struct bpf_struct_ops bpf_mptcp_sched_ops = {
 
 BTF_SET8_START(bpf_mptcp_sched_kfunc_ids)
 BTF_ID_FLAGS(func, mptcp_subflow_set_scheduled)
+BTF_ID_FLAGS(func, mptcp_sched_data_set_contexts)
 BTF_SET8_END(bpf_mptcp_sched_kfunc_ids)
 
 static const struct btf_kfunc_id_set bpf_mptcp_sched_kfunc_set = {
-- 
2.35.3

[PATCH mptcp-next v19 7/9] Squash to "selftests/bpf: Add bpf_first scheduler"

[Geliang Tang]

Use new API.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 tools/testing/selftests/bpf/progs/mptcp_bpf_first.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/bpf/progs/mptcp_bpf_first.c b/tools/testing/selftests/bpf/progs/mptcp_bpf_first.c
index fcd733e88b02..e4caa2dd8c6f 100644
--- a/tools/testing/selftests/bpf/progs/mptcp_bpf_first.c
+++ b/tools/testing/selftests/bpf/progs/mptcp_bpf_first.c
@@ -16,16 +16,24 @@ void BPF_PROG(mptcp_sched_first_release, const struct mptcp_sock *msk)
 {
 }
 
-void BPF_STRUCT_OPS(bpf_first_get_subflow, const struct mptcp_sock *msk,
+void BPF_STRUCT_OPS(bpf_first_data_init, const struct mptcp_sock *msk,
 		    struct mptcp_sched_data *data)
+{
+	mptcp_sched_data_set_contexts(msk, data);
+}
+
+int BPF_STRUCT_OPS(bpf_first_get_subflow, const struct mptcp_sock *msk,
+		   struct mptcp_sched_data *data)
 {
 	mptcp_subflow_set_scheduled(data->contexts[0], true);
+	return 0;
 }
 
 SEC(".struct_ops")
 struct mptcp_sched_ops first = {
 	.init		= (void *)mptcp_sched_first_init,
 	.release	= (void *)mptcp_sched_first_release,
+	.data_init	= (void *)bpf_first_data_init,
 	.get_subflow	= (void *)bpf_first_get_subflow,
 	.name		= "bpf_first",
 };
-- 
2.35.3

[PATCH mptcp-next v19 8/9] Squash to "selftests/bpf: Add bpf_bkup scheduler"

[Geliang Tang]

Use new API.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 tools/testing/selftests/bpf/progs/mptcp_bpf_bkup.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/bpf/progs/mptcp_bpf_bkup.c b/tools/testing/selftests/bpf/progs/mptcp_bpf_bkup.c
index 949e053e980c..b2724426676e 100644
--- a/tools/testing/selftests/bpf/progs/mptcp_bpf_bkup.c
+++ b/tools/testing/selftests/bpf/progs/mptcp_bpf_bkup.c
@@ -16,8 +16,14 @@ void BPF_PROG(mptcp_sched_bkup_release, const struct mptcp_sock *msk)
 {
 }
 
-void BPF_STRUCT_OPS(bpf_bkup_get_subflow, const struct mptcp_sock *msk,
+void BPF_STRUCT_OPS(bpf_bkup_data_init, const struct mptcp_sock *msk,
 		    struct mptcp_sched_data *data)
+{
+	mptcp_sched_data_set_contexts(msk, data);
+}
+
+int BPF_STRUCT_OPS(bpf_bkup_get_subflow, const struct mptcp_sock *msk,
+		   struct mptcp_sched_data *data)
 {
 	int nr = 0;
 
@@ -32,12 +38,14 @@ void BPF_STRUCT_OPS(bpf_bkup_get_subflow, const struct mptcp_sock *msk,
 	}
 
 	mptcp_subflow_set_scheduled(data->contexts[nr], true);
+	return 0;
 }
 
 SEC(".struct_ops")
 struct mptcp_sched_ops bkup = {
 	.init		= (void *)mptcp_sched_bkup_init,
 	.release	= (void *)mptcp_sched_bkup_release,
+	.data_init	= (void *)bpf_bkup_data_init,
 	.get_subflow	= (void *)bpf_bkup_get_subflow,
 	.name		= "bpf_bkup",
 };
-- 
2.35.3

[PATCH mptcp-next v19 9/9] Squash to "selftests/bpf: Add bpf_rr scheduler"

[Geliang Tang]

Use new API.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 tools/testing/selftests/bpf/progs/mptcp_bpf_rr.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/bpf/progs/mptcp_bpf_rr.c b/tools/testing/selftests/bpf/progs/mptcp_bpf_rr.c
index ce4e98f83e43..e101428e5906 100644
--- a/tools/testing/selftests/bpf/progs/mptcp_bpf_rr.c
+++ b/tools/testing/selftests/bpf/progs/mptcp_bpf_rr.c
@@ -16,8 +16,14 @@ void BPF_PROG(mptcp_sched_rr_release, const struct mptcp_sock *msk)
 {
 }
 
-void BPF_STRUCT_OPS(bpf_rr_get_subflow, const struct mptcp_sock *msk,
+void BPF_STRUCT_OPS(bpf_rr_data_init, const struct mptcp_sock *msk,
 		    struct mptcp_sched_data *data)
+{
+	mptcp_sched_data_set_contexts(msk, data);
+}
+
+int BPF_STRUCT_OPS(bpf_rr_get_subflow, const struct mptcp_sock *msk,
+		   struct mptcp_sched_data *data)
 {
 	int nr = 0;
 
@@ -35,12 +41,14 @@ void BPF_STRUCT_OPS(bpf_rr_get_subflow, const struct mptcp_sock *msk,
 	}
 
 	mptcp_subflow_set_scheduled(data->contexts[nr], true);
+	return 0;
 }
 
 SEC(".struct_ops")
 struct mptcp_sched_ops rr = {
 	.init		= (void *)mptcp_sched_rr_init,
 	.release	= (void *)mptcp_sched_rr_release,
+	.data_init	= (void *)bpf_rr_data_init,
 	.get_subflow	= (void *)bpf_rr_get_subflow,
 	.name		= "bpf_rr",
 };
-- 
2.35.3

Re: Squash to "selftests/bpf: Add bpf_rr scheduler": Build Failure

[MPTCP CI]

Hi Geliang,

Thank you for your modifications, that's great!

But sadly, our CI spotted some issues with it when trying to build it.

You can find more details there:

  https://patchwork.kernel.org/project/mptcp/patch/d1d8d0acb444c6ac39eb6ae945bf439d633e74b3.1668151131.git.geliang.tang@suse.com/
  https://github.com/multipath-tcp/mptcp_net-next/actions/runs/3443397477

Status: failure
Initiator: MPTCPimporter
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/b52f03a91914

Feel free to reply to this email if you cannot access logs, if you need
some support to fix the error, if this doesn't seem to be caused by your
modifications or if the error is a false positive one.

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: Squash to "selftests/bpf: Add bpf_rr scheduler": Tests Results

[MPTCP CI]

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal:
  - Unstable: 1 failed test(s): mptcp_connect_mmap 🔴:
  - Task: https://cirrus-ci.com/task/5566491222343680
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5566491222343680/summary/summary.txt

- KVM Validation: debug:
  - Unstable: 2 failed test(s): mptcp_connect_mmap packetdrill_add_addr - Critical: Global Timeout ❌:
  - Task: https://cirrus-ci.com/task/6692391129186304
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/6692391129186304/summary/summary.txt

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/b52f03a91914


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: [PATCH mptcp-next v19 0/9] BPF redundant scheduler, part 1

[Mat Martineau]

On Fri, 11 Nov 2022, Geliang Tang wrote:

> v19:
> - update patch 1.
> - split "BPF redundant scheduler" into two parts.
> - rebased on "export/20221111T055536".
>

Thanks Geliang.

I think v19 (part 1) looks ok for the export branch. simult_flows.sh was 
slightly faster (small sample size :) ) on my VM, for what it's worth.

Looks like the first three patches go at the beginning of the "features 
net-next-next" section, correct?

Reviewed-by: Mat Martineau <mathew.j.martineau@linux.intel.com>

> Geliang Tang (9):
>  mptcp: refactor push_pending logic
>  mptcp: drop last_snd and MPTCP_RESET_SCHEDULER
>  mptcp: add sched_data_set_contexts helper
>  Squash to "mptcp: add struct mptcp_sched_ops"
>  Squash to "bpf: Add bpf_mptcp_sched_ops"
>  Squash to "bpf: Add bpf_mptcp_sched_kfunc_set"
>  Squash to "selftests/bpf: Add bpf_first scheduler"
>  Squash to "selftests/bpf: Add bpf_bkup scheduler"
>  Squash to "selftests/bpf: Add bpf_rr scheduler"
>
> include/net/mptcp.h                           |   6 +-
> net/mptcp/bpf.c                               |   1 +
> net/mptcp/pm.c                                |   9 +-
> net/mptcp/pm_netlink.c                        |   3 -
> net/mptcp/protocol.c                          | 161 +++++++++---------
> net/mptcp/protocol.h                          |   1 -
> net/mptcp/sched.c                             |  21 ++-
> tools/testing/selftests/bpf/bpf_tcp_helpers.h |   8 +-
> .../selftests/bpf/progs/mptcp_bpf_bkup.c      |  10 +-
> .../selftests/bpf/progs/mptcp_bpf_first.c     |  10 +-
> .../selftests/bpf/progs/mptcp_bpf_rr.c        |  10 +-
> 11 files changed, 143 insertions(+), 97 deletions(-)
>
> -- 
> 2.35.3
>
>
>

--
Mat Martineau
Intel

Re: Squash to "selftests/bpf: Add bpf_rr scheduler": Build Failure

[MPTCP CI]

Hi Geliang,

Thank you for your modifications, that's great!

But sadly, our CI spotted some issues with it when trying to build it.

You can find more details there:

  https://patchwork.kernel.org/project/mptcp/patch/d1d8d0acb444c6ac39eb6ae945bf439d633e74b3.1668151131.git.geliang.tang@suse.com/
  https://github.com/multipath-tcp/mptcp_net-next/actions/runs/3475381466

Status: failure
Initiator: MPTCPimporter
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/4174181fc84f

Feel free to reply to this email if you cannot access logs, if you need
some support to fix the error, if this doesn't seem to be caused by your
modifications or if the error is a false positive one.

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: [PATCH mptcp-next v19 0/9] BPF redundant scheduler, part 1

[Geliang Tang]

On Tue, Nov 15, 2022 at 04:41:15PM -0800, Mat Martineau wrote:
> On Fri, 11 Nov 2022, Geliang Tang wrote:
> 
> > v19:
> > - update patch 1.
> > - split "BPF redundant scheduler" into two parts.
> > - rebased on "export/20221111T055536".
> > 
> 
> Thanks Geliang.
> 
> I think v19 (part 1) looks ok for the export branch. simult_flows.sh was
> slightly faster (small sample size :) ) on my VM, for what it's worth.
> 
> Looks like the first three patches go at the beginning of the "features
> net-next-next" section, correct?

Yes, the first two patches ("mptcp: refactor push_pending logic" and
"mptcp: drop last_snd and MPTCP_RESET_SCHEDULER") go at the beginning
of the "features net-next-next" section. But the third one "mptcp: add
sched_data_set_contexts helper" should be inserted between the commits
"mptcp: add scheduled in mptcp_subflow_context" and "bpf: Add
bpf_mptcp_sched_ops".

Thanks,
-Geliang

> 
> Reviewed-by: Mat Martineau <mathew.j.martineau@linux.intel.com>
> 
> > Geliang Tang (9):
> >  mptcp: refactor push_pending logic
> >  mptcp: drop last_snd and MPTCP_RESET_SCHEDULER
> >  mptcp: add sched_data_set_contexts helper
> >  Squash to "mptcp: add struct mptcp_sched_ops"
> >  Squash to "bpf: Add bpf_mptcp_sched_ops"
> >  Squash to "bpf: Add bpf_mptcp_sched_kfunc_set"
> >  Squash to "selftests/bpf: Add bpf_first scheduler"
> >  Squash to "selftests/bpf: Add bpf_bkup scheduler"
> >  Squash to "selftests/bpf: Add bpf_rr scheduler"
> > 
> > include/net/mptcp.h                           |   6 +-
> > net/mptcp/bpf.c                               |   1 +
> > net/mptcp/pm.c                                |   9 +-
> > net/mptcp/pm_netlink.c                        |   3 -
> > net/mptcp/protocol.c                          | 161 +++++++++---------
> > net/mptcp/protocol.h                          |   1 -
> > net/mptcp/sched.c                             |  21 ++-
> > tools/testing/selftests/bpf/bpf_tcp_helpers.h |   8 +-
> > .../selftests/bpf/progs/mptcp_bpf_bkup.c      |  10 +-
> > .../selftests/bpf/progs/mptcp_bpf_first.c     |  10 +-
> > .../selftests/bpf/progs/mptcp_bpf_rr.c        |  10 +-
> > 11 files changed, 143 insertions(+), 97 deletions(-)
> > 
> > -- 
> > 2.35.3
> > 
> > 
> > 
> 
> --
> Mat Martineau
> Intel

Re: Squash to "selftests/bpf: Add bpf_rr scheduler": Tests Results

[MPTCP CI]

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal:
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/5459335883522048
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5459335883522048/summary/summary.txt

- KVM Validation: debug:
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/6585235790364672
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/6585235790364672/summary/summary.txt

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/4174181fc84f


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: [PATCH mptcp-next v19 0/9] BPF redundant scheduler, part 1

[Matthieu Baerts]

Hi Geliang, Mat,

On 11/11/2022 08:47, Geliang Tang wrote:
> v19:
>  - update patch 1.
>  - split "BPF redundant scheduler" into two parts.
>  - rebased on "export/20221111T055536".
> 
> Geliang Tang (9):
>   mptcp: refactor push_pending logic
>   mptcp: drop last_snd and MPTCP_RESET_SCHEDULER
>   mptcp: add sched_data_set_contexts helper
>   Squash to "mptcp: add struct mptcp_sched_ops"
>   Squash to "bpf: Add bpf_mptcp_sched_ops"
>   Squash to "bpf: Add bpf_mptcp_sched_kfunc_set"
>   Squash to "selftests/bpf: Add bpf_first scheduler"
>   Squash to "selftests/bpf: Add bpf_bkup scheduler"
>   Squash to "selftests/bpf: Add bpf_rr scheduler"

Thank you for the patches and the reviews!

I just applied them (+ an additional squash-to patch) in our tree (feat.
for other trees (bpf)) with Mat's RvB tag.

Patches 1/9 and 2/9 have been added at the beginning. Patch 3/9 depends
on "mptcp: add struct mptcp_sched_ops" and it has been applied just
before "bpf: Add bpf_mptcp_sched_ops" as mentioned by Geliang.

Please also note that patch 5/9 has been squashed in "selftests/bpf: Add
mptcp sched structs" instead of "bpf: Add bpf_mptcp_sched_ops. Do not
hesitate to tell me if something is not OK!

New patches for t/upstream:
- 602fba0955d1: mptcp: refactor push_pending logic
- 343f8eed9abb: mptcp: drop last_snd and MPTCP_RESET_SCHEDULER
- Results: 69720c6cd168..ac857086fdb8 (export)

- 65e9504361f6: mptcp: add sched_data_set_contexts helper
- Results: ac857086fdb8..ae9c4d4cf58f (export)

- 82721c73112e: "squashed" patch 4/9 in "mptcp: add struct mptcp_sched_ops"
- a7c2367dd1d2: "squashed" patch 5/9 in "selftests/bpf: Add mptcp sched
structs"
- ee74ff707e4b: "squashed" patch 6/9 in "bpf: Add bpf_mptcp_sched_kfunc_set"
- 813537758eb5: "squashed" patch 7/9 in "selftests/bpf: Add bpf_first
scheduler"
- 762c5b194975: "squashed" patch 8/9 in "selftests/bpf: Add bpf_bkup
scheduler"
- 5db53c2d3ef6: "squashed" patch 9/9 in "selftests/bpf: Add bpf_rr
scheduler"
- Results: ae9c4d4cf58f..0814e9ff0a94 (export)

- 7b3058027867: "squashed" in "mptcp: refactor push_pending logic"
- Results: 0814e9ff0a94..d98411bcedd1 (export)

Tests are now in progress:

https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20221118T145003

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net

Re: [PATCH mptcp-next v19 0/9] BPF redundant scheduler, part 1

[Geliang Tang]

On Fri, Nov 18, 2022 at 03:52:38PM +0100, Matthieu Baerts wrote:
> Hi Geliang, Mat,
> 
> On 11/11/2022 08:47, Geliang Tang wrote:
> > v19:
> >  - update patch 1.
> >  - split "BPF redundant scheduler" into two parts.
> >  - rebased on "export/20221111T055536".
> > 
> > Geliang Tang (9):
> >   mptcp: refactor push_pending logic
> >   mptcp: drop last_snd and MPTCP_RESET_SCHEDULER
> >   mptcp: add sched_data_set_contexts helper
> >   Squash to "mptcp: add struct mptcp_sched_ops"
> >   Squash to "bpf: Add bpf_mptcp_sched_ops"
> >   Squash to "bpf: Add bpf_mptcp_sched_kfunc_set"
> >   Squash to "selftests/bpf: Add bpf_first scheduler"
> >   Squash to "selftests/bpf: Add bpf_bkup scheduler"
> >   Squash to "selftests/bpf: Add bpf_rr scheduler"
> 
> Thank you for the patches and the reviews!
> 
> I just applied them (+ an additional squash-to patch) in our tree (feat.
> for other trees (bpf)) with Mat's RvB tag.
> 
> Patches 1/9 and 2/9 have been added at the beginning. Patch 3/9 depends
> on "mptcp: add struct mptcp_sched_ops" and it has been applied just
> before "bpf: Add bpf_mptcp_sched_ops" as mentioned by Geliang.
> 
> Please also note that patch 5/9 has been squashed in "selftests/bpf: Add
> mptcp sched structs" instead of "bpf: Add bpf_mptcp_sched_ops. Do not
> hesitate to tell me if something is not OK!

Yes, it should be squashed in "selftests/bpf: Add mptcp sched structs".
Thanks for fix this.

-Geliang

> 
> New patches for t/upstream:
> - 602fba0955d1: mptcp: refactor push_pending logic
> - 343f8eed9abb: mptcp: drop last_snd and MPTCP_RESET_SCHEDULER
> - Results: 69720c6cd168..ac857086fdb8 (export)
> 
> - 65e9504361f6: mptcp: add sched_data_set_contexts helper
> - Results: ac857086fdb8..ae9c4d4cf58f (export)
> 
> - 82721c73112e: "squashed" patch 4/9 in "mptcp: add struct mptcp_sched_ops"
> - a7c2367dd1d2: "squashed" patch 5/9 in "selftests/bpf: Add mptcp sched
> structs"
> - ee74ff707e4b: "squashed" patch 6/9 in "bpf: Add bpf_mptcp_sched_kfunc_set"
> - 813537758eb5: "squashed" patch 7/9 in "selftests/bpf: Add bpf_first
> scheduler"
> - 762c5b194975: "squashed" patch 8/9 in "selftests/bpf: Add bpf_bkup
> scheduler"
> - 5db53c2d3ef6: "squashed" patch 9/9 in "selftests/bpf: Add bpf_rr
> scheduler"
> - Results: ae9c4d4cf58f..0814e9ff0a94 (export)
> 
> - 7b3058027867: "squashed" in "mptcp: refactor push_pending logic"
> - Results: 0814e9ff0a94..d98411bcedd1 (export)
> 
> Tests are now in progress:
> 
> https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20221118T145003
> 
> Cheers,
> Matt
> -- 
> Tessares | Belgium | Hybrid Access Solutions
> www.tessares.net

Re: [PATCH mptcp-next v19 3/9] mptcp: add sched_data_set_contexts helper

[Matthieu Baerts]

Hi Geliang, Mat,

On 11/11/2022 08:47, Geliang Tang wrote:
> Add a new helper mptcp_sched_data_set_contexts() to set the subflow
> pointers array in struct mptcp_sched_data. It will be invoked by the
> BPF schedulers to export the subflow pointers to the BPF contexts.

I have an issue with the CI and this commit: it adds a new helper which
is not used before before "selftests/bpf: Add mptcp sched structs" commit.

We then have this warning when compiling with W=1:

  $ make W=1 net/mptcp/sched.o
  net/mptcp/sched.c:97:6: warning: symbol
'mptcp_sched_data_set_contexts' was not declared. Should it be static?

I'm not sure what to do:
- modify the script to parse error and ignore it (I would prefer to
avoid case per case)
- move the commit after "selftests/bpf: Add mptcp sched structs" and
directly use it (extern + kfunc)
- squash it elsewhere?
- add a patch like that:

------------------- 8< -------------------
diff --git a/net/mptcp/sched.c b/net/mptcp/sched.c
index 0d7c73e9562e..e9b1ff932a3b 100644
--- a/net/mptcp/sched.c
+++ b/net/mptcp/sched.c
@@ -94,6 +94,8 @@ void mptcp_subflow_set_scheduled(struct
mptcp_subflow_context *subflow,
        WRITE_ONCE(subflow->scheduled, scheduled);
 }

+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wmissing-prototypes"
 void mptcp_sched_data_set_contexts(const struct mptcp_sock *msk,
                                   struct mptcp_sched_data *data)
 {
@@ -112,3 +114,4 @@ void mptcp_sched_data_set_contexts(const struct
mptcp_sock *msk,
        for (; i < MPTCP_SUBFLOWS_MAX; i++)
                data->contexts[i] = NULL;
 }
+#pragma GCC diagnostic pop
------------------- 8< -------------------

But well... :-)

I suggest to apply this patch for the moment to avoid errors with all
new patches and check later which solution to use.

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net

Re: [PATCH mptcp-next v19 3/9] mptcp: add sched_data_set_contexts helper

[Mat Martineau]

On Fri, 18 Nov 2022, Matthieu Baerts wrote:

> Hi Geliang, Mat,
>
> On 11/11/2022 08:47, Geliang Tang wrote:
>> Add a new helper mptcp_sched_data_set_contexts() to set the subflow
>> pointers array in struct mptcp_sched_data. It will be invoked by the
>> BPF schedulers to export the subflow pointers to the BPF contexts.
>
> I have an issue with the CI and this commit: it adds a new helper which
> is not used before before "selftests/bpf: Add mptcp sched structs" commit.
>
> We then have this warning when compiling with W=1:
>
>  $ make W=1 net/mptcp/sched.o
>  net/mptcp/sched.c:97:6: warning: symbol
> 'mptcp_sched_data_set_contexts' was not declared. Should it be static?
>
> I'm not sure what to do:
> - modify the script to parse error and ignore it (I would prefer to
> avoid case per case)
> - move the commit after "selftests/bpf: Add mptcp sched structs" and
> directly use it (extern + kfunc)
> - squash it elsewhere?
> - add a patch like that:
>
> ------------------- 8< -------------------
> diff --git a/net/mptcp/sched.c b/net/mptcp/sched.c
> index 0d7c73e9562e..e9b1ff932a3b 100644
> --- a/net/mptcp/sched.c
> +++ b/net/mptcp/sched.c
> @@ -94,6 +94,8 @@ void mptcp_subflow_set_scheduled(struct
> mptcp_subflow_context *subflow,
>        WRITE_ONCE(subflow->scheduled, scheduled);
> }
>
> +#pragma GCC diagnostic push
> +#pragma GCC diagnostic ignored "-Wmissing-prototypes"
> void mptcp_sched_data_set_contexts(const struct mptcp_sock *msk,
>                                   struct mptcp_sched_data *data)
> {
> @@ -112,3 +114,4 @@ void mptcp_sched_data_set_contexts(const struct
> mptcp_sock *msk,
>        for (; i < MPTCP_SUBFLOWS_MAX; i++)
>                data->contexts[i] = NULL;
> }
> +#pragma GCC diagnostic pop
> ------------------- 8< -------------------

One more option:

Add a declaration in net/mptcp/protocol.h


>
> But well... :-)
>
> I suggest to apply this patch for the moment to avoid errors with all
> new patches and check later which solution to use.

This is fine with me, if you'd rather wait for a squash-to patch.

--
Mat Martineau
Intel

Re: [PATCH mptcp-next v19 3/9] mptcp: add sched_data_set_contexts helper

[Geliang Tang]

On Fri, Nov 18, 2022 at 10:26:01AM -0800, Mat Martineau wrote:
> On Fri, 18 Nov 2022, Matthieu Baerts wrote:
> 
> > Hi Geliang, Mat,
> > 
> > On 11/11/2022 08:47, Geliang Tang wrote:
> > > Add a new helper mptcp_sched_data_set_contexts() to set the subflow
> > > pointers array in struct mptcp_sched_data. It will be invoked by the
> > > BPF schedulers to export the subflow pointers to the BPF contexts.
> > 
> > I have an issue with the CI and this commit: it adds a new helper which
> > is not used before before "selftests/bpf: Add mptcp sched structs" commit.
> > 
> > We then have this warning when compiling with W=1:
> > 
> >  $ make W=1 net/mptcp/sched.o
> >  net/mptcp/sched.c:97:6: warning: symbol
> > 'mptcp_sched_data_set_contexts' was not declared. Should it be static?
> > 
> > I'm not sure what to do:
> > - modify the script to parse error and ignore it (I would prefer to
> > avoid case per case)
> > - move the commit after "selftests/bpf: Add mptcp sched structs" and
> > directly use it (extern + kfunc)
> > - squash it elsewhere?
> > - add a patch like that:
> > 
> > ------------------- 8< -------------------
> > diff --git a/net/mptcp/sched.c b/net/mptcp/sched.c
> > index 0d7c73e9562e..e9b1ff932a3b 100644
> > --- a/net/mptcp/sched.c
> > +++ b/net/mptcp/sched.c
> > @@ -94,6 +94,8 @@ void mptcp_subflow_set_scheduled(struct
> > mptcp_subflow_context *subflow,
> >        WRITE_ONCE(subflow->scheduled, scheduled);
> > }
> > 
> > +#pragma GCC diagnostic push
> > +#pragma GCC diagnostic ignored "-Wmissing-prototypes"
> > void mptcp_sched_data_set_contexts(const struct mptcp_sock *msk,
> >                                   struct mptcp_sched_data *data)
> > {
> > @@ -112,3 +114,4 @@ void mptcp_sched_data_set_contexts(const struct
> > mptcp_sock *msk,
> >        for (; i < MPTCP_SUBFLOWS_MAX; i++)
> >                data->contexts[i] = NULL;
> > }
> > +#pragma GCC diagnostic pop
> > ------------------- 8< -------------------
> 
> One more option:
> 
> Add a declaration in net/mptcp/protocol.h
> 
> 
> > 
> > But well... :-)
> > 
> > I suggest to apply this patch for the moment to avoid errors with all
> > new patches and check later which solution to use.
> 
> This is fine with me, if you'd rather wait for a squash-to patch.

Hi Mat, Matt,

I just sent a squash-to patch to fix this.

Thanks,
-Geliang


> 
> --
> Mat Martineau
> Intel

Re: [PATCH mptcp-next v19 0/9] BPF redundant scheduler, part 1

[Geliang Tang]

On Fri, Nov 18, 2022 at 03:52:38PM +0100, Matthieu Baerts wrote:
> Hi Geliang, Mat,
> 
> On 11/11/2022 08:47, Geliang Tang wrote:
> > v19:
> >  - update patch 1.
> >  - split "BPF redundant scheduler" into two parts.
> >  - rebased on "export/20221111T055536".
> > 
> > Geliang Tang (9):
> >   mptcp: refactor push_pending logic
> >   mptcp: drop last_snd and MPTCP_RESET_SCHEDULER
> >   mptcp: add sched_data_set_contexts helper
> >   Squash to "mptcp: add struct mptcp_sched_ops"
> >   Squash to "bpf: Add bpf_mptcp_sched_ops"
> >   Squash to "bpf: Add bpf_mptcp_sched_kfunc_set"

Hi Matt,

The commit logs in these three patches become mismatched since the squashed
code, please update them for me:

'''
mptcp: add struct mptcp_sched_ops

This patch defines struct mptcp_sched_ops, which has three struct members,
name, owner and list, and four function pointers: init(), release(),
data_init() and get_subflow().

The scheduler functions data_init() and get_subflow() have a struct
mptcp_sched_data parameter, which contains a reinject flag and a
mptcp_subflow_context array.

Add the scheduler registering, unregistering and finding functions to add,
delete and find a packet scheduler on the global list mptcp_sched_list.


bpf: Add bpf_mptcp_sched_kfunc_set

This patch adds a new struct btf_kfunc_id_set for MPTCP scheduler. Add
mptcp_subflow_set_scheduled() and mptcp_sched_data_set_contexts() helpers
into this id_set, and register it in bpf_mptcp_sched_kfunc_init() to make
sure these helpers can be accessed from the BPF context.


selftests/bpf: Add mptcp sched structs

This patch adds three MPTCP scheduler structures: struct mptcp_sched_ops,
struct mptcp_sched_data and struct mptcp_subflow_context; and exports
mptcp_subflow_set_scheduled() and mptcp_sched_data_set_contexts() helpers
for bpf selftests.
'''

Thanks,
-Geliang

> >   Squash to "selftests/bpf: Add bpf_first scheduler"
> >   Squash to "selftests/bpf: Add bpf_bkup scheduler"
> >   Squash to "selftests/bpf: Add bpf_rr scheduler"
> 
> Thank you for the patches and the reviews!
> 
> I just applied them (+ an additional squash-to patch) in our tree (feat.
> for other trees (bpf)) with Mat's RvB tag.
> 
> Patches 1/9 and 2/9 have been added at the beginning. Patch 3/9 depends
> on "mptcp: add struct mptcp_sched_ops" and it has been applied just
> before "bpf: Add bpf_mptcp_sched_ops" as mentioned by Geliang.
> 
> Please also note that patch 5/9 has been squashed in "selftests/bpf: Add
> mptcp sched structs" instead of "bpf: Add bpf_mptcp_sched_ops. Do not
> hesitate to tell me if something is not OK!
> 
> New patches for t/upstream:
> - 602fba0955d1: mptcp: refactor push_pending logic
> - 343f8eed9abb: mptcp: drop last_snd and MPTCP_RESET_SCHEDULER
> - Results: 69720c6cd168..ac857086fdb8 (export)
> 
> - 65e9504361f6: mptcp: add sched_data_set_contexts helper
> - Results: ac857086fdb8..ae9c4d4cf58f (export)
> 
> - 82721c73112e: "squashed" patch 4/9 in "mptcp: add struct mptcp_sched_ops"
> - a7c2367dd1d2: "squashed" patch 5/9 in "selftests/bpf: Add mptcp sched
> structs"
> - ee74ff707e4b: "squashed" patch 6/9 in "bpf: Add bpf_mptcp_sched_kfunc_set"
> - 813537758eb5: "squashed" patch 7/9 in "selftests/bpf: Add bpf_first
> scheduler"
> - 762c5b194975: "squashed" patch 8/9 in "selftests/bpf: Add bpf_bkup
> scheduler"
> - 5db53c2d3ef6: "squashed" patch 9/9 in "selftests/bpf: Add bpf_rr
> scheduler"
> - Results: ae9c4d4cf58f..0814e9ff0a94 (export)
> 
> - 7b3058027867: "squashed" in "mptcp: refactor push_pending logic"
> - Results: 0814e9ff0a94..d98411bcedd1 (export)
> 
> Tests are now in progress:
> 
> https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20221118T145003
> 
> Cheers,
> Matt
> -- 
> Tessares | Belgium | Hybrid Access Solutions
> www.tessares.net

Re: [PATCH mptcp-next v19 0/9] BPF redundant scheduler, part 1

[Matthieu Baerts]

Hi Geliang,

On 09/12/2022 04:18, Geliang Tang wrote:
> On Fri, Nov 18, 2022 at 03:52:38PM +0100, Matthieu Baerts wrote:
>> Hi Geliang, Mat,
>>
>> On 11/11/2022 08:47, Geliang Tang wrote:
>>> v19:
>>>  - update patch 1.
>>>  - split "BPF redundant scheduler" into two parts.
>>>  - rebased on "export/20221111T055536".
>>>
>>> Geliang Tang (9):
>>>   mptcp: refactor push_pending logic
>>>   mptcp: drop last_snd and MPTCP_RESET_SCHEDULER
>>>   mptcp: add sched_data_set_contexts helper
>>>   Squash to "mptcp: add struct mptcp_sched_ops"
>>>   Squash to "bpf: Add bpf_mptcp_sched_ops"
>>>   Squash to "bpf: Add bpf_mptcp_sched_kfunc_set"
> 
> Hi Matt,
> 
> The commit logs in these three patches become mismatched since the squashed
> code, please update them for me:

Good catch!

Sure, just did:

New patches for t/upstream:
- 8685053d6d06: tg:msg: update as requested by Geliang
- b81ee295d042: tg:msg: update as requested by Geliang
- e399a35fdc24: tg:msg: update as requested by Geliang
- Results: 6d7665253eef..8ef79d8e4dfb (export)

Tests are now in progress:

https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20221214T173325

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net