[PATCH mptcp-next v10 00/13] BPF packet scheduler updates part 1

[PATCH mptcp-next v10 00/13] BPF packet scheduler updates part 1

[Geliang Tang]

v10:
 - add mptcp_storage map type.
 - update scheduler API.

v9:
 - merge 'Squash to "BPF packet scheduler updates v8"' and 'mptcp:
 make the contexts array of sched_data invisible in BPF'

v8:
 address Paolo's comments in v7:
 - add sched_data into mptcp_sock as new field, not an pointer.
 - drop rename __mptcp_set_timeout, export mptcp_set_timeout into BPF
   context instead.
 - add a comment in bpf_burst to explan why not use
   "ssk = send_info[SSK_MODE_ACTIVE].ssk;" directly.
 - move mptcp_subflow_tcp_sock into bpf_tcp_helpers.h.

v7:
 - drop more last_snd, only set it in bpf_rr.
 - 1-10: save sched_data at mptcp_sock, for issues #342
 - 11-16: add bpf_burst scheduler
 - 17-19: add bpf_stale scheduler, for issues #349

v6:
 - Only update patches 14, 15. Add a helper bpf_mptcp_subflow_ctx, fix
   bug in bpf_burst_get_retrans().

v5:
 - 1-9: save sched_data at mptcp_sock, for issues #342
 - 10-15: add bpf_burst scheduler
 - 16-18: add bpf_stale scheduler, for issues #349

v4:
 - drop "mptcp: add three helpers for bpf_burst", implement the helpers
   in the bpf context directly.

v3:
 - address Florian's comments in v2.
 - split into three more patches.

v2:
 - fix this error reported by CI:
KASAN: slab-use-after-free in __mptcp_close_ssk (net/mptcp/protocol.c:2461)
 - add bpf burst scheduler.

This patchset adds sched_data pointer into mptcp_sock to to save some
data at MPTCP and subflows levels.

With these changes, the old patch "mptcp: register default scheduler" in
[1] now works.

https://patchwork.kernel.org/project/mptcp/cover/cover.1665753926.git.geliang.tang@suse.com/ [1]
Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/342
Closes: https://github.com/multipath-tcp/mptcp_net-next/issues/349

Geliang Tang (13):
  Squash to "mptcp: drop last_snd and MPTCP_RESET_SCHEDULER"
  Squash to "mptcp: add struct mptcp_sched_ops"
  Squash to "mptcp: add sched_data_set_contexts helper"
  mptcp: register default scheduler
  Squash to "bpf: Add bpf_mptcp_sched_ops"
  Squash to "bpf: Add bpf_mptcp_sched_kfunc_set"
  bpf: Add mptcp_storage map type
  bpf: Add mptcp_storage helpers
  Squash to "selftests/bpf: Add mptcp sched structs"
  Squash to "selftests/bpf: Add bpf_first scheduler"
  Squash to "selftests/bpf: Add bpf_bkup scheduler"
  Squash to "selftests/bpf: Add bpf_rr scheduler"
  Squash to "selftests/bpf: Add bpf_red scheduler"

 include/linux/bpf.h                           |   2 +
 include/linux/bpf_types.h                     |   1 +
 include/linux/btf_ids.h                       |   1 +
 include/net/mptcp.h                           |  11 +-
 include/uapi/linux/bpf.h                      |  40 +++
 kernel/bpf/helpers.c                          |   4 +
 kernel/bpf/syscall.c                          |   6 +-
 kernel/bpf/verifier.c                         |  14 +-
 net/mptcp/bpf.c                               | 245 ++++++++++++++++++
 net/mptcp/protocol.c                          |  10 +-
 net/mptcp/protocol.h                          |   7 +-
 net/mptcp/sched.c                             |  46 +++-
 tools/include/uapi/linux/bpf.h                |  40 +++
 tools/lib/bpf/libbpf.c                        |   1 +
 tools/lib/bpf/libbpf_probes.c                 |   1 +
 tools/testing/selftests/bpf/bpf_tcp_helpers.h |  20 +-
 .../selftests/bpf/progs/mptcp_bpf_bkup.c      |  17 +-
 .../selftests/bpf/progs/mptcp_bpf_first.c     |   8 +-
 .../selftests/bpf/progs/mptcp_bpf_red.c       |  12 +-
 .../selftests/bpf/progs/mptcp_bpf_rr.c        |  43 ++-
 20 files changed, 475 insertions(+), 54 deletions(-)

-- 
2.35.3

[PATCH mptcp-next v10 01/13] Squash to "mptcp: drop last_snd and MPTCP_RESET_SCHEDULER"

[Geliang Tang]

Drop all last_snd.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 net/mptcp/protocol.c | 9 ---------
 net/mptcp/protocol.h | 1 -
 2 files changed, 10 deletions(-)

diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index aad21e0bf536..3ae0b94d25c0 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -1573,7 +1573,6 @@ void __mptcp_push_pending(struct sock *sk, unsigned int flags)
 					continue;
 				}
 				do_check_data_fin = true;
-				msk->last_snd = ssk;
 			}
 		}
 	}
@@ -1614,7 +1613,6 @@ static void __mptcp_subflow_push_pending(struct sock *sk, struct sock *ssk, bool
 			if (ret <= 0)
 				break;
 			copied += ret;
-			msk->last_snd = ssk;
 			continue;
 		}
 
@@ -1627,7 +1625,6 @@ static void __mptcp_subflow_push_pending(struct sock *sk, struct sock *ssk, bool
 			if (ret <= 0)
 				keep_pushing = false;
 			copied += ret;
-			msk->last_snd = ssk;
 		}
 
 		mptcp_for_each_subflow(msk, subflow) {
@@ -2410,9 +2407,6 @@ static void __mptcp_close_ssk(struct sock *sk, struct sock *ssk,
 		WRITE_ONCE(msk->first, NULL);
 
 out:
-	if (ssk == msk->last_snd)
-		msk->last_snd = NULL;
-
 	if (need_push)
 		__mptcp_push_pending(sk, 0);
 }
@@ -2592,8 +2586,6 @@ static void __mptcp_retrans(struct sock *sk)
 			}
 
 			release_sock(ssk);
-
-			msk->last_snd = ssk;
 		}
 	}
 
@@ -3104,7 +3096,6 @@ static int mptcp_disconnect(struct sock *sk, int flags)
 	 * subflow
 	 */
 	mptcp_destroy_common(msk, MPTCP_CF_FASTCLOSE);
-	msk->last_snd = NULL;
 	WRITE_ONCE(msk->flags, 0);
 	msk->cb_flags = 0;
 	msk->push_pending = 0;
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index e9ca30189731..8c9b2a03dc94 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -268,7 +268,6 @@ struct mptcp_sock {
 	u64		rcv_data_fin_seq;
 	u64		bytes_retrans;
 	int		rmem_fwd_alloc;
-	struct sock	*last_snd;
 	int		snd_burst;
 	int		old_wspace;
 	u64		recovery_snd_nxt;	/* in recovery mode accept up to this seq;
-- 
2.35.3

[PATCH mptcp-next v10 02/13] Squash to "mptcp: add struct mptcp_sched_ops"

[Geliang Tang]

- Add subflows in mptcp_sched_data.

- Drop const before msk in data_init() and get_subflow(), since
msk->bpf_storage needs to be writable.

- Add const before data in get_subflow().

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 include/net/mptcp.h | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/include/net/mptcp.h b/include/net/mptcp.h
index 828b10ddabee..ba0e1ee68a05 100644
--- a/include/net/mptcp.h
+++ b/include/net/mptcp.h
@@ -101,21 +101,22 @@ struct mptcp_out_options {
 
 struct mptcp_sched_data {
 	bool	reinject;
+	u8	subflows;
 	struct mptcp_subflow_context *contexts[MPTCP_SUBFLOWS_MAX];
 };
 
 struct mptcp_sched_ops {
-	void (*data_init)(const struct mptcp_sock *msk,
+	void (*data_init)(struct mptcp_sock *msk,
 			  struct mptcp_sched_data *data);
-	int (*get_subflow)(const struct mptcp_sock *msk,
-			   struct mptcp_sched_data *data);
+	int (*get_subflow)(struct mptcp_sock *msk,
+			   const struct mptcp_sched_data *data);
 
 	char			name[MPTCP_SCHED_NAME_MAX];
 	struct module		*owner;
 	struct list_head	list;
 
-	void (*init)(const struct mptcp_sock *msk);
-	void (*release)(const struct mptcp_sock *msk);
+	void (*init)(struct mptcp_sock *msk);
+	void (*release)(struct mptcp_sock *msk);
 } ____cacheline_aligned_in_smp;
 
 #ifdef CONFIG_MPTCP
-- 
2.35.3

[PATCH mptcp-next v10 03/13] Squash to "mptcp: add sched_data_set_contexts helper"

[Geliang Tang]

mptcp: add sched_data helpers

Add a new helper mptcp_sched_data_set_contexts() to set the subflow
pointers array in struct mptcp_sched_data. Add a new helper
mptcp_subflow_ctx_by_pos() to get the given pos subflow from the
contexts array in struct mptcp_sched_data. They will be invoked by
the BPF schedulers to export the subflow pointers to the BPF contexts.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 net/mptcp/protocol.h | 2 ++
 net/mptcp/sched.c    | 9 +++++++++
 2 files changed, 11 insertions(+)

diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index 8c9b2a03dc94..cb2f34c8ed09 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -664,6 +664,8 @@ void mptcp_subflow_set_scheduled(struct mptcp_subflow_context *subflow,
 				 bool scheduled);
 void mptcp_sched_data_set_contexts(const struct mptcp_sock *msk,
 				   struct mptcp_sched_data *data);
+struct mptcp_subflow_context *
+mptcp_subflow_ctx_by_pos(const struct mptcp_sched_data *data, unsigned int pos);
 struct sock *mptcp_subflow_get_send(struct mptcp_sock *msk);
 struct sock *mptcp_subflow_get_retrans(struct mptcp_sock *msk);
 int mptcp_sched_get_send(struct mptcp_sock *msk);
diff --git a/net/mptcp/sched.c b/net/mptcp/sched.c
index c7c167e48d72..82665263bb94 100644
--- a/net/mptcp/sched.c
+++ b/net/mptcp/sched.c
@@ -108,11 +108,20 @@ void mptcp_sched_data_set_contexts(const struct mptcp_sock *msk,
 		mptcp_subflow_set_scheduled(subflow, false);
 		data->contexts[i++] = subflow;
 	}
+	data->subflows = i;
 
 	for (; i < MPTCP_SUBFLOWS_MAX; i++)
 		data->contexts[i] = NULL;
 }
 
+struct mptcp_subflow_context *
+mptcp_subflow_ctx_by_pos(const struct mptcp_sched_data *data, unsigned int pos)
+{
+	if (pos >= MPTCP_SUBFLOWS_MAX)
+		return NULL;
+	return data->contexts[pos];
+}
+
 int mptcp_sched_get_send(struct mptcp_sock *msk)
 {
 	struct mptcp_subflow_context *subflow;
-- 
2.35.3

[PATCH mptcp-next v10 04/13] mptcp: register default scheduler

[Geliang Tang]

This patch defines the default packet scheduler mptcp_sched_default.
Register it in mptcp_sched_init(), which is invoked in mptcp_proto_init().
Skip deleting this default scheduler in mptcp_unregister_scheduler().

Set msk->sched to the default scheduler when the input parameter of
mptcp_init_sched() is NULL.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 net/mptcp/protocol.c |  1 +
 net/mptcp/protocol.h |  1 +
 net/mptcp/sched.c    | 37 +++++++++++++++++++++++++++++++++++--
 3 files changed, 37 insertions(+), 2 deletions(-)

diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 3ae0b94d25c0..5f9f046b2124 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -3970,6 +3970,7 @@ void __init mptcp_proto_init(void)
 
 	mptcp_subflow_init();
 	mptcp_pm_init();
+	mptcp_sched_init();
 	mptcp_token_init();
 
 	if (proto_register(&mptcp_prot, 1) != 0)
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index cb2f34c8ed09..bb4d50c8c398 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -657,6 +657,7 @@ void mptcp_info2sockaddr(const struct mptcp_addr_info *info,
 struct mptcp_sched_ops *mptcp_sched_find(const char *name);
 int mptcp_register_scheduler(struct mptcp_sched_ops *sched);
 void mptcp_unregister_scheduler(struct mptcp_sched_ops *sched);
+void mptcp_sched_init(void);
 int mptcp_init_sched(struct mptcp_sock *msk,
 		     struct mptcp_sched_ops *sched);
 void mptcp_release_sched(struct mptcp_sock *msk);
diff --git a/net/mptcp/sched.c b/net/mptcp/sched.c
index 82665263bb94..a80cf0481edf 100644
--- a/net/mptcp/sched.c
+++ b/net/mptcp/sched.c
@@ -16,6 +16,32 @@
 static DEFINE_SPINLOCK(mptcp_sched_list_lock);
 static LIST_HEAD(mptcp_sched_list);
 
+static void mptcp_sched_default_data_init(struct mptcp_sock *msk,
+					  struct mptcp_sched_data *data)
+{
+}
+
+static int mptcp_sched_default_get_subflow(struct mptcp_sock *msk,
+					   const struct mptcp_sched_data *data)
+{
+	struct sock *ssk;
+
+	ssk = data->reinject ? mptcp_subflow_get_retrans(msk) :
+			       mptcp_subflow_get_send(msk);
+	if (!ssk)
+		return -EINVAL;
+
+	mptcp_subflow_set_scheduled(mptcp_subflow_ctx(ssk), true);
+	return 0;
+}
+
+static struct mptcp_sched_ops mptcp_sched_default = {
+	.data_init	= mptcp_sched_default_data_init,
+	.get_subflow	= mptcp_sched_default_get_subflow,
+	.name		= "default",
+	.owner		= THIS_MODULE,
+};
+
 /* Must be called with rcu read lock held */
 struct mptcp_sched_ops *mptcp_sched_find(const char *name)
 {
@@ -50,16 +76,24 @@ int mptcp_register_scheduler(struct mptcp_sched_ops *sched)
 
 void mptcp_unregister_scheduler(struct mptcp_sched_ops *sched)
 {
+	if (sched == &mptcp_sched_default)
+		return;
+
 	spin_lock(&mptcp_sched_list_lock);
 	list_del_rcu(&sched->list);
 	spin_unlock(&mptcp_sched_list_lock);
 }
 
+void mptcp_sched_init(void)
+{
+	mptcp_register_scheduler(&mptcp_sched_default);
+}
+
 int mptcp_init_sched(struct mptcp_sock *msk,
 		     struct mptcp_sched_ops *sched)
 {
 	if (!sched)
-		goto out;
+		sched = &mptcp_sched_default;
 
 	if (!bpf_try_module_get(sched, sched->owner))
 		return -EBUSY;
@@ -70,7 +104,6 @@ int mptcp_init_sched(struct mptcp_sock *msk,
 
 	pr_debug("sched=%s", msk->sched->name);
 
-out:
 	return 0;
 }
 
-- 
2.35.3

[PATCH mptcp-next v10 05/13] Squash to "bpf: Add bpf_mptcp_sched_ops"

[Geliang Tang]

Add write accesses for avg_pacing_rate of struct mptcp_subflow_context
in .btf_struct_access. They will be used in the bpf_burst selftests.

Add write accesses for all bit flags of struct mptcp_subflow_context
between map_csum_len and data_avail in .btf_struct_access. The stale
flag will be used in the bpf_stale selftests.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 net/mptcp/bpf.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/net/mptcp/bpf.c b/net/mptcp/bpf.c
index dd1208670c54..b9fbee8d7bac 100644
--- a/net/mptcp/bpf.c
+++ b/net/mptcp/bpf.c
@@ -50,6 +50,12 @@ static int bpf_mptcp_sched_btf_struct_access(struct bpf_verifier_log *log,
 	case offsetof(struct mptcp_subflow_context, scheduled):
 		end = offsetofend(struct mptcp_subflow_context, scheduled);
 		break;
+	case offsetofend(struct mptcp_subflow_context, map_csum_len):
+		end = offsetof(struct mptcp_subflow_context, data_avail);
+		break;
+	case offsetof(struct mptcp_subflow_context, avg_pacing_rate):
+		end = offsetofend(struct mptcp_subflow_context, avg_pacing_rate);
+		break;
 	default:
 		bpf_log(log, "no write support to mptcp_subflow_context at off %d\n", off);
 		return -EACCES;
-- 
2.35.3

[PATCH mptcp-next v10 06/13] Squash to "bpf: Add bpf_mptcp_sched_kfunc_set"

[Geliang Tang]

Export mptcp_subflow_ctx_by_pos too.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 net/mptcp/bpf.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/net/mptcp/bpf.c b/net/mptcp/bpf.c
index b9fbee8d7bac..7f29c68a0dc3 100644
--- a/net/mptcp/bpf.c
+++ b/net/mptcp/bpf.c
@@ -166,6 +166,7 @@ struct bpf_struct_ops bpf_mptcp_sched_ops = {
 BTF_SET8_START(bpf_mptcp_sched_kfunc_ids)
 BTF_ID_FLAGS(func, mptcp_subflow_set_scheduled)
 BTF_ID_FLAGS(func, mptcp_sched_data_set_contexts)
+BTF_ID_FLAGS(func, mptcp_subflow_ctx_by_pos)
 BTF_SET8_END(bpf_mptcp_sched_kfunc_ids)
 
 static const struct btf_kfunc_id_set bpf_mptcp_sched_kfunc_set = {
-- 
2.35.3

[PATCH mptcp-next v10 07/13] bpf: Add mptcp_storage map type

[Geliang Tang]

Add a new bpf map type MPTCP_STORAGE, define its mptcp_storage_map_ops.
Add this storage into struct mptcp_sock to store persistent schedulers
data.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 include/linux/bpf_types.h      |   1 +
 include/uapi/linux/bpf.h       |   1 +
 kernel/bpf/syscall.c           |   6 +-
 kernel/bpf/verifier.c          |   1 +
 net/mptcp/bpf.c                | 154 +++++++++++++++++++++++++++++++++
 net/mptcp/protocol.h           |   3 +
 tools/include/uapi/linux/bpf.h |   1 +
 tools/lib/bpf/libbpf.c         |   1 +
 tools/lib/bpf/libbpf_probes.c  |   1 +
 9 files changed, 167 insertions(+), 2 deletions(-)

diff --git a/include/linux/bpf_types.h b/include/linux/bpf_types.h
index fc0d6f32c687..ef942ba2e176 100644
--- a/include/linux/bpf_types.h
+++ b/include/linux/bpf_types.h
@@ -122,6 +122,7 @@ BPF_MAP_TYPE(BPF_MAP_TYPE_XSKMAP, xsk_map_ops)
 BPF_MAP_TYPE(BPF_MAP_TYPE_SOCKMAP, sock_map_ops)
 BPF_MAP_TYPE(BPF_MAP_TYPE_SOCKHASH, sock_hash_ops)
 BPF_MAP_TYPE(BPF_MAP_TYPE_REUSEPORT_SOCKARRAY, reuseport_array_ops)
+BPF_MAP_TYPE(BPF_MAP_TYPE_MPTCP_STORAGE, mptcp_storage_map_ops)
 #endif
 #endif
 BPF_MAP_TYPE(BPF_MAP_TYPE_QUEUE, queue_map_ops)
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 6961a7b70028..f43cf69e47ea 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -943,6 +943,7 @@ enum bpf_map_type {
 	BPF_MAP_TYPE_BLOOM_FILTER,
 	BPF_MAP_TYPE_USER_RINGBUF,
 	BPF_MAP_TYPE_CGRP_STORAGE,
+	BPF_MAP_TYPE_MPTCP_STORAGE,
 };
 
 /* Note that tracing related programs such as
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 4497b193dd20..8b96df76b780 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -1060,7 +1060,8 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf,
 				    map->map_type != BPF_MAP_TYPE_SK_STORAGE &&
 				    map->map_type != BPF_MAP_TYPE_INODE_STORAGE &&
 				    map->map_type != BPF_MAP_TYPE_TASK_STORAGE &&
-				    map->map_type != BPF_MAP_TYPE_CGRP_STORAGE) {
+				    map->map_type != BPF_MAP_TYPE_CGRP_STORAGE &&
+				    map->map_type != BPF_MAP_TYPE_MPTCP_STORAGE) {
 					ret = -EOPNOTSUPP;
 					goto free_map_tab;
 				}
@@ -1085,7 +1086,8 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf,
 				    map->map_type != BPF_MAP_TYPE_SK_STORAGE &&
 				    map->map_type != BPF_MAP_TYPE_INODE_STORAGE &&
 				    map->map_type != BPF_MAP_TYPE_TASK_STORAGE &&
-				    map->map_type != BPF_MAP_TYPE_CGRP_STORAGE) {
+				    map->map_type != BPF_MAP_TYPE_CGRP_STORAGE &&
+				    map->map_type != BPF_MAP_TYPE_MPTCP_STORAGE) {
 					ret = -EOPNOTSUPP;
 					goto free_map_tab;
 				}
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index b54193de762b..d8482a4ef465 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -16577,6 +16577,7 @@ static int check_map_prog_compatibility(struct bpf_verifier_env *env,
 		case BPF_MAP_TYPE_SK_STORAGE:
 		case BPF_MAP_TYPE_TASK_STORAGE:
 		case BPF_MAP_TYPE_CGRP_STORAGE:
+		case BPF_MAP_TYPE_MPTCP_STORAGE:
 			break;
 		default:
 			verbose(env,
diff --git a/net/mptcp/bpf.c b/net/mptcp/bpf.c
index 7f29c68a0dc3..764c670e843b 100644
--- a/net/mptcp/bpf.c
+++ b/net/mptcp/bpf.c
@@ -10,6 +10,7 @@
 #define pr_fmt(fmt) "MPTCP: " fmt
 
 #include <linux/bpf.h>
+#include <linux/bpf_local_storage.h>
 #include <linux/bpf_verifier.h>
 #include <linux/btf.h>
 #include <linux/btf_ids.h>
@@ -180,6 +181,159 @@ static int __init bpf_mptcp_sched_kfunc_init(void)
 					 &bpf_mptcp_sched_kfunc_set);
 }
 late_initcall(bpf_mptcp_sched_kfunc_init);
+
+/* mptcp sched storage */
+DEFINE_BPF_STORAGE_CACHE(mptcp_cache);
+
+static DEFINE_PER_CPU(int, bpf_mptcp_storage_busy);
+
+BTF_ID_LIST_GLOBAL_SINGLE(bpf_mptcp_btf_id, struct, mptcp_sock)
+
+static void bpf_mptcp_storage_lock(void)
+{
+	migrate_disable();
+	this_cpu_inc(bpf_mptcp_storage_busy);
+}
+
+static void bpf_mptcp_storage_unlock(void)
+{
+	this_cpu_dec(bpf_mptcp_storage_busy);
+	migrate_enable();
+}
+
+static struct bpf_local_storage __rcu **mptcp_storage_ptr(void *owner)
+{
+	struct mptcp_sock *msk = owner;
+
+	return &msk->bpf_storage;
+}
+
+static struct bpf_local_storage_data *
+mptcp_storage_lookup(struct mptcp_sock *msk, struct bpf_map *map,
+		     bool cacheit_lockit)
+{
+	struct bpf_local_storage *msk_storage;
+	struct bpf_local_storage_map *smap;
+
+	msk_storage = rcu_dereference_check(msk->bpf_storage, bpf_rcu_lock_held());
+	if (!msk_storage)
+		return NULL;
+
+	smap = (struct bpf_local_storage_map *)map;
+	return bpf_local_storage_lookup(msk_storage, smap, cacheit_lockit);
+}
+
+static void *bpf_mptcp_storage_lookup_elem(struct bpf_map *map, void *key)
+{
+	struct bpf_local_storage_data *sdata;
+	struct mptcp_sock *msk;
+	struct socket *sock;
+	int err, fd;
+
+	fd = *(int *)key;
+	sock = sockfd_lookup(fd, &err);
+	if (!sock)
+		return NULL;
+
+	msk = bpf_mptcp_sock_from_subflow(sock->sk);
+	if (!msk)
+		return NULL;
+
+	bpf_mptcp_storage_lock();
+	sdata = mptcp_storage_lookup(msk, map, true);
+	bpf_mptcp_storage_unlock();
+	fput(sock->file);
+	return sdata ? sdata->data : NULL;
+}
+
+static long bpf_mptcp_storage_update_elem(struct bpf_map *map, void *key,
+					  void *value, u64 map_flags)
+{
+	struct bpf_local_storage_data *sdata;
+	struct mptcp_sock *msk;
+	struct socket *sock;
+	int err, fd;
+
+	fd = *(int *)key;
+	sock = sockfd_lookup(fd, &err);
+	if (!sock)
+		return PTR_ERR(sock);
+
+	msk = bpf_mptcp_sock_from_subflow(sock->sk);
+	if (IS_ERR(msk))
+		return PTR_ERR(msk);
+
+	bpf_mptcp_storage_lock();
+	sdata = bpf_local_storage_update(msk, (struct bpf_local_storage_map *)map,
+					 value, map_flags, GFP_ATOMIC);
+	bpf_mptcp_storage_unlock();
+	fput(sock->file);
+	return PTR_ERR_OR_ZERO(sdata);
+}
+
+static int mptcp_storage_delete(struct mptcp_sock *msk, struct bpf_map *map)
+{
+	struct bpf_local_storage_data *sdata;
+
+	sdata = mptcp_storage_lookup(msk, map, false);
+	if (!sdata)
+		return -ENOENT;
+
+	bpf_selem_unlink(SELEM(sdata), false);
+	return 0;
+}
+
+static long bpf_mptcp_storage_delete_elem(struct bpf_map *map, void *key)
+{
+	struct mptcp_sock *msk;
+	struct socket *sock;
+	int ret, err, fd;
+
+	fd = *(int *)key;
+	sock = sockfd_lookup(fd, &err);
+	if (!sock)
+		return PTR_ERR(sock);
+
+	msk = bpf_mptcp_sock_from_subflow(sock->sk);
+	if (IS_ERR(msk))
+		return PTR_ERR(msk);
+
+	bpf_mptcp_storage_lock();
+	ret = mptcp_storage_delete(msk, map);
+	bpf_mptcp_storage_unlock();
+	fput(sock->file);
+	return ret;
+}
+
+static int notsupp_get_next_key(struct bpf_map *map, void *key, void *next_key)
+{
+	return -ENOTSUPP;
+}
+
+static struct bpf_map *mptcp_storage_map_alloc(union bpf_attr *attr)
+{
+	return bpf_local_storage_map_alloc(attr, &mptcp_cache, true);
+}
+
+static void mptcp_storage_map_free(struct bpf_map *map)
+{
+	bpf_local_storage_map_free(map, &mptcp_cache, &bpf_mptcp_storage_busy);
+}
+
+const struct bpf_map_ops mptcp_storage_map_ops = {
+	.map_meta_equal = bpf_map_meta_equal,
+	.map_alloc_check = bpf_local_storage_map_alloc_check,
+	.map_alloc = mptcp_storage_map_alloc,
+	.map_free = mptcp_storage_map_free,
+	.map_get_next_key = notsupp_get_next_key,
+	.map_lookup_elem = bpf_mptcp_storage_lookup_elem,
+	.map_update_elem = bpf_mptcp_storage_update_elem,
+	.map_delete_elem = bpf_mptcp_storage_delete_elem,
+	.map_check_btf = bpf_local_storage_map_check_btf,
+	.map_mem_usage = bpf_local_storage_map_mem_usage,
+	.map_btf_id = &bpf_local_storage_map_btf_id[0],
+	.map_owner_storage_ptr = mptcp_storage_ptr,
+};
 #endif /* CONFIG_BPF_JIT */
 
 struct mptcp_sock *bpf_mptcp_sock_from_subflow(struct sock *sk)
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index bb4d50c8c398..9e2aecae8923 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -313,6 +313,9 @@ struct mptcp_sock {
 				   */
 	struct sock	*first;
 	struct mptcp_pm_data	pm;
+#ifdef CONFIG_BPF_SYSCALL
+	struct bpf_local_storage __rcu	*bpf_storage;
+#endif
 	struct mptcp_sched_ops	*sched;
 	struct {
 		u32	space;	/* bytes copied in last measurement window */
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 6961a7b70028..f43cf69e47ea 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -943,6 +943,7 @@ enum bpf_map_type {
 	BPF_MAP_TYPE_BLOOM_FILTER,
 	BPF_MAP_TYPE_USER_RINGBUF,
 	BPF_MAP_TYPE_CGRP_STORAGE,
+	BPF_MAP_TYPE_MPTCP_STORAGE,
 };
 
 /* Note that tracing related programs such as
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 214f828ece6b..3365b6a34bb4 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -168,6 +168,7 @@ static const char * const map_type_name[] = {
 	[BPF_MAP_TYPE_BLOOM_FILTER]		= "bloom_filter",
 	[BPF_MAP_TYPE_USER_RINGBUF]             = "user_ringbuf",
 	[BPF_MAP_TYPE_CGRP_STORAGE]		= "cgrp_storage",
+	[BPF_MAP_TYPE_MPTCP_STORAGE]		= "mptcp_storage",
 };
 
 static const char * const prog_type_name[] = {
diff --git a/tools/lib/bpf/libbpf_probes.c b/tools/lib/bpf/libbpf_probes.c
index 9c4db90b92b6..521cdc2dbcd2 100644
--- a/tools/lib/bpf/libbpf_probes.c
+++ b/tools/lib/bpf/libbpf_probes.c
@@ -308,6 +308,7 @@ static int probe_map_create(enum bpf_map_type map_type)
 	case BPF_MAP_TYPE_INODE_STORAGE:
 	case BPF_MAP_TYPE_TASK_STORAGE:
 	case BPF_MAP_TYPE_CGRP_STORAGE:
+	case BPF_MAP_TYPE_MPTCP_STORAGE:
 		btf_key_type_id = 1;
 		btf_value_type_id = 3;
 		value_size = 8;
-- 
2.35.3

[PATCH mptcp-next v10 08/13] bpf: Add mptcp_storage helpers

[Geliang Tang]

Add two helpers bpf_mptcp_storage_get() and bpf_mptcp_storage_delete()
for mptcp_storage map type to get or delete a bpf_local_storage from
the given msk.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 include/linux/bpf.h            |  2 +
 include/linux/btf_ids.h        |  1 +
 include/uapi/linux/bpf.h       | 39 ++++++++++++++++
 kernel/bpf/helpers.c           |  4 ++
 kernel/bpf/verifier.c          | 13 +++++-
 net/mptcp/bpf.c                | 84 ++++++++++++++++++++++++++++++++++
 tools/include/uapi/linux/bpf.h | 39 ++++++++++++++++
 7 files changed, 181 insertions(+), 1 deletion(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index f58895830ada..ac4b5748451a 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -2909,6 +2909,8 @@ extern const struct bpf_func_proto bpf_get_retval_proto;
 extern const struct bpf_func_proto bpf_user_ringbuf_drain_proto;
 extern const struct bpf_func_proto bpf_cgrp_storage_get_proto;
 extern const struct bpf_func_proto bpf_cgrp_storage_delete_proto;
+extern const struct bpf_func_proto bpf_mptcp_storage_get_proto;
+extern const struct bpf_func_proto bpf_mptcp_storage_delete_proto;
 
 const struct bpf_func_proto *tracing_prog_func_proto(
   enum bpf_func_id func_id, const struct bpf_prog *prog);
diff --git a/include/linux/btf_ids.h b/include/linux/btf_ids.h
index 00950cc03bff..abd56cfcb03f 100644
--- a/include/linux/btf_ids.h
+++ b/include/linux/btf_ids.h
@@ -267,5 +267,6 @@ MAX_BTF_TRACING_TYPE,
 extern u32 btf_tracing_ids[];
 extern u32 bpf_cgroup_btf_id[];
 extern u32 bpf_local_storage_map_btf_id[];
+extern u32 bpf_mptcp_btf_id[];
 
 #endif
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index f43cf69e47ea..137e0620fd69 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -5570,6 +5570,43 @@ union bpf_attr {
  *		0 on success.
  *
  *		**-ENOENT** if the bpf_local_storage cannot be found.
+ *
+ * void *bpf_mptcp_storage_get(struct bpf_map *map, struct mptcp_sock *msk, void *value, u64 flags)
+ *	Description
+ *		Get a bpf_local_storage from the *msk*.
+ *
+ *		Logically, it could be thought of as getting the value from
+ *		a *map* with *msk* as the **key**.  From this
+ *		perspective,  the usage is not much different from
+ *		**bpf_map_lookup_elem**\ (*map*, **&**\ *msk*) except this
+ *		helper enforces the key must be a mptcp_sock and the map must also
+ *		be a **BPF_MAP_TYPE_MPTCP_STORAGE**.
+ *
+ *		Underneath, the value is stored locally at *msk* instead of
+ *		the *map*.  The *map* is used as the bpf-local-storage
+ *		"type". The bpf-local-storage "type" (i.e. the *map*) is
+ *		searched against all bpf_local_storage residing at *msk*.
+ *
+ *		An optional *flags* (**BPF_LOCAL_STORAGE_GET_F_CREATE**) can be
+ *		used such that a new bpf_local_storage will be
+ *		created if one does not exist.  *value* can be used
+ *		together with **BPF_LOCAL_STORAGE_GET_F_CREATE** to specify
+ *		the initial value of a bpf_local_storage.  If *value* is
+ *		**NULL**, the new bpf_local_storage will be zero initialized.
+ *	Return
+ *		A bpf_local_storage pointer is returned on success.
+ *
+ *		**NULL** if not found or there was an error in adding
+ *		a new bpf_local_storage.
+ *
+ * long bpf_mptcp_storage_delete(struct bpf_map *map, struct mptcp_sock *msk)
+ *	Description
+ *		Delete a bpf_local_storage from the *msk*.
+ *
+ *	Return
+ *		0 on success.
+ *
+ *		**-ENOENT** if the bpf_local_storage cannot be found.
  */
 #define ___BPF_FUNC_MAPPER(FN, ctx...)			\
 	FN(unspec, 0, ##ctx)				\
@@ -5784,6 +5821,8 @@ union bpf_attr {
 	FN(user_ringbuf_drain, 209, ##ctx)		\
 	FN(cgrp_storage_get, 210, ##ctx)		\
 	FN(cgrp_storage_delete, 211, ##ctx)		\
+	FN(mptcp_storage_get, 212, ##ctx)		\
+	FN(mptcp_storage_delete, 213, ##ctx)		\
 	/* */
 
 /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't
diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index 4ef4c4f8a355..381b680007fb 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -1768,6 +1768,10 @@ bpf_base_func_proto(enum bpf_func_id func_id)
 	case BPF_FUNC_get_current_ancestor_cgroup_id:
 		return &bpf_get_current_ancestor_cgroup_id_proto;
 #endif
+	case BPF_FUNC_mptcp_storage_get:
+		return &bpf_mptcp_storage_get_proto;
+	case BPF_FUNC_mptcp_storage_delete:
+		return &bpf_mptcp_storage_delete_proto;
 	default:
 		break;
 	}
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index d8482a4ef465..c7479628acf7 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -553,7 +553,8 @@ static bool is_storage_get_function(enum bpf_func_id func_id)
 	return func_id == BPF_FUNC_sk_storage_get ||
 	       func_id == BPF_FUNC_inode_storage_get ||
 	       func_id == BPF_FUNC_task_storage_get ||
-	       func_id == BPF_FUNC_cgrp_storage_get;
+	       func_id == BPF_FUNC_cgrp_storage_get ||
+	       func_id == BPF_FUNC_mptcp_storage_get;
 }
 
 static bool helper_multiple_ref_obj_use(enum bpf_func_id func_id,
@@ -8241,6 +8242,11 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env,
 		    func_id != BPF_FUNC_map_push_elem)
 			goto error;
 		break;
+	case BPF_MAP_TYPE_MPTCP_STORAGE:
+		if (func_id != BPF_FUNC_mptcp_storage_get &&
+		    func_id != BPF_FUNC_mptcp_storage_delete)
+			goto error;
+		break;
 	default:
 		break;
 	}
@@ -8353,6 +8359,11 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env,
 		if (map->map_type != BPF_MAP_TYPE_CGRP_STORAGE)
 			goto error;
 		break;
+	case BPF_FUNC_mptcp_storage_get:
+	case BPF_FUNC_mptcp_storage_delete:
+		if (map->map_type != BPF_MAP_TYPE_MPTCP_STORAGE)
+			goto error;
+		break;
 	default:
 		break;
 	}
diff --git a/net/mptcp/bpf.c b/net/mptcp/bpf.c
index 764c670e843b..0bf3409cf288 100644
--- a/net/mptcp/bpf.c
+++ b/net/mptcp/bpf.c
@@ -201,6 +201,17 @@ static void bpf_mptcp_storage_unlock(void)
 	migrate_enable();
 }
 
+static bool bpf_mptcp_storage_trylock(void)
+{
+	migrate_disable();
+	if (unlikely(this_cpu_inc_return(bpf_mptcp_storage_busy) != 1)) {
+		this_cpu_dec(bpf_mptcp_storage_busy);
+		migrate_enable();
+		return false;
+	}
+	return true;
+}
+
 static struct bpf_local_storage __rcu **mptcp_storage_ptr(void *owner)
 {
 	struct mptcp_sock *msk = owner;
@@ -305,6 +316,59 @@ static long bpf_mptcp_storage_delete_elem(struct bpf_map *map, void *key)
 	return ret;
 }
 
+/* Called by bpf_mptcp_storage_get*() helpers */
+static void *__bpf_mptcp_storage_get(struct bpf_map *map,
+				     struct mptcp_sock *msk, void *value,
+				     u64 flags, gfp_t gfp_flags, bool nobusy)
+{
+        struct bpf_local_storage_data *sdata;
+
+        sdata = mptcp_storage_lookup(msk, map, nobusy);
+        if (sdata)
+                return sdata->data;
+
+        if ((flags & BPF_LOCAL_STORAGE_GET_F_CREATE) && nobusy) {
+                sdata = bpf_local_storage_update(
+                        msk, (struct bpf_local_storage_map *)map, value,
+                        BPF_NOEXIST, gfp_flags);
+                return IS_ERR(sdata) ? NULL : sdata->data;
+        }
+
+        return NULL;
+}
+
+/* *gfp_flags* is a hidden argument provided by the verifier */
+BPF_CALL_5(bpf_mptcp_storage_get, struct bpf_map *, map, struct mptcp_sock *, msk,
+	   void *, value, u64, flags, gfp_t, gfp_flags)
+{
+	void *data;
+
+	WARN_ON_ONCE(!bpf_rcu_lock_held());
+	if (flags & ~BPF_LOCAL_STORAGE_GET_F_CREATE || !msk)
+		return (unsigned long)NULL;
+
+	bpf_mptcp_storage_lock();
+	data = __bpf_mptcp_storage_get(map, msk, value, flags, gfp_flags, true);
+	bpf_mptcp_storage_unlock();
+	return (unsigned long)data;
+}
+
+BPF_CALL_2(bpf_mptcp_storage_delete, struct bpf_map *, map, struct mptcp_sock *, msk)
+{
+	int ret;
+
+	WARN_ON_ONCE(!bpf_rcu_lock_held());
+	if (!msk)
+		return -EINVAL;
+
+	if (!bpf_mptcp_storage_trylock())
+		return -EBUSY;
+
+	ret = mptcp_storage_delete(msk, map);
+	bpf_mptcp_storage_unlock();
+	return ret;
+}
+
 static int notsupp_get_next_key(struct bpf_map *map, void *key, void *next_key)
 {
 	return -ENOTSUPP;
@@ -334,6 +398,26 @@ const struct bpf_map_ops mptcp_storage_map_ops = {
 	.map_btf_id = &bpf_local_storage_map_btf_id[0],
 	.map_owner_storage_ptr = mptcp_storage_ptr,
 };
+
+const struct bpf_func_proto bpf_mptcp_storage_get_proto = {
+	.func = bpf_mptcp_storage_get,
+	.gpl_only = false,
+	.ret_type = RET_PTR_TO_MAP_VALUE_OR_NULL,
+	.arg1_type = ARG_CONST_MAP_PTR,
+	.arg2_type = ARG_PTR_TO_BTF_ID_OR_NULL,
+	.arg2_btf_id = &bpf_mptcp_btf_id[0],
+	.arg3_type = ARG_PTR_TO_MAP_VALUE_OR_NULL,
+	.arg4_type = ARG_ANYTHING,
+};
+
+const struct bpf_func_proto bpf_mptcp_storage_delete_proto = {
+	.func           = bpf_mptcp_storage_delete,
+	.gpl_only       = false,
+	.ret_type       = RET_INTEGER,
+	.arg1_type      = ARG_CONST_MAP_PTR,
+	.arg2_type      = ARG_PTR_TO_BTF_ID_OR_NULL,
+	.arg2_btf_id    = &bpf_mptcp_btf_id[0],
+};
 #endif /* CONFIG_BPF_JIT */
 
 struct mptcp_sock *bpf_mptcp_sock_from_subflow(struct sock *sk)
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index f43cf69e47ea..137e0620fd69 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -5570,6 +5570,43 @@ union bpf_attr {
  *		0 on success.
  *
  *		**-ENOENT** if the bpf_local_storage cannot be found.
+ *
+ * void *bpf_mptcp_storage_get(struct bpf_map *map, struct mptcp_sock *msk, void *value, u64 flags)
+ *	Description
+ *		Get a bpf_local_storage from the *msk*.
+ *
+ *		Logically, it could be thought of as getting the value from
+ *		a *map* with *msk* as the **key**.  From this
+ *		perspective,  the usage is not much different from
+ *		**bpf_map_lookup_elem**\ (*map*, **&**\ *msk*) except this
+ *		helper enforces the key must be a mptcp_sock and the map must also
+ *		be a **BPF_MAP_TYPE_MPTCP_STORAGE**.
+ *
+ *		Underneath, the value is stored locally at *msk* instead of
+ *		the *map*.  The *map* is used as the bpf-local-storage
+ *		"type". The bpf-local-storage "type" (i.e. the *map*) is
+ *		searched against all bpf_local_storage residing at *msk*.
+ *
+ *		An optional *flags* (**BPF_LOCAL_STORAGE_GET_F_CREATE**) can be
+ *		used such that a new bpf_local_storage will be
+ *		created if one does not exist.  *value* can be used
+ *		together with **BPF_LOCAL_STORAGE_GET_F_CREATE** to specify
+ *		the initial value of a bpf_local_storage.  If *value* is
+ *		**NULL**, the new bpf_local_storage will be zero initialized.
+ *	Return
+ *		A bpf_local_storage pointer is returned on success.
+ *
+ *		**NULL** if not found or there was an error in adding
+ *		a new bpf_local_storage.
+ *
+ * long bpf_mptcp_storage_delete(struct bpf_map *map, struct mptcp_sock *msk)
+ *	Description
+ *		Delete a bpf_local_storage from the *msk*.
+ *
+ *	Return
+ *		0 on success.
+ *
+ *		**-ENOENT** if the bpf_local_storage cannot be found.
  */
 #define ___BPF_FUNC_MAPPER(FN, ctx...)			\
 	FN(unspec, 0, ##ctx)				\
@@ -5784,6 +5821,8 @@ union bpf_attr {
 	FN(user_ringbuf_drain, 209, ##ctx)		\
 	FN(cgrp_storage_get, 210, ##ctx)		\
 	FN(cgrp_storage_delete, 211, ##ctx)		\
+	FN(mptcp_storage_get, 212, ##ctx)		\
+	FN(mptcp_storage_delete, 213, ##ctx)		\
 	/* */
 
 /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't
-- 
2.35.3

[PATCH mptcp-next v10 09/13] Squash to "selftests/bpf: Add mptcp sched structs"

[Geliang Tang]

Update scheduler API.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 tools/testing/selftests/bpf/bpf_tcp_helpers.h | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/tools/testing/selftests/bpf/bpf_tcp_helpers.h b/tools/testing/selftests/bpf/bpf_tcp_helpers.h
index 72c618037386..b4b766c7a68f 100644
--- a/tools/testing/selftests/bpf/bpf_tcp_helpers.h
+++ b/tools/testing/selftests/bpf/bpf_tcp_helpers.h
@@ -240,19 +240,19 @@ struct mptcp_subflow_context {
 
 struct mptcp_sched_data {
 	bool	reinject;
-	struct mptcp_subflow_context *contexts[MPTCP_SUBFLOWS_MAX];
+	__u8	subflows;
 } __attribute__((preserve_access_index));
 
 struct mptcp_sched_ops {
 	char name[MPTCP_SCHED_NAME_MAX];
 
-	void (*init)(const struct mptcp_sock *msk);
-	void (*release)(const struct mptcp_sock *msk);
+	void (*init)(struct mptcp_sock *msk);
+	void (*release)(struct mptcp_sock *msk);
 
-	void (*data_init)(const struct mptcp_sock *msk,
+	void (*data_init)(struct mptcp_sock *msk,
 			  struct mptcp_sched_data *data);
-	int (*get_subflow)(const struct mptcp_sock *msk,
-			   struct mptcp_sched_data *data);
+	int (*get_subflow)(struct mptcp_sock *msk,
+			   const struct mptcp_sched_data *data);
 	void *owner;
 };
 
@@ -269,5 +269,7 @@ extern void mptcp_subflow_set_scheduled(struct mptcp_subflow_context *subflow,
 					bool scheduled) __ksym;
 extern void mptcp_sched_data_set_contexts(const struct mptcp_sock *msk,
 					  struct mptcp_sched_data *data) __ksym;
+extern struct mptcp_subflow_context *
+mptcp_subflow_ctx_by_pos(const struct mptcp_sched_data *data, unsigned int pos) __ksym;
 
 #endif
-- 
2.35.3

[PATCH mptcp-next v10 10/13] Squash to "selftests/bpf: Add bpf_first scheduler"

[Geliang Tang]

Update API.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 tools/testing/selftests/bpf/progs/mptcp_bpf_first.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/bpf/progs/mptcp_bpf_first.c b/tools/testing/selftests/bpf/progs/mptcp_bpf_first.c
index e4caa2dd8c6f..75591256067a 100644
--- a/tools/testing/selftests/bpf/progs/mptcp_bpf_first.c
+++ b/tools/testing/selftests/bpf/progs/mptcp_bpf_first.c
@@ -16,16 +16,16 @@ void BPF_PROG(mptcp_sched_first_release, const struct mptcp_sock *msk)
 {
 }
 
-void BPF_STRUCT_OPS(bpf_first_data_init, const struct mptcp_sock *msk,
+void BPF_STRUCT_OPS(bpf_first_data_init, struct mptcp_sock *msk,
 		    struct mptcp_sched_data *data)
 {
 	mptcp_sched_data_set_contexts(msk, data);
 }
 
-int BPF_STRUCT_OPS(bpf_first_get_subflow, const struct mptcp_sock *msk,
-		   struct mptcp_sched_data *data)
+int BPF_STRUCT_OPS(bpf_first_get_subflow, struct mptcp_sock *msk,
+		   const struct mptcp_sched_data *data)
 {
-	mptcp_subflow_set_scheduled(data->contexts[0], true);
+	mptcp_subflow_set_scheduled(mptcp_subflow_ctx_by_pos(data, 0), true);
 	return 0;
 }
 
-- 
2.35.3

[PATCH mptcp-next v10 11/13] Squash to "selftests/bpf: Add bpf_bkup scheduler"

[Geliang Tang]

Update API.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 .../selftests/bpf/progs/mptcp_bpf_bkup.c        | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/tools/testing/selftests/bpf/progs/mptcp_bpf_bkup.c b/tools/testing/selftests/bpf/progs/mptcp_bpf_bkup.c
index b2724426676e..2ff69f8d66ed 100644
--- a/tools/testing/selftests/bpf/progs/mptcp_bpf_bkup.c
+++ b/tools/testing/selftests/bpf/progs/mptcp_bpf_bkup.c
@@ -16,28 +16,31 @@ void BPF_PROG(mptcp_sched_bkup_release, const struct mptcp_sock *msk)
 {
 }
 
-void BPF_STRUCT_OPS(bpf_bkup_data_init, const struct mptcp_sock *msk,
+void BPF_STRUCT_OPS(bpf_bkup_data_init, struct mptcp_sock *msk,
 		    struct mptcp_sched_data *data)
 {
 	mptcp_sched_data_set_contexts(msk, data);
 }
 
-int BPF_STRUCT_OPS(bpf_bkup_get_subflow, const struct mptcp_sock *msk,
-		   struct mptcp_sched_data *data)
+int BPF_STRUCT_OPS(bpf_bkup_get_subflow, struct mptcp_sock *msk,
+		   const struct mptcp_sched_data *data)
 {
 	int nr = 0;
 
-	for (int i = 0; i < MPTCP_SUBFLOWS_MAX; i++) {
-		if (!data->contexts[i])
+	for (int i = 0; i < data->subflows && i < MPTCP_SUBFLOWS_MAX; i++) {
+		struct mptcp_subflow_context *subflow;
+
+		subflow = mptcp_subflow_ctx_by_pos(data, i);
+		if (!subflow)
 			break;
 
-		if (!BPF_CORE_READ_BITFIELD_PROBED(data->contexts[i], backup)) {
+		if (!BPF_CORE_READ_BITFIELD_PROBED(subflow, backup)) {
 			nr = i;
 			break;
 		}
 	}
 
-	mptcp_subflow_set_scheduled(data->contexts[nr], true);
+	mptcp_subflow_set_scheduled(mptcp_subflow_ctx_by_pos(data, nr), true);
 	return 0;
 }
 
-- 
2.35.3

[PATCH mptcp-next v10 12/13] Squash to "selftests/bpf: Add bpf_rr scheduler"

[Geliang Tang]

Use mptcp_storage instead of msk->last_snd.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 tools/testing/selftests/bpf/bpf_tcp_helpers.h |  6 ++-
 .../selftests/bpf/progs/mptcp_bpf_rr.c        | 43 ++++++++++++++-----
 2 files changed, 38 insertions(+), 11 deletions(-)

diff --git a/tools/testing/selftests/bpf/bpf_tcp_helpers.h b/tools/testing/selftests/bpf/bpf_tcp_helpers.h
index b4b766c7a68f..945dd46c98c0 100644
--- a/tools/testing/selftests/bpf/bpf_tcp_helpers.h
+++ b/tools/testing/selftests/bpf/bpf_tcp_helpers.h
@@ -259,7 +259,6 @@ struct mptcp_sched_ops {
 struct mptcp_sock {
 	struct inet_connection_sock	sk;
 
-	struct sock	*last_snd;
 	__u32		token;
 	struct sock	*first;
 	char		ca_name[TCP_CA_NAME_MAX];
@@ -271,5 +270,10 @@ extern void mptcp_sched_data_set_contexts(const struct mptcp_sock *msk,
 					  struct mptcp_sched_data *data) __ksym;
 extern struct mptcp_subflow_context *
 mptcp_subflow_ctx_by_pos(const struct mptcp_sched_data *data, unsigned int pos) __ksym;
+static inline struct sock *
+mptcp_subflow_tcp_sock(const struct mptcp_subflow_context *subflow)
+{
+	return subflow->tcp_sock;
+}
 
 #endif
diff --git a/tools/testing/selftests/bpf/progs/mptcp_bpf_rr.c b/tools/testing/selftests/bpf/progs/mptcp_bpf_rr.c
index e101428e5906..a6f1aa55a16b 100644
--- a/tools/testing/selftests/bpf/progs/mptcp_bpf_rr.c
+++ b/tools/testing/selftests/bpf/progs/mptcp_bpf_rr.c
@@ -6,33 +6,49 @@
 
 char _license[] SEC("license") = "GPL";
 
+struct mptcp_rr_storage {
+	struct sock *last_snd;
+};
+struct sock *last_snd;
+
+struct {
+	__uint(type, BPF_MAP_TYPE_MPTCP_STORAGE);
+	__uint(map_flags, BPF_F_NO_PREALLOC);
+	__type(key, int);
+	__type(value, struct mptcp_rr_storage);
+} mptcp_rr_map SEC(".maps");
+
 SEC("struct_ops/mptcp_sched_rr_init")
-void BPF_PROG(mptcp_sched_rr_init, const struct mptcp_sock *msk)
+void BPF_PROG(mptcp_sched_rr_init, struct mptcp_sock *msk)
 {
 }
 
 SEC("struct_ops/mptcp_sched_rr_release")
-void BPF_PROG(mptcp_sched_rr_release, const struct mptcp_sock *msk)
+void BPF_PROG(mptcp_sched_rr_release, struct mptcp_sock *msk)
 {
+	bpf_mptcp_storage_delete(&mptcp_rr_map, msk);
 }
 
-void BPF_STRUCT_OPS(bpf_rr_data_init, const struct mptcp_sock *msk,
+void BPF_STRUCT_OPS(bpf_rr_data_init, struct mptcp_sock *msk,
 		    struct mptcp_sched_data *data)
 {
 	mptcp_sched_data_set_contexts(msk, data);
 }
 
-int BPF_STRUCT_OPS(bpf_rr_get_subflow, const struct mptcp_sock *msk,
-		   struct mptcp_sched_data *data)
+int BPF_STRUCT_OPS(bpf_rr_get_subflow, struct mptcp_sock *msk,
+		   const struct mptcp_sched_data *data)
 {
+	struct mptcp_subflow_context *subflow;
+	struct mptcp_rr_storage *ptr;
 	int nr = 0;
 
-	for (int i = 0; i < MPTCP_SUBFLOWS_MAX; i++) {
-		if (!msk->last_snd || !data->contexts[i])
+	for (int i = 0; i < data->subflows && i < MPTCP_SUBFLOWS_MAX; i++) {
+		subflow = mptcp_subflow_ctx_by_pos(data, i);
+		if (!last_snd || !subflow)
 			break;
 
-		if (data->contexts[i]->tcp_sock == msk->last_snd) {
-			if (i + 1 == MPTCP_SUBFLOWS_MAX || !data->contexts[i + 1])
+		if (mptcp_subflow_tcp_sock(subflow) == last_snd) {
+			if (i + 1 == MPTCP_SUBFLOWS_MAX || !mptcp_subflow_ctx_by_pos(data, i + 1))
 				break;
 
 			nr = i + 1;
@@ -40,7 +56,14 @@ int BPF_STRUCT_OPS(bpf_rr_get_subflow, const struct mptcp_sock *msk,
 		}
 	}
 
-	mptcp_subflow_set_scheduled(data->contexts[nr], true);
+	subflow = mptcp_subflow_ctx_by_pos(data, nr);
+	if (!subflow)
+		return -1;
+	mptcp_subflow_set_scheduled(subflow, true);
+	last_snd = mptcp_subflow_tcp_sock(subflow);
+	ptr = bpf_mptcp_storage_get(&mptcp_rr_map, msk, 0, BPF_LOCAL_STORAGE_GET_F_CREATE);
+	if (ptr)
+		ptr->last_snd = last_snd;
 	return 0;
 }
 
-- 
2.35.3

[PATCH mptcp-next v10 13/13] Squash to "selftests/bpf: Add bpf_red scheduler"

[Geliang Tang]

Update API.

Signed-off-by: Geliang Tang <geliang.tang@suse.com>
---
 tools/testing/selftests/bpf/progs/mptcp_bpf_red.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/tools/testing/selftests/bpf/progs/mptcp_bpf_red.c b/tools/testing/selftests/bpf/progs/mptcp_bpf_red.c
index 30dd6f521b7f..52f40e39fb07 100644
--- a/tools/testing/selftests/bpf/progs/mptcp_bpf_red.c
+++ b/tools/testing/selftests/bpf/progs/mptcp_bpf_red.c
@@ -16,20 +16,20 @@ void BPF_PROG(mptcp_sched_red_release, const struct mptcp_sock *msk)
 {
 }
 
-void BPF_STRUCT_OPS(bpf_red_data_init, const struct mptcp_sock *msk,
+void BPF_STRUCT_OPS(bpf_red_data_init, struct mptcp_sock *msk,
 		    struct mptcp_sched_data *data)
 {
 	mptcp_sched_data_set_contexts(msk, data);
 }
 
-int BPF_STRUCT_OPS(bpf_red_get_subflow, const struct mptcp_sock *msk,
-		   struct mptcp_sched_data *data)
+int BPF_STRUCT_OPS(bpf_red_get_subflow, struct mptcp_sock *msk,
+		   const struct mptcp_sched_data *data)
 {
-	for (int i = 0; i < MPTCP_SUBFLOWS_MAX; i++) {
-		if (!data->contexts[i])
+	for (int i = 0; i < data->subflows && i < MPTCP_SUBFLOWS_MAX; i++) {
+		if (!mptcp_subflow_ctx_by_pos(data, i))
 			break;
 
-		mptcp_subflow_set_scheduled(data->contexts[i], true);
+		mptcp_subflow_set_scheduled(mptcp_subflow_ctx_by_pos(data, i), true);
 	}
 
 	return 0;
-- 
2.35.3

Re: Squash to "selftests/bpf: Add bpf_red scheduler": Build Failure

[MPTCP CI]

Hi Geliang,

Thank you for your modifications, that's great!

But sadly, our CI spotted some issues with it when trying to build it.

You can find more details there:

  https://patchwork.kernel.org/project/mptcp/patch/cae101ed32a7f97ffcd7d4e25e7c96cd85819e87.1687746021.git.geliang.tang@suse.com/
  https://github.com/multipath-tcp/mptcp_net-next/actions/runs/5373236226

Status: failure
Initiator: MPTCPimporter
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/d70b6a579bb6

Feel free to reply to this email if you cannot access logs, if you need
some support to fix the error, if this doesn't seem to be caused by your
modifications or if the error is a false positive one.

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: Squash to "selftests/bpf: Add bpf_red scheduler": Tests Results

[MPTCP CI]

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join):
  - Script error! ❓:
  - Task: https://cirrus-ci.com/task/6610932755857408
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/6610932755857408/summary/summary.txt

- KVM Validation: normal (only selftest_mptcp_join):
  - Script error! ❓:
  - Task: https://cirrus-ci.com/task/4781345407238144
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/4781345407238144/summary/summary.txt

- KVM Validation: debug (except selftest_mptcp_join):
  - Script error! ❓:
  - Task: https://cirrus-ci.com/task/5907245314080768
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5907245314080768/summary/summary.txt

- KVM Validation: debug (only selftest_mptcp_join):
  - Script error! ❓:
  - Task: https://cirrus-ci.com/task/5344295360659456
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5344295360659456/summary/summary.txt

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/d70b6a579bb6


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: Squash to "selftests/bpf: Add bpf_red scheduler": Tests Results

[Matthieu Baerts]

Hi Geliang,

On 26/06/2023 05:07, MPTCP CI wrote:
> Hi Geliang,
> 
> Thank you for your modifications, that's great!
> 
> Our CI did some validations and here is its report:
> 
> - KVM Validation: normal (except selftest_mptcp_join):
>   - Script error! ❓:
>   - Task: https://cirrus-ci.com/task/6610932755857408

Please note that these errors are due to issues in net-next:

https://lore.kernel.org/r/20230626112847.2ef3d422@canb.auug.org.au/

I just applied a fix in our tree. I also asked Patchew to re-apply the
patches on top of the latest export branch but I don't think if this
will work.


Please also note that the "Build Failure" email you also got is *not*
linked to these errors, they are likely linked to the modifications you did.

https://github.com/multipath-tcp/mptcp_net-next/actions/runs/5373236226

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net

Re: [PATCH mptcp-next v10 02/13] Squash to "mptcp: add struct mptcp_sched_ops"

[Geliang Tang]

On Mon, Jun 26, 2023 at 10:23:03AM +0800, Geliang Tang wrote:
> - Add subflows in mptcp_sched_data.
> 
> - Drop const before msk in data_init() and get_subflow(), since
> msk->bpf_storage needs to be writable.

No need to change msk type, we can keep it as const:

#define MPTCP_SCHED_NAME_MAX    16
#define MPTCP_SUBFLOWS_MAX      8

struct mptcp_sched_data {
        bool    reinject;
        u8      subflows;
        struct mptcp_subflow_context *contexts[MPTCP_SUBFLOWS_MAX];
};

struct mptcp_sched_ops {
        void (*data_init)(const struct mptcp_sock *msk,
                          struct mptcp_sched_data *data);
        int (*get_subflow)(struct mptcp_sock *msk,
                           const struct mptcp_sched_data *data);

        char                    name[MPTCP_SCHED_NAME_MAX];
        struct module           *owner;
        struct list_head        list;

        void (*init)(const struct mptcp_sock *msk);
        void (*release)(const struct mptcp_sock *msk);
} ____cacheline_aligned_in_smp;

Will update in v11.

-Geliang

> 
> - Add const before data in get_subflow().
> 
> Signed-off-by: Geliang Tang <geliang.tang@suse.com>
> ---
>  include/net/mptcp.h | 11 ++++++-----
>  1 file changed, 6 insertions(+), 5 deletions(-)
> 
> diff --git a/include/net/mptcp.h b/include/net/mptcp.h
> index 828b10ddabee..ba0e1ee68a05 100644
> --- a/include/net/mptcp.h
> +++ b/include/net/mptcp.h
> @@ -101,21 +101,22 @@ struct mptcp_out_options {
>  
>  struct mptcp_sched_data {
>  	bool	reinject;
> +	u8	subflows;
>  	struct mptcp_subflow_context *contexts[MPTCP_SUBFLOWS_MAX];
>  };
>  
>  struct mptcp_sched_ops {
> -	void (*data_init)(const struct mptcp_sock *msk,
> +	void (*data_init)(struct mptcp_sock *msk,
>  			  struct mptcp_sched_data *data);
> -	int (*get_subflow)(const struct mptcp_sock *msk,
> -			   struct mptcp_sched_data *data);
> +	int (*get_subflow)(struct mptcp_sock *msk,
> +			   const struct mptcp_sched_data *data);
>  
>  	char			name[MPTCP_SCHED_NAME_MAX];
>  	struct module		*owner;
>  	struct list_head	list;
>  
> -	void (*init)(const struct mptcp_sock *msk);
> -	void (*release)(const struct mptcp_sock *msk);
> +	void (*init)(struct mptcp_sock *msk);
> +	void (*release)(struct mptcp_sock *msk);
>  } ____cacheline_aligned_in_smp;
>  
>  #ifdef CONFIG_MPTCP
> -- 
> 2.35.3
> 

Re: Squash to "selftests/bpf: Add bpf_red scheduler": Build Failure

[MPTCP CI]

Hi Geliang,

Thank you for your modifications, that's great!

But sadly, our CI spotted some issues with it when trying to build it.

You can find more details there:

  https://patchwork.kernel.org/project/mptcp/patch/cae101ed32a7f97ffcd7d4e25e7c96cd85819e87.1687746021.git.geliang.tang@suse.com/
  https://github.com/multipath-tcp/mptcp_net-next/actions/runs/5376448855

Status: failure
Initiator: MPTCPimporter
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/be983431b4a0

Feel free to reply to this email if you cannot access logs, if you need
some support to fix the error, if this doesn't seem to be caused by your
modifications or if the error is a false positive one.

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)

Re: [PATCH mptcp-next v10 08/13] bpf: Add mptcp_storage helpers

[Geliang Tang]

On Mon, Jun 26, 2023 at 10:23:09AM +0800, Geliang Tang wrote:
> Add two helpers bpf_mptcp_storage_get() and bpf_mptcp_storage_delete()
> for mptcp_storage map type to get or delete a bpf_local_storage from
> the given msk.
> 
> Signed-off-by: Geliang Tang <geliang.tang@suse.com>
> ---
>  include/linux/bpf.h            |  2 +
>  include/linux/btf_ids.h        |  1 +
>  include/uapi/linux/bpf.h       | 39 ++++++++++++++++
>  kernel/bpf/helpers.c           |  4 ++
>  kernel/bpf/verifier.c          | 13 +++++-
>  net/mptcp/bpf.c                | 84 ++++++++++++++++++++++++++++++++++
>  tools/include/uapi/linux/bpf.h | 39 ++++++++++++++++
>  7 files changed, 181 insertions(+), 1 deletion(-)
> 
> diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> index f58895830ada..ac4b5748451a 100644
> --- a/include/linux/bpf.h
> +++ b/include/linux/bpf.h
> @@ -2909,6 +2909,8 @@ extern const struct bpf_func_proto bpf_get_retval_proto;
>  extern const struct bpf_func_proto bpf_user_ringbuf_drain_proto;
>  extern const struct bpf_func_proto bpf_cgrp_storage_get_proto;
>  extern const struct bpf_func_proto bpf_cgrp_storage_delete_proto;
> +extern const struct bpf_func_proto bpf_mptcp_storage_get_proto;
> +extern const struct bpf_func_proto bpf_mptcp_storage_delete_proto;
>  
>  const struct bpf_func_proto *tracing_prog_func_proto(
>    enum bpf_func_id func_id, const struct bpf_prog *prog);
> diff --git a/include/linux/btf_ids.h b/include/linux/btf_ids.h
> index 00950cc03bff..abd56cfcb03f 100644
> --- a/include/linux/btf_ids.h
> +++ b/include/linux/btf_ids.h
> @@ -267,5 +267,6 @@ MAX_BTF_TRACING_TYPE,
>  extern u32 btf_tracing_ids[];
>  extern u32 bpf_cgroup_btf_id[];
>  extern u32 bpf_local_storage_map_btf_id[];
> +extern u32 bpf_mptcp_btf_id[];
>  
>  #endif
> diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
> index f43cf69e47ea..137e0620fd69 100644
> --- a/include/uapi/linux/bpf.h
> +++ b/include/uapi/linux/bpf.h
> @@ -5570,6 +5570,43 @@ union bpf_attr {
>   *		0 on success.
>   *
>   *		**-ENOENT** if the bpf_local_storage cannot be found.
> + *
> + * void *bpf_mptcp_storage_get(struct bpf_map *map, struct mptcp_sock *msk, void *value, u64 flags)
> + *	Description
> + *		Get a bpf_local_storage from the *msk*.
> + *
> + *		Logically, it could be thought of as getting the value from
> + *		a *map* with *msk* as the **key**.  From this
> + *		perspective,  the usage is not much different from
> + *		**bpf_map_lookup_elem**\ (*map*, **&**\ *msk*) except this
> + *		helper enforces the key must be a mptcp_sock and the map must also
> + *		be a **BPF_MAP_TYPE_MPTCP_STORAGE**.
> + *
> + *		Underneath, the value is stored locally at *msk* instead of
> + *		the *map*.  The *map* is used as the bpf-local-storage
> + *		"type". The bpf-local-storage "type" (i.e. the *map*) is
> + *		searched against all bpf_local_storage residing at *msk*.
> + *
> + *		An optional *flags* (**BPF_LOCAL_STORAGE_GET_F_CREATE**) can be
> + *		used such that a new bpf_local_storage will be
> + *		created if one does not exist.  *value* can be used
> + *		together with **BPF_LOCAL_STORAGE_GET_F_CREATE** to specify
> + *		the initial value of a bpf_local_storage.  If *value* is
> + *		**NULL**, the new bpf_local_storage will be zero initialized.
> + *	Return
> + *		A bpf_local_storage pointer is returned on success.
> + *
> + *		**NULL** if not found or there was an error in adding
> + *		a new bpf_local_storage.
> + *
> + * long bpf_mptcp_storage_delete(struct bpf_map *map, struct mptcp_sock *msk)
> + *	Description
> + *		Delete a bpf_local_storage from the *msk*.
> + *
> + *	Return
> + *		0 on success.
> + *
> + *		**-ENOENT** if the bpf_local_storage cannot be found.
>   */
>  #define ___BPF_FUNC_MAPPER(FN, ctx...)			\
>  	FN(unspec, 0, ##ctx)				\
> @@ -5784,6 +5821,8 @@ union bpf_attr {
>  	FN(user_ringbuf_drain, 209, ##ctx)		\
>  	FN(cgrp_storage_get, 210, ##ctx)		\
>  	FN(cgrp_storage_delete, 211, ##ctx)		\
> +	FN(mptcp_storage_get, 212, ##ctx)		\
> +	FN(mptcp_storage_delete, 213, ##ctx)		\
>  	/* */
>  
>  /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't
> diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
> index 4ef4c4f8a355..381b680007fb 100644
> --- a/kernel/bpf/helpers.c
> +++ b/kernel/bpf/helpers.c
> @@ -1768,6 +1768,10 @@ bpf_base_func_proto(enum bpf_func_id func_id)
>  	case BPF_FUNC_get_current_ancestor_cgroup_id:
>  		return &bpf_get_current_ancestor_cgroup_id_proto;
>  #endif
> +	case BPF_FUNC_mptcp_storage_get:
> +		return &bpf_mptcp_storage_get_proto;
> +	case BPF_FUNC_mptcp_storage_delete:
> +		return &bpf_mptcp_storage_delete_proto;
>  	default:
>  		break;
>  	}
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index d8482a4ef465..c7479628acf7 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -553,7 +553,8 @@ static bool is_storage_get_function(enum bpf_func_id func_id)
>  	return func_id == BPF_FUNC_sk_storage_get ||
>  	       func_id == BPF_FUNC_inode_storage_get ||
>  	       func_id == BPF_FUNC_task_storage_get ||
> -	       func_id == BPF_FUNC_cgrp_storage_get;
> +	       func_id == BPF_FUNC_cgrp_storage_get ||
> +	       func_id == BPF_FUNC_mptcp_storage_get;
>  }
>  
>  static bool helper_multiple_ref_obj_use(enum bpf_func_id func_id,
> @@ -8241,6 +8242,11 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env,
>  		    func_id != BPF_FUNC_map_push_elem)
>  			goto error;
>  		break;
> +	case BPF_MAP_TYPE_MPTCP_STORAGE:
> +		if (func_id != BPF_FUNC_mptcp_storage_get &&
> +		    func_id != BPF_FUNC_mptcp_storage_delete)
> +			goto error;
> +		break;
>  	default:
>  		break;
>  	}
> @@ -8353,6 +8359,11 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env,
>  		if (map->map_type != BPF_MAP_TYPE_CGRP_STORAGE)
>  			goto error;
>  		break;
> +	case BPF_FUNC_mptcp_storage_get:
> +	case BPF_FUNC_mptcp_storage_delete:
> +		if (map->map_type != BPF_MAP_TYPE_MPTCP_STORAGE)
> +			goto error;
> +		break;
>  	default:
>  		break;
>  	}
> diff --git a/net/mptcp/bpf.c b/net/mptcp/bpf.c
> index 764c670e843b..0bf3409cf288 100644
> --- a/net/mptcp/bpf.c
> +++ b/net/mptcp/bpf.c
> @@ -201,6 +201,17 @@ static void bpf_mptcp_storage_unlock(void)
>  	migrate_enable();
>  }
>  
> +static bool bpf_mptcp_storage_trylock(void)
> +{
> +	migrate_disable();
> +	if (unlikely(this_cpu_inc_return(bpf_mptcp_storage_busy) != 1)) {
> +		this_cpu_dec(bpf_mptcp_storage_busy);
> +		migrate_enable();
> +		return false;
> +	}
> +	return true;
> +}
> +
>  static struct bpf_local_storage __rcu **mptcp_storage_ptr(void *owner)
>  {
>  	struct mptcp_sock *msk = owner;
> @@ -305,6 +316,59 @@ static long bpf_mptcp_storage_delete_elem(struct bpf_map *map, void *key)
>  	return ret;
>  }
>  
> +/* Called by bpf_mptcp_storage_get*() helpers */
> +static void *__bpf_mptcp_storage_get(struct bpf_map *map,
> +				     struct mptcp_sock *msk, void *value,
> +				     u64 flags, gfp_t gfp_flags, bool nobusy)
> +{
> +        struct bpf_local_storage_data *sdata;
> +
> +        sdata = mptcp_storage_lookup(msk, map, nobusy);
> +        if (sdata)
> +                return sdata->data;
> +
> +        if ((flags & BPF_LOCAL_STORAGE_GET_F_CREATE) && nobusy) {
> +                sdata = bpf_local_storage_update(
> +                        msk, (struct bpf_local_storage_map *)map, value,
> +                        BPF_NOEXIST, gfp_flags);
> +                return IS_ERR(sdata) ? NULL : sdata->data;
> +        }
> +
> +        return NULL;
> +}
> +
> +/* *gfp_flags* is a hidden argument provided by the verifier */
> +BPF_CALL_5(bpf_mptcp_storage_get, struct bpf_map *, map, struct mptcp_sock *, msk,
> +	   void *, value, u64, flags, gfp_t, gfp_flags)
> +{
> +	void *data;
> +
> +	WARN_ON_ONCE(!bpf_rcu_lock_held());
> +	if (flags & ~BPF_LOCAL_STORAGE_GET_F_CREATE || !msk)
> +		return (unsigned long)NULL;
> +
> +	bpf_mptcp_storage_lock();
> +	data = __bpf_mptcp_storage_get(map, msk, value, flags, gfp_flags, true);
> +	bpf_mptcp_storage_unlock();
> +	return (unsigned long)data;
> +}
> +
> +BPF_CALL_2(bpf_mptcp_storage_delete, struct bpf_map *, map, struct mptcp_sock *, msk)
> +{
> +	int ret;
> +
> +	WARN_ON_ONCE(!bpf_rcu_lock_held());
> +	if (!msk)
> +		return -EINVAL;
> +
> +	if (!bpf_mptcp_storage_trylock())
> +		return -EBUSY;
> +
> +	ret = mptcp_storage_delete(msk, map);
> +	bpf_mptcp_storage_unlock();
> +	return ret;
> +}
> +
>  static int notsupp_get_next_key(struct bpf_map *map, void *key, void *next_key)
>  {
>  	return -ENOTSUPP;
> @@ -334,6 +398,26 @@ const struct bpf_map_ops mptcp_storage_map_ops = {
>  	.map_btf_id = &bpf_local_storage_map_btf_id[0],
>  	.map_owner_storage_ptr = mptcp_storage_ptr,
>  };
> +
> +const struct bpf_func_proto bpf_mptcp_storage_get_proto = {
> +	.func = bpf_mptcp_storage_get,
> +	.gpl_only = false,
> +	.ret_type = RET_PTR_TO_MAP_VALUE_OR_NULL,
> +	.arg1_type = ARG_CONST_MAP_PTR,
> +	.arg2_type = ARG_PTR_TO_BTF_ID_OR_NULL,
> +	.arg2_btf_id = &bpf_mptcp_btf_id[0],

No need to add this bpf_mptcp_btf_id, we can use
&btf_sock_ids[BTF_SOCK_TYPE_MPTCP] instead.

> +	.arg3_type = ARG_PTR_TO_MAP_VALUE_OR_NULL,
> +	.arg4_type = ARG_ANYTHING,
> +};
> +
> +const struct bpf_func_proto bpf_mptcp_storage_delete_proto = {
> +	.func           = bpf_mptcp_storage_delete,
> +	.gpl_only       = false,
> +	.ret_type       = RET_INTEGER,
> +	.arg1_type      = ARG_CONST_MAP_PTR,
> +	.arg2_type      = ARG_PTR_TO_BTF_ID_OR_NULL,
> +	.arg2_btf_id    = &bpf_mptcp_btf_id[0],

Add here too.

-Geliang

> +};
>  #endif /* CONFIG_BPF_JIT */
>  
>  struct mptcp_sock *bpf_mptcp_sock_from_subflow(struct sock *sk)
> diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
> index f43cf69e47ea..137e0620fd69 100644
> --- a/tools/include/uapi/linux/bpf.h
> +++ b/tools/include/uapi/linux/bpf.h
> @@ -5570,6 +5570,43 @@ union bpf_attr {
>   *		0 on success.
>   *
>   *		**-ENOENT** if the bpf_local_storage cannot be found.
> + *
> + * void *bpf_mptcp_storage_get(struct bpf_map *map, struct mptcp_sock *msk, void *value, u64 flags)
> + *	Description
> + *		Get a bpf_local_storage from the *msk*.
> + *
> + *		Logically, it could be thought of as getting the value from
> + *		a *map* with *msk* as the **key**.  From this
> + *		perspective,  the usage is not much different from
> + *		**bpf_map_lookup_elem**\ (*map*, **&**\ *msk*) except this
> + *		helper enforces the key must be a mptcp_sock and the map must also
> + *		be a **BPF_MAP_TYPE_MPTCP_STORAGE**.
> + *
> + *		Underneath, the value is stored locally at *msk* instead of
> + *		the *map*.  The *map* is used as the bpf-local-storage
> + *		"type". The bpf-local-storage "type" (i.e. the *map*) is
> + *		searched against all bpf_local_storage residing at *msk*.
> + *
> + *		An optional *flags* (**BPF_LOCAL_STORAGE_GET_F_CREATE**) can be
> + *		used such that a new bpf_local_storage will be
> + *		created if one does not exist.  *value* can be used
> + *		together with **BPF_LOCAL_STORAGE_GET_F_CREATE** to specify
> + *		the initial value of a bpf_local_storage.  If *value* is
> + *		**NULL**, the new bpf_local_storage will be zero initialized.
> + *	Return
> + *		A bpf_local_storage pointer is returned on success.
> + *
> + *		**NULL** if not found or there was an error in adding
> + *		a new bpf_local_storage.
> + *
> + * long bpf_mptcp_storage_delete(struct bpf_map *map, struct mptcp_sock *msk)
> + *	Description
> + *		Delete a bpf_local_storage from the *msk*.
> + *
> + *	Return
> + *		0 on success.
> + *
> + *		**-ENOENT** if the bpf_local_storage cannot be found.
>   */
>  #define ___BPF_FUNC_MAPPER(FN, ctx...)			\
>  	FN(unspec, 0, ##ctx)				\
> @@ -5784,6 +5821,8 @@ union bpf_attr {
>  	FN(user_ringbuf_drain, 209, ##ctx)		\
>  	FN(cgrp_storage_get, 210, ##ctx)		\
>  	FN(cgrp_storage_delete, 211, ##ctx)		\
> +	FN(mptcp_storage_get, 212, ##ctx)		\
> +	FN(mptcp_storage_delete, 213, ##ctx)		\
>  	/* */
>  
>  /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't
> -- 
> 2.35.3
> 

Re: Squash to "selftests/bpf: Add bpf_red scheduler": Tests Results

[MPTCP CI]

Hi Geliang,

Thank you for your modifications, that's great!

Our CI did some validations and here is its report:

- KVM Validation: normal (except selftest_mptcp_join):
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/5075409738399744
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5075409738399744/summary/summary.txt

- KVM Validation: debug (except selftest_mptcp_join):
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/4793934761689088
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/4793934761689088/summary/summary.txt

- KVM Validation: debug (only selftest_mptcp_join):
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/5919834668531712
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/5919834668531712/summary/summary.txt

- KVM Validation: normal (only selftest_mptcp_join):
  - Success! ✅:
  - Task: https://cirrus-ci.com/task/6201309645242368
  - Summary: https://api.cirrus-ci.com/v1/artifact/task/6201309645242368/summary/summary.txt

Initiator: Patchew Applier
Commits: https://github.com/multipath-tcp/mptcp_net-next/commits/be983431b4a0


If there are some issues, you can reproduce them using the same environment as
the one used by the CI thanks to a docker image, e.g.:

    $ cd [kernel source code]
    $ docker run -v "${PWD}:${PWD}:rw" -w "${PWD}" --privileged --rm -it \
        --pull always mptcp/mptcp-upstream-virtme-docker:latest \
        auto-debug

For more details:

    https://github.com/multipath-tcp/mptcp-upstream-virtme-docker


Please note that despite all the efforts that have been already done to have a
stable tests suite when executed on a public CI like here, it is possible some
reported issues are not due to your modifications. Still, do not hesitate to
help us improve that ;-)

Cheers,
MPTCP GH Action bot
Bot operated by Matthieu Baerts (Tessares)