Re: [PATCH mptcp-next v2 14/36] mptcp: refactor dump_addr with id bitmap

[Matthieu Baerts <matttbe@kernel.org>]

Hi Geliang,

(I only see this patch now, and better understand the reaction you had
in [1])

[1]
https://lore.kernel.org/mptcp/20241025-mptcp-pm-lookup_addr_rcu-v2-2-1478f6c4b205@kernel.org/T/#u

On 22/10/2024 11:14, Geliang Tang wrote:
> From: Geliang Tang <tanggeliang@kylinos.cn>
> Subject: mptcp: refactor dump_addr with id bitmap

Be careful when you use this 'refactor' word: if I'm not mistaken,
generally it means that you modified the code around to ease integration
of a new feature or the maintenance, but without changing the behaviour
or fixing anything. Here, yes you moved the code around, but to fix
something, to change the behaviour.

> With the help of get_addr(), we can refactor dump_addr() interfaces to
> reuse send_nlmsg code between the netlink PM and userspace PM.
> 
> The current dump_addr() flow looks like this:
> 
> 	lock();
> 	for_each_entry(entry)
> 		send_nlmsg(entry);
> 	unlock();
> 
> After holding the lock, get every entry by walking the address list,
> send each one looply, and finally release the lock.
> 
> This set changes the process by copying the address list to an id
> bitmap while holding the lock, then release the lock immediately.
> After that, without locking, walking the copied id bitmap to get
> every copy of entry by using get_addr(), and send each one looply.
> 
> This patch is the first part of refactoring dump_addr().
> 
> Without changing the position of the locks, the dump process is split
> into two parts: copying the ID bitmap first, and then traversing the
> ID bitmap, use lookup_addr_by_id() to get the entry, then send each
> one through nlmsg:
> 
> 	lock();
> 	for_each_entry(entry)
> 		set_bit(bitmap);
> 	for_each_bit(bitmap) {
> 		entry = lookup_addr_by_id();
> 		send_nlmsg(entry);
> 	}
> 	unlock();

After the discussions we had on [1], do you still think this
modification is needed? Do you do that for the consistency, to unify the
code, or because you require this for the BPF PM?

If it is for the consistency, please check the last question from Mat on
[1]:

> Do you see a reason that bitmap/addr_list inconsistencies would cause issues?

Also, with the modification here (mainly with the next patch), can we
not have more inconsistencies? The bitmap can point to ID that have been
freed/should not be read, no?

> 
> Signed-off-by: Geliang Tang <tanggeliang@kylinos.cn>
> ---
>  net/mptcp/pm_netlink.c   |  6 ++++-
>  net/mptcp/pm_userspace.c | 54 +++++++++++++++++++++++++++++-----------
>  2 files changed, 45 insertions(+), 15 deletions(-)
> 
> diff --git a/net/mptcp/pm_netlink.c b/net/mptcp/pm_netlink.c
> index c030a79cd1dd..4cee5942200c 100644
> --- a/net/mptcp/pm_netlink.c
> +++ b/net/mptcp/pm_netlink.c
> @@ -1869,16 +1869,20 @@ static int mptcp_pm_nl_dump_addr(struct sk_buff *msg,
>  {
>  	struct net *net = sock_net(msg->sk);
>  	struct mptcp_pm_addr_entry *entry;
> +	struct mptcp_id_bitmap *bitmap;
>  	struct pm_nl_pernet *pernet;
>  	int id = cb->args[0];
>  	void *hdr;
>  	int i;
>  
> +	bitmap = (struct mptcp_id_bitmap *)cb->ctx;
>  	pernet = pm_nl_get_pernet(net);
>  
>  	spin_lock_bh(&pernet->lock);
> +	if (!id)
> +		bitmap_copy(bitmap->map, pernet->id_bitmap.map, MPTCP_PM_MAX_ADDR_ID + 1);
>  	for (i = id; i < MPTCP_PM_MAX_ADDR_ID + 1; i++) {
> -		if (test_bit(i, pernet->id_bitmap.map)) {
> +		if (test_bit(i, bitmap->map)) {
>  			entry = __lookup_addr_by_id(pernet, i);
>  			if (!entry)
>  				break;
> diff --git a/net/mptcp/pm_userspace.c b/net/mptcp/pm_userspace.c
> index cb4f2a174d0d..237383cfd12e 100644
> --- a/net/mptcp/pm_userspace.c
> +++ b/net/mptcp/pm_userspace.c
> @@ -583,6 +583,21 @@ int mptcp_userspace_pm_set_flags(struct sk_buff *skb, struct genl_info *info)
>  	return ret;
>  }
>  
> +static int mptcp_userspace_pm_set_bitmap(struct mptcp_sock *msk,
> +					 struct mptcp_id_bitmap *bitmap)
> +{
> +	struct mptcp_pm_addr_entry *entry;
> +
> +	mptcp_for_each_address(msk, entry) {
> +		if (test_bit(entry->addr.id, bitmap->map))
> +			continue;
> +
> +		__set_bit(entry->addr.id, bitmap->map);

It is not clear to me what you are trying to do here. Is this helper
(the name is not clear) only called the first time, then when the bitmap
is reset, no? Then why do you need the 'test_bit'?

> +	}
> +
> +	return 0;
> +}
> +
>  int mptcp_userspace_pm_dump_addr(struct sk_buff *msg,
>  				 struct netlink_callback *cb)
>  {
> @@ -590,9 +605,11 @@ int mptcp_userspace_pm_dump_addr(struct sk_buff *msg,
>  	struct mptcp_pm_addr_entry *entry;
>  	struct mptcp_id_bitmap *bitmap;
>  	struct mptcp_sock *msk;
> +	int id = cb->args[0];
>  	int ret = -EINVAL;
>  	struct sock *sk;
>  	void *hdr;
> +	int i;
>  
>  	bitmap = (struct mptcp_id_bitmap *)cb->ctx;
>  
> @@ -604,24 +621,33 @@ int mptcp_userspace_pm_dump_addr(struct sk_buff *msg,
>  
>  	lock_sock(sk);
>  	spin_lock_bh(&msk->pm.lock);
> -	mptcp_for_each_address(msk, entry) {
> -		if (test_bit(entry->addr.id, bitmap->map))
> -			continue;
> +	if (!id)
> +		ret = mptcp_userspace_pm_set_bitmap(msk, bitmap);
> +	for (i = id; i < MPTCP_PM_MAX_ADDR_ID + 1; i++) {
> +		if (test_bit(i, bitmap->map)) {
> +			entry = mptcp_userspace_pm_lookup_addr_by_id(msk, i);
> +			if (!entry)
> +				break;
> +
> +			if (id && entry->addr.id <= id)
> +				continue;
>  
> -		hdr = genlmsg_put(msg, NETLINK_CB(cb->skb).portid,
> -				  cb->nlh->nlmsg_seq, &mptcp_genl_family,
> -				  NLM_F_MULTI, MPTCP_PM_CMD_GET_ADDR);
> -		if (!hdr)
> -			break;
> +			hdr = genlmsg_put(msg, NETLINK_CB(cb->skb).portid,
> +					  cb->nlh->nlmsg_seq, &mptcp_genl_family,
> +					  NLM_F_MULTI, MPTCP_PM_CMD_GET_ADDR);
> +			if (!hdr)
> +				break;
>  
> -		if (mptcp_nl_fill_addr(msg, entry) < 0) {
> -			genlmsg_cancel(msg, hdr);
> -			break;
> -		}
> +			if (mptcp_nl_fill_addr(msg, entry) < 0) {
> +				genlmsg_cancel(msg, hdr);
> +				break;
> +			}
>  
> -		__set_bit(entry->addr.id, bitmap->map);
> -		genlmsg_end(msg, hdr);
> +			id = entry->addr.id;
> +			genlmsg_end(msg, hdr);
> +		}
>  	}
> +	cb->args[0] = id;
>  	spin_unlock_bh(&msk->pm.lock);
>  	release_sock(sk);
>  	ret = msg->len;

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.