From: Jakub Sitnicki <jakub@cloudflare.com>
To: Eric Dumazet <eric.dumazet@gmail.com>,
John Fastabend <john.fastabend@gmail.com>
Cc: bpf@vger.kernel.org, netdev@vger.kernel.org,
kernel-team@cloudflare.com,
John Fastabend <john.fastabend@gmail.com>,
Lorenz Bauer <lmb@cloudflare.com>, Martin Lau <kafai@fb.com>
Subject: Re: [PATCH bpf-next v4 02/12] net, sk_msg: Annotate lockless access to sk_prot on clone
Date: Thu, 23 Jan 2020 19:56:53 +0100 [thread overview]
Message-ID: <874kwm2e8a.fsf@cloudflare.com> (raw)
In-Reply-To: <a6bf279e-a998-84ab-4371-cd6c1ccbca5d@gmail.com>
On Thu, Jan 23, 2020 at 06:18 PM CET, Eric Dumazet wrote:
> On 1/23/20 7:55 AM, Jakub Sitnicki wrote:
>> sk_msg and ULP frameworks override protocol callbacks pointer in
>> sk->sk_prot, while tcp accesses it locklessly when cloning the listening
>> socket, that is with neither sk_lock nor sk_callback_lock held.
>>
>> Once we enable use of listening sockets with sockmap (and hence sk_msg),
>> there will be shared access to sk->sk_prot if socket is getting cloned
>> while being inserted/deleted to/from the sockmap from another CPU:
>>
>> Read side:
>>
>> tcp_v4_rcv
>> sk = __inet_lookup_skb(...)
>> tcp_check_req(sk)
>> inet_csk(sk)->icsk_af_ops->syn_recv_sock
>> tcp_v4_syn_recv_sock
>> tcp_create_openreq_child
>> inet_csk_clone_lock
>> sk_clone_lock
>> READ_ONCE(sk->sk_prot)
>>
>> Write side:
>>
>> sock_map_ops->map_update_elem
>> sock_map_update_elem
>> sock_map_update_common
>> sock_map_link_no_progs
>> tcp_bpf_init
>> tcp_bpf_update_sk_prot
>> sk_psock_update_proto
>> WRITE_ONCE(sk->sk_prot, ops)
>>
>> sock_map_ops->map_delete_elem
>> sock_map_delete_elem
>> __sock_map_delete
>> sock_map_unref
>> sk_psock_put
>> sk_psock_drop
>> sk_psock_restore_proto
>> tcp_update_ulp
>> WRITE_ONCE(sk->sk_prot, proto)
>>
>> Mark the shared access with READ_ONCE/WRITE_ONCE annotations.
>>
>> Acked-by: Martin KaFai Lau <kafai@fb.com>
>> Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
>> ---
>> include/linux/skmsg.h | 3 ++-
>> net/core/sock.c | 5 +++--
>> net/ipv4/tcp_bpf.c | 4 +++-
>> net/ipv4/tcp_ulp.c | 3 ++-
>> net/tls/tls_main.c | 3 ++-
>> 5 files changed, 12 insertions(+), 6 deletions(-)
>>
>> diff --git a/include/linux/skmsg.h b/include/linux/skmsg.h
>> index 41ea1258d15e..55c834a5c25e 100644
>> --- a/include/linux/skmsg.h
>> +++ b/include/linux/skmsg.h
>> @@ -352,7 +352,8 @@ static inline void sk_psock_update_proto(struct sock *sk,
>> psock->saved_write_space = sk->sk_write_space;
>>
>> psock->sk_proto = sk->sk_prot;
>> - sk->sk_prot = ops;
>> + /* Pairs with lockless read in sk_clone_lock() */
>> + WRITE_ONCE(sk->sk_prot, ops);
>
>
> Note there are dozens of calls like
>
> if (sk->sk_prot->handler)
> sk->sk_prot->handler(...);
>
> Some of them being done lockless.
>
> I know it is painful, but presumably we need
>
> const struct proto *ops = READ_ONCE(sk->sk_prot);
>
> if (ops->handler)
> ops->handler(....);
Yikes! That will be quite an audit. Thank you for taking a look.
Now I think I understand what John had in mind when asking for pushing
these annotations to the bpf tree as well [0].
Considering these are lacking today, can I do it as a follow up?
[0] https://lore.kernel.org/bpf/20200110105027.257877-1-jakub@cloudflare.com/T/#m6a4f84a922a393719a7ea7b33dafdb6c66b72827
next prev parent reply other threads:[~2020-01-23 18:56 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-23 15:55 [PATCH bpf-next v4 00/12] Extend SOCKMAP to store listening sockets Jakub Sitnicki
2020-01-23 15:55 ` [PATCH bpf-next v4 01/12] bpf, sk_msg: Don't clear saved sock proto on restore Jakub Sitnicki
2020-01-23 15:55 ` [PATCH bpf-next v4 02/12] net, sk_msg: Annotate lockless access to sk_prot on clone Jakub Sitnicki
2020-01-23 17:18 ` Eric Dumazet
2020-01-23 18:56 ` Jakub Sitnicki [this message]
2020-01-23 19:15 ` John Fastabend
2020-01-27 9:36 ` Jakub Sitnicki
2020-01-23 15:55 ` [PATCH bpf-next v4 03/12] net, sk_msg: Clear sk_user_data pointer on clone if tagged Jakub Sitnicki
2020-01-23 15:55 ` [PATCH bpf-next v4 04/12] tcp_bpf: Don't let child socket inherit parent protocol ops on copy Jakub Sitnicki
2020-01-23 15:55 ` [PATCH bpf-next v4 05/12] bpf, sockmap: Allow inserting listening TCP sockets into sockmap Jakub Sitnicki
2020-01-23 15:55 ` [PATCH bpf-next v4 06/12] bpf, sockmap: Don't set up sockmap progs for listening sockets Jakub Sitnicki
2020-01-23 15:55 ` [PATCH bpf-next v4 07/12] bpf, sockmap: Return socket cookie on lookup from syscall Jakub Sitnicki
2020-01-23 15:55 ` [PATCH bpf-next v4 08/12] bpf, sockmap: Let all kernel-land lookup values in SOCKMAP Jakub Sitnicki
2020-01-23 15:55 ` [PATCH bpf-next v4 09/12] bpf: Allow selecting reuseport socket from a SOCKMAP Jakub Sitnicki
2020-01-23 15:55 ` [PATCH bpf-next v4 10/12] net: Generate reuseport group ID on group creation Jakub Sitnicki
2020-01-23 17:02 ` Martin Lau
2020-01-23 15:55 ` [PATCH bpf-next v4 11/12] selftests/bpf: Extend SK_REUSEPORT tests to cover SOCKMAP Jakub Sitnicki
2020-01-23 15:55 ` [PATCH bpf-next v4 12/12] selftests/bpf: Tests for SOCKMAP holding listening sockets Jakub Sitnicki
2020-01-27 11:13 ` [PATCH bpf-next v4 00/12] Extend SOCKMAP to store " Daniel Borkmann
2020-01-27 13:03 ` Jakub Sitnicki
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=874kwm2e8a.fsf@cloudflare.com \
--to=jakub@cloudflare.com \
--cc=bpf@vger.kernel.org \
--cc=eric.dumazet@gmail.com \
--cc=john.fastabend@gmail.com \
--cc=kafai@fb.com \
--cc=kernel-team@cloudflare.com \
--cc=lmb@cloudflare.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.