Re: [PATCH bpf-next v4 6/7] bpf, net: Support SO_REUSEPORT sockets with bpf_sk_assign

[Kuniyuki Iwashima <kuniyu@amazon.com>]

From: Lorenz Bauer <lmb@isovalent.com>
Date: Mon, 3 Jul 2023 10:57:23 +0100
> On Wed, Jun 28, 2023 at 7:54 PM Kuniyuki Iwashima <kuniyu@amazon.com> wrote:
> 
> > > +     reuse_sk = inet6_lookup_reuseport(net, sk, skb, doff,
> > > +                                       saddr, sport, daddr, ntohs(dport),
> > > +                                       ehashfn);
> > > +     if (!reuse_sk || reuse_sk == sk)
> > > +             return sk;
> > > +
> > > +     /* We've chosen a new reuseport sock which is never refcounted. This
> > > +      * implies that sk also isn't refcounted.
> > > +      */
> > > +     WARN_ON_ONCE(*refcounted);
> >
> > One more nit.
> >
> > WARN_ON_ONCE() should be tested before inet6?_lookup_reuseport() not to
> > miss the !reuse_sk case.
> 
> I was just pondering that as well, but I came to the opposite
> conclusion. In the !reuse_sk case we don't really know anything about
> sk, except that it isn't part of a reuseport group. How can we be sure
> that it's not refcounted?

Sorry for late reply.

What we know about sk before inet6?_lookup_reuseport() are

  (1) sk was full socket in bpf_sk_assign()
  (2) sk had SOCK_RCU_FREE in bpf_sk_assign()
  (3) sk was TCP_LISTEN here if TCP

After bpf_sk_assign(), reqsk is never converted to fullsock, and UDP
never clears SOCK_RCU_FREE.  If sk is TCP, now we are in the RCU grace
period and confirmed sk->sk_state was TCP_LISTEN.  Then, TCP_LISTEN sk
cannot be reused and SOCK_RCU_FREE is never cleared.

So, before/after inet6?_lookup_reuseport(), the fact that sk is not
refcounted here should not change in spite of that reuse_sk is NULL.

What do you think ?