RE: [bug report] one possible out-of-order issue in sockmap

[John Fastabend <john.fastabend@gmail.com>]

liujian (CE) wrote:
> 
> 
> > -----Original Message-----
> > From: Cong Wang [mailto:xiyou.wangcong@gmail.com]
> > Sent: Monday, September 26, 2022 2:26 AM
> > To: liujian (CE) <liujian56@huawei.com>
> > Cc: John Fastabend <john.fastabend@gmail.com>; Jakub Sitnicki
> > <jakub@cloudflare.com>; Eric Dumazet <edumazet@google.com>; davem
> > <davem@davemloft.net>; yoshfuji@linux-ipv6.org; dsahern@kernel.org;
> > Jakub Kicinski <kuba@kernel.org>; Paolo Abeni <pabeni@redhat.com>;
> > netdev <netdev@vger.kernel.org>; bpf@vger.kernel.org
> > Subject: Re: [bug report] one possible out-of-order issue in sockmap
> > 
> > On Sat, Sep 24, 2022 at 07:59:15AM +0000, liujian (CE) wrote:
> > > Hello,
> > >
> > > I had a scp failure problem here. I analyze the code, and the reasons may
> > be as follows:
> > >
> > > From commit e7a5f1f1cd00 ("bpf/sockmap: Read psock ingress_msg
> > before
> > > sk_receive_queue", if we use sockops
> > > (BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB
> > > and BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB) to enable socket's
> > sockmap
> > > function, and don't enable strparse and verdict function, the
> > > out-of-order problem may occur in the following process.
> > >
> > > client SK                                   server SK
> > > ----------------------------------------------------------------------
> > > ----
> > > tcp_rcv_synsent_state_process
> > >   tcp_finish_connect
> > >     tcp_init_transfer
> > >       tcp_set_state(sk, TCP_ESTABLISHED);
> > >       // insert SK to sockmap
> > >     wake up waitter
> > >     tcp_send_ack
> > >
> > > tcp_bpf_sendmsg(msgA)
> > > // msgA will go tcp stack
> > >                                             tcp_rcv_state_process
> > >                                               tcp_init_transfer
> > >                                                 //insert SK to sockmap
> > >                                               tcp_set_state(sk,
> > >                                                      TCP_ESTABLISHED)
> > >                                               wake up waitter
> > 
> > Here after the socket is inserted to a sockmap, its ->sk_data_ready() is
> > already replaced with sk_psock_verdict_data_ready(), so msgA should go to
> > sockmap, not TCP stack?
> > 
> It is TCP stack.  Here I only enable BPF_SK_MSG_VERDICT type.
> bpftool prog load bpf_redir.o /sys/fs/bpf/bpf_redir map name sock_ops_map pinned /sys/fs/bpf/sock_ops_map
> bpftool prog attach pinned /sys/fs/bpf/bpf_redir msg_verdict pinned /sys/fs/bpf/sock_ops_map

Is the sender using FAST_OPEN by any chance? We know this bug exists
in this case. Fix tbd.