Re: [RFC PATCH 0/2] udp: avoid false sharing on sk_tsflags

[Willem de Bruijn <willemdebruijn.kernel@gmail.com>]

Eric Dumazet wrote:
> On Mon, Feb 10, 2025 at 5:16 PM Paolo Abeni <pabeni@redhat.com> wrote:
> >
> > On 2/10/25 4:13 PM, Eric Dumazet wrote:
> > > On Mon, Feb 10, 2025 at 5:00 AM Willem de Bruijn
> > > <willemdebruijn.kernel@gmail.com> wrote:
> > >>
> > >> Paolo Abeni wrote:
> > >>> While benchmarking the recently shared page frag revert, I observed a
> > >>> lot of cache misses in the UDP RX path due to false sharing between the
> > >>> sk_tsflags and the sk_forward_alloc sk fields.
> > >>>
> > >>> Here comes a solution attempt for such a problem, inspired by commit
> > >>> f796feabb9f5 ("udp: add local "peek offset enabled" flag").
> > >>>
> > >>> The first patch adds a new proto op allowing protocol specific operation
> > >>> on tsflags updates, and the 2nd one leverages such operation to cache
> > >>> the problematic field in a cache friendly manner.
> > >>>
> > >>> The need for a new operation is possibly suboptimal, hence the RFC tag,
> > >>> but I could not find other good solutions. I considered:
> > >>> - moving the sk_tsflags just before 'sk_policy', in the 'sock_read_rxtx'
> > >>>   group. It arguably belongs to such group, but the change would create
> > >>>   a couple of holes, increasing the 'struct sock' size and would have
> > >>>   side effects on other protocols
> > >>> - moving the sk_tsflags just before 'sk_stamp'; similar to the above,
> > >>>   would possibly reduce the side effects, as most of 'struct sock'
> > >>>   layout will be unchanged. Could increase the number of cacheline
> > >>>   accessed in the TX path.
> > >>>
> > >>> I opted for the present solution as it should minimize the side effects
> > >>> to other protocols.
> > >>
> > >> The code looks solid at a high level to me.
> > >>
> > >> But if the issue can be adddressed by just moving a field, that is
> > >> quite appealing. So have no reviewed closely yet.
> > >>
> > >
> > > sk_tsflags has not been put in an optimal group, I would indeed move it,
> > > even if this creates one hole.
> > >
> > > Holes tend to be used quite fast anyway with new fields.
> > >
> > > Perhaps sock_read_tx group would be the best location,
> > > because tcp_recv_timestamp() is not called in the fast path.
> >
> > Just to wrap my head on the above reasoning: for UDP such a change could
> > possibly increase the number of `struct sock` cache-line accessed in the
> > RX path (the `sock_write_tx` group should not be touched otherwise) but
> > that will not matter much, because we expect a low number of UDP sockets
> > in the system, right?
> 
> Are you referring to UDP applications needing timestamps ?
> 
> Because sk_tsflags is mostly always used in TX

I thought the issue on rx was with the test in sock_recv_cmsgs.