Re: [PATCH net-next v2 2/3] net: rps: protect filter locklessly

[Eric Dumazet <edumazet@google.com>]

On Wed, Apr 17, 2024 at 8:27 AM Jason Xing <kerneljasonxing@gmail.com> wrote:
>
> From: Jason Xing <kernelxing@tencent.com>
>
> As we can see, rflow->filter can be written/read concurrently, so
> lockless access is needed.
>
> Signed-off-by: Jason Xing <kernelxing@tencent.com>
> ---
> I'm not very sure if the READ_ONCE in set_rps_cpu() is useful. I
> scaned/checked the codes and found no lock can prevent multiple
> threads from calling set_rps_cpu() and handling the same flow
> simultaneously. The same question still exists in patch [3/3].
> ---
>  net/core/dev.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/net/core/dev.c b/net/core/dev.c
> index 2003b9a61e40..40a535158e45 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -4524,8 +4524,8 @@ set_rps_cpu(struct net_device *dev, struct sk_buff *skb,
>                         goto out;
>                 old_rflow = rflow;
>                 rflow = &flow_table->flows[flow_id];
> -               rflow->filter = rc;
> -               if (old_rflow->filter == rflow->filter)
> +               WRITE_ONCE(rflow->filter, rc);
> +               if (old_rflow->filter == READ_ONCE(rflow->filter))

You missed the obvious opportunity to use

               if (old_rflow->filter ==  rc)

Here your code is going to force the compiler to read the memory right
after a prior write, adding a stall on some arches.

>                         old_rflow->filter = RPS_NO_FILTER;
>         out:
>  #endif
> @@ -4666,7 +4666,7 @@ bool rps_may_expire_flow(struct net_device *dev, u16 rxq_index,
>         if (flow_table && flow_id <= flow_table->mask) {
>                 rflow = &flow_table->flows[flow_id];
>                 cpu = READ_ONCE(rflow->cpu);
> -               if (rflow->filter == filter_id && cpu < nr_cpu_ids &&
> +               if (READ_ONCE(rflow->filter) == filter_id && cpu < nr_cpu_ids &&
>                     ((int)(READ_ONCE(per_cpu(softnet_data, cpu).input_queue_head) -
>                            READ_ONCE(rflow->last_qtail)) <
>                      (int)(10 * flow_table->mask)))
> --
> 2.37.3
>