Re: [PATCH bpf 1/2] bpf: Avoid iter->offset making backward progress in bpf_iter_udp

[Martin KaFai Lau <martin.lau@linux.dev>]

On 12/21/23 5:21 AM, Daniel Borkmann wrote:
> On 12/21/23 5:45 AM, Martin KaFai Lau wrote:
>> On 12/20/23 11:10 AM, Martin KaFai Lau wrote:
>>> Good catch. It will unnecessary skip in the following batch/bucket if there 
>>> is changes in the current batch/bucket.
>>>
>>>  From looking at the loop again, I think it is better not to change the 
>>> iter->offset during the for loop. Only update iter->offset after the for loop 
>>> has concluded.
>>>
>>> The non-zero iter->offset is only useful for the first bucket, so does a test 
>>> on the first bucket (state->bucket == bucket) before skipping sockets. 
>>> Something like this:
>>>
>>> diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
>>> index 89e5a806b82e..a993f364d6ae 100644
>>> --- a/net/ipv4/udp.c
>>> +++ b/net/ipv4/udp.c
>>> @@ -3139,6 +3139,7 @@ static struct sock *bpf_iter_udp_batch(struct seq_file 
>>> *seq)
>>>       struct net *net = seq_file_net(seq);
>>>       struct udp_table *udptable;
>>>       unsigned int batch_sks = 0;
>>> +    int bucket, bucket_offset;
>>>       bool resized = false;
>>>       struct sock *sk;
>>>
>>> @@ -3162,14 +3163,14 @@ static struct sock *bpf_iter_udp_batch(struct 
>>> seq_file *seq)
>>>       iter->end_sk = 0;
>>>       iter->st_bucket_done = false;
>>>       batch_sks = 0;
>>> +    bucket = state->bucket;
>>> +    bucket_offset = 0;
>>>
>>>       for (; state->bucket <= udptable->mask; state->bucket++) {
>>>           struct udp_hslot *hslot2 = &udptable->hash2[state->bucket];
>>>
>>> -        if (hlist_empty(&hslot2->head)) {
>>> -            iter->offset = 0;
>>> +        if (hlist_empty(&hslot2->head))
>>>               continue;
>>> -        }
>>>
>>>           spin_lock_bh(&hslot2->lock);
>>>           udp_portaddr_for_each_entry(sk, &hslot2->head) {
>>> @@ -3177,8 +3178,9 @@ static struct sock *bpf_iter_udp_batch(struct seq_file 
>>> *seq)
>>>                   /* Resume from the last iterated socket at the
>>>                    * offset in the bucket before iterator was stopped.
>>>                    */
>>> -                if (iter->offset) {
>>> -                    --iter->offset;
>>> +                if (state->bucket == bucket &&
>>> +                    bucket_offset < iter->offset) {
>>> +                    ++bucket_offset;
>>>                       continue;
>>>                   }
>>>                   if (iter->end_sk < iter->max_sk) {
>>> @@ -3192,10 +3194,10 @@ static struct sock *bpf_iter_udp_batch(struct 
>>> seq_file *seq)
>>>
>>>           if (iter->end_sk)
>>>               break;
>>> +    }
>>>
>>> -        /* Reset the current bucket's offset before moving to the next 
>>> bucket. */
>>> +    if (state->bucket != bucket)
>>>           iter->offset = 0;
>>> -    }
>>>
>>>       /* All done: no batch made. */
>>>       if (!iter->end_sk)
>>
>> I think I found another bug in the current bpf_iter_udp_batch(). The 
>> "state->bucket--;" at the end of the batch() function is wrong also. It does 
>> not need to go back to the previous bucket. After realloc with a larger batch 
>> array, it should retry on the "state->bucket" as is. I tried to force the 
>> bind() to use bucket 0 and bind a larger so_reuseport set (24 sockets). 
>> WARN_ON(state->bucket < 0) triggered.
>>
>> Going back to this bug (backward progress on --iter->offset), I think it is a 
>> bit cleaner to always reset iter->offset to 0 and advance iter->offset to the 
>> resume_offset only when needed. Something like this:
> 
> Hm, my assumption was.. why not do something like the below, and fully start over?
> 
> I'm mostly puzzled about the side-effects here, in particular, if for the rerun 
> the sockets
> in the bucket could already have changed.. maybe I'm still missing something - 
> what do
> we need to deal with exactly worst case when we need to go and retry everything, 
> and what
> guarantees do we have?
> 
> (only compile tested)
> 
> diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
> index 89e5a806b82e..ca62a4bb7bec 100644
> --- a/net/ipv4/udp.c
> +++ b/net/ipv4/udp.c
> @@ -3138,7 +3138,8 @@ static struct sock *bpf_iter_udp_batch(struct seq_file *seq)
>       struct udp_iter_state *state = &iter->state;
>       struct net *net = seq_file_net(seq);
>       struct udp_table *udptable;
> -    unsigned int batch_sks = 0;
> +    int orig_bucket, orig_offset;
> +    unsigned int i, batch_sks = 0;
>       bool resized = false;
>       struct sock *sk;
> 
> @@ -3149,7 +3150,8 @@ static struct sock *bpf_iter_udp_batch(struct seq_file *seq)
>       }
> 
>       udptable = udp_get_table_seq(seq, net);
> -
> +    orig_bucket = state->bucket;
> +    orig_offset = iter->offset;
>   again:
>       /* New batch for the next bucket.
>        * Iterate over the hash table to find a bucket with sockets matching
> @@ -3211,9 +3213,15 @@ static struct sock *bpf_iter_udp_batch(struct seq_file *seq)
>       if (!resized && !bpf_iter_udp_realloc_batch(iter, batch_sks * 3 / 2)) {
>           resized = true;
>           /* After allocating a larger batch, retry one more time to grab
> -         * the whole bucket.
> +         * the whole bucket. Drop the current refs since for the next
> +         * attempt the composition could have changed, thus start over.
>            */
> -        state->bucket--;
> +        for (i = 0; i < iter->end_sk; i++) {
> +            sock_put(iter->batch[i]);
> +            iter->batch[i] = NULL;
> +        }
> +        state->bucket = orig_bucket;
> +        iter->offset = orig_offset;

It does not need to start over from the orig_bucket. Once it advanced to the 
next bucket (state->bucket++), the orig_bucket is done. Otherwise, it may need 
to make backward progress here on the state->bucket. The batch size too small 
happens on the current state->bucket, so it should retry with the same 
state->bucket after realloc_batch(). If the state->bucket happens to be the 
orig_bucket (mean it has not advanced), it will skip the same orig_offset.

If the orig_bucket had changed (e.g. having more sockets than the last time it 
was batched) after state->bucket++, it is arguably fine because it was added 
after the orig_bucket was completely captured in a batch before. The same goes 
for (orig_bucket-1) that could have changed during the whole udp_table iteration.

>           goto again;
>       }
>   done: