Re: [PATCH net-next v5 3/5] veth: implement Byte Queue Limits (BQL) for latency reduction

[Jesper Dangaard Brouer <hawk@kernel.org>]

On 11/05/2026 11.55, Simon Schippers wrote:
> On 5/11/26 10:11, Jesper Dangaard Brouer wrote:
>>
>>
>> On 10/05/2026 17.56, Jakub Kicinski wrote:
>>> On Sat, 9 May 2026 11:09:51 +0200 Jesper Dangaard Brouer wrote:
>>>> On 09/05/2026 04.06, Jakub Kicinski wrote:
>>>>> On Thu, 7 May 2026 21:09:09 +0200 Jesper Dangaard Brouer wrote:
>>>>>> Not against being able to modify VETH_RING_SIZE, but I don't think it is
>>>>>> the solution here.
>>>>>
>>>>> Was it evaluated, tho?
>>>>>
>>>>> It's obviously super easy these days have AI spew no end of complex
>>>>> code. So it'd be great to have some solid, ideally production-like
>>>>> data to back this all up.
>>>>>
>>>>> VETH_RING_SIZE seems trivial, ethtool set ringparam
>>>>
>>>> No, unfortunately we cannot just decrease the VETH_RING_SIZE.
>>>
>>> To be clear - I said may it configurable with ethtool -G
>>> not change the default.
>>>
>>
>> Sure, I understand the desire to make VETH_RING_SIZE configurable.
>> If doing so we are making Linux network stack harder to tune and setup
>> correctly. E.g. adding a qdisc to veth would also require changing the
>> ring size, but if system also uses XDP then tuning below 64 (likely 128)
>> will lead to hard-to-find packet drops.
> 
> I mean 64 still could be a 4x improvement at least.
> 

No not really, setting it to 64 will give same (bad) latency from "BQL
off" which that patchset is trying to address.

>>
>> I prefer adding something (like BQL) that auto-tune how much of the ring
>> queue we are using.  Good queues function as shock absorbers when
>> concurrent processes in the OS have scheduling noise.
>>
>> I acknowledge that Simon Schippers found that the BQL implementation was
>> actually not auto-tuning.  We need to work on this, my prototype
>> implementation [1] [2] works surprisingly well.
>>
>>
>> - [1] https://lore.kernel.org/all/3e43117f-356d-4086-a176-abd7fe2e6f0a@kernel.org/2-09-veth-time-based-bql-coalescing.patch
>> - [2] https://lore.kernel.org/all/3e43117f-356d-4086-a176-abd7fe2e6f0a@kernel.org/
>>
>>
>>>> The reason is that XDP-redirect into veth don't have any
>>>> back-pressure and would simply drop packets if queue size becomes
>>>> less than the NAPI budget (64). (Yes, we use both normal path and
>>>> XDP-redirect in production).
>>>
>>> Doesn't this mean you have a queue which is not under BQL control?
>>>
>>
>> It is a matter of perspective. BQL needs between 17-55 elements in the
>> 256 queue.  At the same time we handle if the ring runs full, e.g. due
>> to a sudden burst of XDP redirected packets, which pushes packets into
>> the qdisc layer.
> 
> You are checking inflight/limit in /sys directory to get the 17-55
> number, right?
> 

Nope, I'm using a bpftrace program to keep track of the inflight/limit
in a BPF hashmap.  Reading from /sys will not be accurate.

I moved the selftests into a github repo [1] to allow us to collaborate
and evaluate the changes more easily.  I explicitly kept the new BPF
based BQL tracking as a commit[2] for your benefit.

  [1] 
https://github.com/netoptimizer/veth-backpressure-performance-testing/tree/main/selftests

  [2] 
https://github.com/netoptimizer/veth-backpressure-performance-testing/commit/f25c5dc92977

Sorry for cutting the remaining of the message, but I ran out of time,
as things are a bit challenging/hectic here at Cloudflare at the moment.

--Jesper