Re: [PATCH net-next v8 00/11] net: tunnel: introduce noref xmit flows for tunnels

[Paolo Abeni <pabeni@redhat.com>]

On 4/6/26 9:58 AM, Marek Mietus wrote:
> W dniu 17.03.2026 o 12:37, Paolo Abeni pisze:
>> On 3/12/26 4:56 PM, Marek Mietus wrote:
>>> Currently, tunnel xmit flows always take a reference on the dst_entry
>>> for each xmitted packet. These atomic operations are redundant in some
>>> flows.
>>>
>>> This patchset introduces the infrastructure required for converting
>>> the tunnel xmit flows to noref, and converts them where possible.
>>>
>>> These changes improve tunnel performance, since less atomic operations
>>> are used.
>>>
>>> There are already noref optimizations in both ipv4 and ip6.
>>> (See __ip_queue_xmit, inet6_csk_xmit)
>>> This patchset implements similar optimizations in ip and udp tunnels.
>>>
>>> Benchmarks:
>>> I used a vxlan tunnel over a pair of veth peers and measured the average
>>> throughput over multiple samples.
>>>
>>> I ran 100 samples on a clean build, and another 100 on a patched
>>> build. Each sample ran for 120 seconds. These were my results:
>>>
>>> clean:      72.52 gb/sec, stddev = 1.39
>>> patched:    75.39 gb/sec, stddev = 0.94
>>>
>>> TL;DR - This patchset results in a 4% improvement in throughput for
>>> vxlan. It's safe to assume that we might see similar results when testing
>>> other tunnels.
>>
>> Sabrina noted I wrongly replied on an old revision. Reporting my
>> statements here for completeness.
>>
>> IMHO this performance delta is not enough to justify this amount of changes.
>>
>> Additionally, the measured impact of removing the dst_hold/dst_release
>> does not fit with my direct experience on the same matter: it should be
>> below noise level in practice, as dst are per-cpu and and no
>> contention/false sharing is expected in a good setup.
>>
>> I think you are observing larger impact because in the veth test
>> dst_release can happen on a remote CPU. Note that this setup (vxlan over
>> veth) is not very relevant in practice.
>>
>> I'm sorry I'm not applying this series.
>>
>> Side note: if you are interested into improving (UDP) tunnel
>> performances have a look to big TCP support work from Alice Mikityanska:
>>
>> https://lore.kernel.org/netdev/20260226201600.222044-1-alice.kernel@fastmail.im/
>>
>> /P
>>
>>
> 
> Hi Paolo,
> 
> Could you clarify what would be a more realistic test setup? I'm assuming using physical interfaces would yield more realistic results.
> Also, what performance benefit would be worth this kind of "code churn"? What kind of margins were you looking for?

veth devices hides a significant part of the memory buffer lifecycles,
as there is a single allocation/free for both sender and receiver. Using
reasonably fast/recent H/W NICs would make the figures much more relevant.

Given the amount of changes included in this series, I think that a
performance improvement well above the noise range in a relevant test
case would be needed to justify it. If you really want a figure, I would
say 5%. Note that the main difference WRT your testing is the setup itself.

To be completely open, I don't see how the refcount change could come
near such figure, as in my testing on H/W NICs the total reference
counting cost is usually well below noise level.

/P