Re: [PATCH v11 0/4] virtio-net: define UDP tunnel offload

Paolo Abeni
2024-11-27T09:48:13Z

On 11/27/24 10:36, Paolo Abeni wrote:
> UDP tunnel usage is ubiquitous in container deployment, and the ability
> to offload UDP encapsulated GSO traffic impacts greatly the performances
> and the CPU utilization of such use cases.
>
> This series includes:
> - clarifications on the current semantic of NEEDS_CSUM offload
> (patch 1) and DATA_VALID offload (patch 2) to make later changes easier
> to follow,
> - new features definition to handle both UDP tunnel segmentation
> offload (patch 3).
> - new features definition to handle UDP tunnel outer checksum offload
> (patch 4).
>
> A PoC implementation is available here:
>
> https://github.com/pabeni/linux-devel/commits/virtio_tunnel_gso_v10/
> https://github.com/pabeni/qemu/tree/virtio_tunnel_gso_v9
>
> Note that both implementations has not been updated since the previous
> iteration, since there are no additional functional changes in v11.
>
> Changes from v10:
> - added patch 2
> - clarified only a single encapsulation level is supported (patch 3)
> - s/TCP or UDP GSO packets/GSO packets/ (patch 3)
> - 'to the outer UDP pseudo header' -> 'to the outer UDP pseudo header
> checksum' (patch 4)
> https://lore.kernel.org/virtio-comment/cover.1731084277.git.pabeni@redhat.com/

As a reference this is the complete diff from v10:
---
diff --git a/device-types/net/description.tex
b/device-types/net/description.tex
index 2d15ca8..efb543f 100644
--- a/device-types/net/description.tex
+++ b/device-types/net/description.tex
@@ -412,6 +412,10 @@ \subsection{Device Initialization}\label{sec:Device
Types / Network Device / Dev
offload, when the outer header does not require checksumming - e.g.
the outer UDP checksum is zero - by negotiating the
VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO feature.
+ GSO over UDP tunnels packets carry two sets of headers: the outer ones
+ and the inner ones. The outer transport protocol is UDP, the inner
+ could be either TCP or UDP. Only a single level of encapsulation
+ offload is supported.

\item If VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is negotiated, a driver can
additionally use TCP segmentation or UDP segmentation on top of UDP
@@ -792,17 +796,17 @@ \subsubsection{Packet
Transmission}\label{sec:Device Types / Network Device / De
If VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is negotiated, the driver MAY set
VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV4 bit or the
VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV6 bit
in \field{gso_type} according to the inner network header protocol type
-to request TCP or UDP GSO packets over UDPv4 or UDPv6 tunnel
-segmentation, otherwise the driver MUST NOT set either the
+to request GSO packets over UDPv4 or UDPv6 tunnel segmentation,
+otherwise the driver MUST NOT set either the
VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV4 bit or the
VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV6 bit
in \field{gso_type}.

-When requesting TCP or UDP GSO segmentation over UDP tunnel, the driver
MUST SET the
+When requesting GSO segmentation over UDP tunnel, the driver MUST SET the
VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV4 bit if the inner network header is
IPv4, i.e. the
packet is a TCPv4 GSO one, otherwise, if the inner network header is
IPv6, the driver
MUST SET the VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV6 bit.

-The driver MUST NOT send to the device TCP or UDP GSO packets over UDP
tunnel
+The driver MUST NOT send to the device GSO packets over UDP tunnel
requiring segmentation and outer UDP checksum offload, unless both the
VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO and
VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM features
are negotiated, in which case the driver MUST set either the
@@ -812,7 +816,7 @@ \subsubsection{Packet Transmission}\label{sec:Device
Types / Network Device / De

If VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO_CSUM is not negotiated, the driver
MUST not set
the VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM bit in the \field{flags} and
-MUST NOT send to the device TCP or UDP GSO packets over UDP tunnel
+MUST NOT send to the device GSO packets over UDP tunnel
requiring segmentation and outer UDP checksum offload.

The driver MUST NOT set the VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV4 bit or the
@@ -892,7 +896,7 @@ \subsubsection{Packet Transmission}\label{sec:Device
Types / Network Device / De
VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV6 bit in \field{gso_type} are set,
the driver MAY set the VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM bit in
\field{flags}, if so the driver MUST set the packet outer UDP header
checksum
-to the outer UDP pseudo header.
+to the outer UDP pseudo header checksum.

\begin{note}
calculating a ones' complement checksum from \field{outer_th_offset}
@@ -1148,10 +1152,10 @@ \subsubsection{Processing of Incoming
Packets}\label{sec:Device Types / Network

\begin{note}
If the VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV4 bit or
VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV6
-bit are set in \field{gso_type}, the \field{csum_start} field refers to
+bit are set in \field{gso_type}, the \field{csum_start} field refers to
the inner transport header offset (see Packet Transmission point 1).
If the VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM bit in \field{flags} is set both
-the inner and the outer header checksums have been validated by
+the inner and the outer header checksums have been validated by
VIRTIO_NET_HDR_F_NEEDS_CSUM, otherwise only the inner transport header
checksum has been validated.
\end{note}
@@ -1228,7 +1232,7 @@ \subsubsection{Processing of Incoming
Packets}\label{sec:Device Types / Network
fully checksummed packet;
\end{enumerate}

-The device MUST NOT send to the driver TCP or UDP GSO packets
encapsulated in UDP
+The device MUST NOT send to the driver GSO packets encapsulated in UDP
tunnel and requiring segmentation offload, unless the
VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO is negotiated, in which case the
device MUST set
the VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV4 bit or the
VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV6
@@ -1238,7 +1242,7 @@ \subsubsection{Processing of Incoming
Packets}\label{sec:Device Types / Network
require checksum offload.

If the VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM feature has not been
negotiated,
-the device MUST NOT send the driver TCP or UDP GSO packets encapsulated
in UDP
+the device MUST NOT send the driver GSO packets encapsulated in UDP
tunnel and requiring segmentation and outer checksum offload.

If none of the VIRTIO_NET_F_GUEST_TSO4, TSO6, UFO, USO4 or USO6 options
have
@@ -1269,8 +1273,9 @@ \subsubsection{Processing of Incoming
Packets}\label{sec:Device Types / Network
checksum. If the VIRTIO_NET_F_GUEST_UDP_TUNNEL_GSO_CSUM feature has
been negotiated, and the VIRTIO_NET_HDR_F_UDP_TUNNEL_CSUM bit set in
\field{flags}, both the outer UDP checksum and the inner transport
-checksum have been validated, otherwise only one level of checksums
-(the outer one in case of tunnels) has been validated.
+checksum have been validated.
+Otherwise level of checksum is validated: in case of multiple
+encapsulated protocols the outermost one.

If either the VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV4 bit or the
VIRTIO_NET_HDR_GSO_UDP_TUNNEL_IPV6 bit in \field{gso_type} are set,