[PATCH] virtio-net: introduce TSO limit feature

[PATCH] virtio-net: introduce TSO limit feature

[Jason Wang]

This patch introduces TSO limit feature which allows the device to
advertise:

- Maximum TCP length of a TSO packet or inner TSO packet when UDP
  tunnel is support
- Maximum number of segment that can be produced by the device after
  segmentation of TSO or inner TSO packet of a UDP tunnel

This is a must to implement TCP jumbogram, as networking stack needs
to know the limitation of the device in order to produce TSO packet as
large as possible.

And it would also help for the case where host has a different TSO
limitation than the assumption (for example, Linux assumes 64K to be
the maximum number of segs and payload length).

Signed-off-by: Jason Wang <jasowang@redhat.com>
---
 device-types/net/description.tex | 46 ++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/device-types/net/description.tex b/device-types/net/description.tex
index 415c7fd..e56df75 100644
--- a/device-types/net/description.tex
+++ b/device-types/net/description.tex
@@ -146,6 +146,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
  when VIRTIO_NET_F_IPSEC is negotiated. When a device offers IPsec feature, it SHOULD
  also offer the VIRTIO_NET_F_OUT_NET_HEADER feature.
 
+\item[VIRTIO_NET_F_HOST_TSO_LIMIT(71)] Device limits the maximum TCP
+  length and the number of segments when performing TCP segmentation.
+
 \end{description}
 
 \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
@@ -184,6 +187,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_VQ_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ along with VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
 \item[VIRTIO_NET_F_RSS_CONTEXT] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
+\item[VIRTIO_NET_F_HOST_TSO_LIMIT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6
 \end{description}
 
 \begin{note}
@@ -220,6 +224,8 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
         le16 rss_max_indirection_table_length;
         le32 supported_hash_types;
         le32 supported_tunnel_types;
+        le32 tso_max_size;
+        le32 tso_max_segs;
 };
 \end{lstlisting}
 
@@ -276,6 +282,19 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
 Encapsulation types are defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets /
 Hash calculation for incoming packets / Encapsulation types supported/enabled for inner header hash}.
 
+The following field, \field{tso_max_size} only exists if
+VIRTIO_NET_F_HOST_TSO_LIMIT is set.
+It specifies the maximum TCP length of a TSO packet that the
+device can process. When VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set,
+it specifies the maximum inner TCP length of a UDP tunnel TSO packet
+that the device can process.
+
+The following field, \field{tso_max_segs} only exists if
+VIRTIO_NET_F_HOST_TSO_LIMIT is set.
+It specifies the maximum number of segments that can be produced by
+the device after performing segmentation on TSO packet or a UDP tunnel
+TSO packet (when VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set).
+
 \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
 
 The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
@@ -326,6 +345,17 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
 The device SHOULD NOT offer VIRTIO_NET_F_CTRL_RX_EXTRA if it
 does not offer VIRTIO_NET_F_CTRL_VQ.
 
+If VIRTIO_NET_F_HOST_TSO_LIMIT and VIRTIO_NET_F_MTU have been
+negotiated, the device SHOULD set \field{tso_max_size} so that a TCP
+segment that fully utilizes the configured MTU can be processed by TSO
+(e.g., for IPv4 without options: at least \field{mtu} - 20; for IPv6
+without extension headers: at least \field{mtu} - 40). This
+recommendation does not account for IPv4 options or IPv6 extension
+headers, which reduce the effective segment size.
+
+If VIRTIO_NET_F_HOST_TSO_LIMIT has been negotiated, the device MUST
+set \field{tso_max_segs} to at least 64.
+
 \drivernormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
 
 The driver MUST NOT write to any of the device configuration fields.
@@ -379,6 +409,22 @@ \subsubsection{Legacy Interface: Device configuration layout}\label{sec:Device T
 which provided a way for drivers to update the MAC without
 negotiating VIRTIO_NET_F_CTRL_MAC_ADDR.
 
+If the driver negotiates VIRTIO_NET_F_HOST_TSO_LIMIT, it MUST NOT
+transmit TSO packets with TCP length exceeding \field{tso_max_size}.
+
+If the driver negotiates both VIRTIO_NET_F_HOST_TSO_LIMIT and
+VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, it MUST NOT transmit UDP tunnel TSO
+packets with inner TCP length exceeding \field{tso_max_size}.
+
+If the driver negotiates VIRTIO_NET_F_HOST_TSO_LIMIT, it MUST NOT
+transmit TSO packets with \field{gso_size} that would cause the device
+to generate more than \field{tso_max_segs} segments.
+
+If the driver negotiates both VIRTIO_NET_F_HOST_TSO_LIMIT and
+VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, it MUST NOT transmit UDP tunnel TSO
+packets with \field{gso_size} that would cause the device to generate
+more than \field{tso_max_segs} segments.
+
 \subsection{Device Initialization}\label{sec:Device Types / Network Device / Device Initialization}
 
 A driver would perform a typical initialization routine like so:
-- 
2.42.0

Re: [PATCH] virtio-net: introduce TSO limit feature

[Michael S. Tsirkin]

thanks for the patch! yet something to improve.
I note issues only when encountered 1st time but please
do go and check all patch for each issue.


On Tue, Oct 14, 2025 at 12:22:43PM +0800, Jason Wang wrote:
> This patch introduces TSO limit feature which allows the device to
> advertise:
> 
> - Maximum TCP length of a TSO packet or inner TSO packet when UDP
>   tunnel is support
> - Maximum number of segment that can be produced by the device after
>   segmentation of TSO or inner TSO packet of a UDP tunnel
> 
> This is a must to implement TCP jumbogram, as networking stack needs
> to know the limitation of the device in order to produce TSO packet as
> large as possible.

I am not sure I understand the use-case. So how is it used,
exactly?

Why is TSO singled out as compared to USO?

> 
> And it would also help for the case where host has a different TSO
> limitation than the assumption (for example, Linux assumes 64K to be
> the maximum number of segs and payload length).

how does it assume it? 64K segments?

> Signed-off-by: Jason Wang <jasowang@redhat.com>




> ---
>  device-types/net/description.tex | 46 ++++++++++++++++++++++++++++++++
>  1 file changed, 46 insertions(+)
> 
> diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> index 415c7fd..e56df75 100644
> --- a/device-types/net/description.tex
> +++ b/device-types/net/description.tex
> @@ -146,6 +146,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
>   when VIRTIO_NET_F_IPSEC is negotiated. When a device offers IPsec feature, it SHOULD
>   also offer the VIRTIO_NET_F_OUT_NET_HEADER feature.
>  
> +\item[VIRTIO_NET_F_HOST_TSO_LIMIT(71)] Device limits the maximum TCP
> +  length and the number of segments when performing TCP segmentation.
> +
>  \end{description}
>  
>  \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
> @@ -184,6 +187,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
>  \item[VIRTIO_NET_F_VQ_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
>  \item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ along with VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
>  \item[VIRTIO_NET_F_RSS_CONTEXT] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> +\item[VIRTIO_NET_F_HOST_TSO_LIMIT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6
>  \end{description}
>  
>  \begin{note}
> @@ -220,6 +224,8 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
>          le16 rss_max_indirection_table_length;
>          le32 supported_hash_types;
>          le32 supported_tunnel_types;
> +        le32 tso_max_size;
> +        le32 tso_max_segs;
>  };
>  \end{lstlisting}
>  
> @@ -276,6 +282,19 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
>  Encapsulation types are defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets /
>  Hash calculation for incoming packets / Encapsulation types supported/enabled for inner header hash}.
>  
> +The following field, \field{tso_max_size} only exists if
> +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> +It specifies the maximum TCP length

what is TCP length?

> of a TSO packet

what is a TSO packet? is length likely to be same for TCP6 and TCP4?

> that the
> +device can process.

process in which direction? you mean device can receive?

> When VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set,
> +it specifies the maximum inner TCP length of a UDP tunnel TSO packet
> +that the device can process.

Rest of spec talks of " GSO over UDP tunnels packets" is this the same?

even if it's actually unused?

this, on the assumption that the length for tunnel is smaller?

I think this kind of things should be explicit.


> +
> +The following field, \field{tso_max_segs} only exists if
> +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> +It specifies the maximum number of segments that can be produced by
> +the device after performing segmentation on TSO packet or a UDP tunnel
> +TSO packet (when VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set).

I don't get this field at all. the assumption is that all segments
are the same size, right? Then it is just based on length?



> +
>  \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
>  
>  The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
> @@ -326,6 +345,17 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
>  The device SHOULD NOT offer VIRTIO_NET_F_CTRL_RX_EXTRA if it
>  does not offer VIRTIO_NET_F_CTRL_VQ.
>  
> +If VIRTIO_NET_F_HOST_TSO_LIMIT and VIRTIO_NET_F_MTU have been
> +negotiated, the device SHOULD set \field{tso_max_size} so that a TCP
> +segment that fully utilizes the configured MTU can be processed by TSO
> +(e.g., for IPv4 without options: at least \field{mtu} - 20; for IPv6
> +without extension headers: at least \field{mtu} - 40). This
> +recommendation does not account for IPv4 options or IPv6 extension
> +headers, which reduce the effective segment size.
> +
> +If VIRTIO_NET_F_HOST_TSO_LIMIT has been negotiated, the device MUST
> +set \field{tso_max_segs} to at least 64.

where does this 64 come from? pls document.

> +
>  \drivernormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
>  
>  The driver MUST NOT write to any of the device configuration fields.
> @@ -379,6 +409,22 @@ \subsubsection{Legacy Interface: Device configuration layout}\label{sec:Device T
>  which provided a way for drivers to update the MAC without
>  negotiating VIRTIO_NET_F_CTRL_MAC_ADDR.
>  
> +If the driver negotiates VIRTIO_NET_F_HOST_TSO_LIMIT, it MUST NOT
> +transmit TSO packets with TCP length exceeding \field{tso_max_size}.
> +
> +If the driver negotiates both VIRTIO_NET_F_HOST_TSO_LIMIT and
> +VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, it MUST NOT transmit UDP tunnel TSO
> +packets with inner TCP length exceeding \field{tso_max_size}.
> +
> +If the driver negotiates VIRTIO_NET_F_HOST_TSO_LIMIT, it MUST NOT
> +transmit TSO packets with \field{gso_size} that would cause the device
> +to generate more than \field{tso_max_segs} segments.

and how does it know?


> +
> +If the driver negotiates both VIRTIO_NET_F_HOST_TSO_LIMIT and
> +VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, it MUST NOT transmit UDP tunnel TSO
> +packets with \field{gso_size} that would cause the device to generate
> +more than \field{tso_max_segs} segments.
> +
>  \subsection{Device Initialization}\label{sec:Device Types / Network Device / Device Initialization}
>  
>  A driver would perform a typical initialization routine like so:
> -- 
> 2.42.0

Re: [PATCH] virtio-net: introduce TSO limit feature

[Jason Wang]

On Tue, Oct 14, 2025 at 4:59 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> thanks for the patch! yet something to improve.
> I note issues only when encountered 1st time but please
> do go and check all patch for each issue.
>
>
> On Tue, Oct 14, 2025 at 12:22:43PM +0800, Jason Wang wrote:
> > This patch introduces TSO limit feature which allows the device to
> > advertise:
> >
> > - Maximum TCP length of a TSO packet or inner TSO packet when UDP
> >   tunnel is support
> > - Maximum number of segment that can be produced by the device after
> >   segmentation of TSO or inner TSO packet of a UDP tunnel
> >
> > This is a must to implement TCP jumbogram, as networking stack needs
> > to know the limitation of the device in order to produce TSO packet as
> > large as possible.
>
> I am not sure I understand the use-case. So how is it used,
> exactly?

It works like a device advertising the TSO limit so the kernel stack
can try to send a TSO packet up to that limit to the device to perform
TCP segmentation offload. E.g without this device information
networking stack can only send a size at most 64K packet to the
device.

>
> Why is TSO singled out as compared to USO?

AFAIK no support for big USO in Linux.

>
> >
> > And it would also help for the case where host has a different TSO
> > limitation than the assumption (for example, Linux assumes 64K to be
> > the maximum number of segs and payload length).
>
> how does it assume it? 64K segments?

Yes.

>
> > Signed-off-by: Jason Wang <jasowang@redhat.com>
>
>
>
>
> > ---
> >  device-types/net/description.tex | 46 ++++++++++++++++++++++++++++++++
> >  1 file changed, 46 insertions(+)
> >
> > diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> > index 415c7fd..e56df75 100644
> > --- a/device-types/net/description.tex
> > +++ b/device-types/net/description.tex
> > @@ -146,6 +146,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> >   when VIRTIO_NET_F_IPSEC is negotiated. When a device offers IPsec feature, it SHOULD
> >   also offer the VIRTIO_NET_F_OUT_NET_HEADER feature.
> >
> > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT(71)] Device limits the maximum TCP
> > +  length and the number of segments when performing TCP segmentation.
> > +
> >  \end{description}
> >
> >  \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
> > @@ -184,6 +187,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> >  \item[VIRTIO_NET_F_VQ_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> >  \item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ along with VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> >  \item[VIRTIO_NET_F_RSS_CONTEXT] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6
> >  \end{description}
> >
> >  \begin{note}
> > @@ -220,6 +224,8 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> >          le16 rss_max_indirection_table_length;
> >          le32 supported_hash_types;
> >          le32 supported_tunnel_types;
> > +        le32 tso_max_size;
> > +        le32 tso_max_segs;
> >  };
> >  \end{lstlisting}
> >
> > @@ -276,6 +282,19 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> >  Encapsulation types are defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets /
> >  Hash calculation for incoming packets / Encapsulation types supported/enabled for inner header hash}.
> >
> > +The following field, \field{tso_max_size} only exists if
> > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > +It specifies the maximum TCP length
>
> what is TCP length?

It's defined in the rfc793:

"""
      The TCP Length is the TCP header length plus the data length in
      octets (this is not an explicitly transmitted quantity, but is
      computed), and it does not count the 12 octets of the pseudo
      header.
"""

>
> > of a TSO packet
>
> what is a TSO packet?

Packet for device to perform TCP segmentation offload.

> is length likely to be same for TCP6 and TCP4?

Technically we can have different limitations for ipv4 and ipv6. But
considering we are designing a fresh new feature, it would be better
to use the same limitation for simplicity.

>
> > that the
> > +device can process.
>
> process in which direction? you mean device can receive?

It works only for TX (as TSO works only for TX).

>
> > When VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set,
> > +it specifies the maximum inner TCP length of a UDP tunnel TSO packet
> > +that the device can process.
>
> Rest of spec talks of " GSO over UDP tunnels packets" is this the same?

Not exactly the same, this is only for TSO not genreal GSO.

>
> even if it's actually unused?
>
> this, on the assumption that the length for tunnel is smaller?

It means the device should have the same limitation for plain TSO and
tunnel TSO.

>
> I think this kind of things should be explicit.
>
>
> > +
> > +The following field, \field{tso_max_segs} only exists if
> > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > +It specifies the maximum number of segments that can be produced by
> > +the device after performing segmentation on TSO packet or a UDP tunnel
> > +TSO packet (when VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set).
>
> I don't get this field at all. the assumption is that all segments
> are the same size, right? Then it is just based on length?

It's the device side limitation, for example a device can produce 100
segments at most, even if the tso_max_size is 256K, when MTU is 1500,
the driver can't send a TSO packet whose TCP length is greater than
(1500 - 20 - 20) * 100 = 146K.

>
>
>
> > +
> >  \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
> >
> >  The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
> > @@ -326,6 +345,17 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> >  The device SHOULD NOT offer VIRTIO_NET_F_CTRL_RX_EXTRA if it
> >  does not offer VIRTIO_NET_F_CTRL_VQ.
> >
> > +If VIRTIO_NET_F_HOST_TSO_LIMIT and VIRTIO_NET_F_MTU have been
> > +negotiated, the device SHOULD set \field{tso_max_size} so that a TCP
> > +segment that fully utilizes the configured MTU can be processed by TSO
> > +(e.g., for IPv4 without options: at least \field{mtu} - 20; for IPv6
> > +without extension headers: at least \field{mtu} - 40). This
> > +recommendation does not account for IPv4 options or IPv6 extension
> > +headers, which reduce the effective segment size.
> > +
> > +If VIRTIO_NET_F_HOST_TSO_LIMIT has been negotiated, the device MUST
> > +set \field{tso_max_segs} to at least 64.
>
> where does this 64 come from? pls document.

A simple backward compatibility which makes sure the value can make
sure 64K TSO can be segmented with 1500 MTU.

>
> > +
> >  \drivernormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
> >
> >  The driver MUST NOT write to any of the device configuration fields.
> > @@ -379,6 +409,22 @@ \subsubsection{Legacy Interface: Device configuration layout}\label{sec:Device T
> >  which provided a way for drivers to update the MAC without
> >  negotiating VIRTIO_NET_F_CTRL_MAC_ADDR.
> >
> > +If the driver negotiates VIRTIO_NET_F_HOST_TSO_LIMIT, it MUST NOT
> > +transmit TSO packets with TCP length exceeding \field{tso_max_size}.
> > +
> > +If the driver negotiates both VIRTIO_NET_F_HOST_TSO_LIMIT and
> > +VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, it MUST NOT transmit UDP tunnel TSO
> > +packets with inner TCP length exceeding \field{tso_max_size}.
> > +
> > +If the driver negotiates VIRTIO_NET_F_HOST_TSO_LIMIT, it MUST NOT
> > +transmit TSO packets with \field{gso_size} that would cause the device
> > +to generate more than \field{tso_max_segs} segments.
>
> and how does it know?

Driver knows the tso_max_segs, so when it finds the TSO packet will
exceed the limitation it would allocate a new skb for example.

>
>
> > +
> > +If the driver negotiates both VIRTIO_NET_F_HOST_TSO_LIMIT and
> > +VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO, it MUST NOT transmit UDP tunnel TSO
> > +packets with \field{gso_size} that would cause the device to generate
> > +more than \field{tso_max_segs} segments.
> > +
> >  \subsection{Device Initialization}\label{sec:Device Types / Network Device / Device Initialization}
> >
> >  A driver would perform a typical initialization routine like so:
> > --
> > 2.42.0

Thanks

>

Re: [PATCH] virtio-net: introduce TSO limit feature

[Michael S. Tsirkin]

On Wed, Oct 15, 2025 at 12:29:13PM +0800, Jason Wang wrote:
> On Tue, Oct 14, 2025 at 4:59 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > thanks for the patch! yet something to improve.
> > I note issues only when encountered 1st time but please
> > do go and check all patch for each issue.
> >
> >
> > On Tue, Oct 14, 2025 at 12:22:43PM +0800, Jason Wang wrote:
> > > This patch introduces TSO limit feature which allows the device to
> > > advertise:
> > >
> > > - Maximum TCP length of a TSO packet or inner TSO packet when UDP
> > >   tunnel is support
> > > - Maximum number of segment that can be produced by the device after
> > >   segmentation of TSO or inner TSO packet of a UDP tunnel
> > >
> > > This is a must to implement TCP jumbogram, as networking stack needs
> > > to know the limitation of the device in order to produce TSO packet as
> > > large as possible.


Maybe you can provide an overview of how this all is used?
I am looking at tcp_fragment and I just do not see
where it looks at any device limits.

-- 
MST

Re: [PATCH] virtio-net: introduce TSO limit feature

[Michael S. Tsirkin]

Thanks for the answers. Some more comments:

On Wed, Oct 15, 2025 at 12:29:13PM +0800, Jason Wang wrote:
> > >  device-types/net/description.tex | 46 ++++++++++++++++++++++++++++++++
> > >  1 file changed, 46 insertions(+)
> > >
> > > diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> > > index 415c7fd..e56df75 100644
> > > --- a/device-types/net/description.tex
> > > +++ b/device-types/net/description.tex
> > > @@ -146,6 +146,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > >   when VIRTIO_NET_F_IPSEC is negotiated. When a device offers IPsec feature, it SHOULD
> > >   also offer the VIRTIO_NET_F_OUT_NET_HEADER feature.
> > >
> > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT(71)] Device limits the maximum TCP
> > > +  length and the number of segments when performing TCP segmentation.
> > > +
> > >  \end{description}
> > >
> > >  \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
> > > @@ -184,6 +187,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> > >  \item[VIRTIO_NET_F_VQ_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> > >  \item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ along with VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> > >  \item[VIRTIO_NET_F_RSS_CONTEXT] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6
> > >  \end{description}
> > >
> > >  \begin{note}
> > > @@ -220,6 +224,8 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > >          le16 rss_max_indirection_table_length;
> > >          le32 supported_hash_types;
> > >          le32 supported_tunnel_types;
> > > +        le32 tso_max_size;
> > > +        le32 tso_max_segs;
> > >  };
> > >  \end{lstlisting}
> > >
> > > @@ -276,6 +282,19 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > >  Encapsulation types are defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets /
> > >  Hash calculation for incoming packets / Encapsulation types supported/enabled for inner header hash}.
> > >
> > > +The following field, \field{tso_max_size} only exists if
> > > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > > +It specifies the maximum TCP length
> >
> > what is TCP length?
> 
> It's defined in the rfc793:
> 
> """
>       The TCP Length is the TCP header length plus the data length in
>       octets (this is not an explicitly transmitted quantity, but is
>       computed), and it does not count the 12 octets of the pseudo
>       header.
> """


But that one is 16 bit so can not exceed 65535.

> >
> > > of a TSO packet
> >
> > what is a TSO packet?
> 
> Packet for device to perform TCP segmentation offload.

pls define terms before use.

> > > that the
> > > +device can process.
> >
> > process in which direction? you mean device can receive?
> 
> It works only for TX (as TSO works only for TX).


rest of spec says "device receives from driver"  for this.
process is ambiguous




> >
> > > When VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set,
> > > +it specifies the maximum inner TCP length of a UDP tunnel TSO packet
> > > +that the device can process.
> >
> > Rest of spec talks of " GSO over UDP tunnels packets" is this the same?
> 
> Not exactly the same, this is only for TSO not genreal GSO.

rest of spec mostly talks of GSO. in fact virtio tso is a kind of
accelerated gso.  either do the same or add a lot of text
explaining tso as opposed to gso.

> >
> > even if it's actually unused?
> >
> > this, on the assumption that the length for tunnel is smaller?
> 
> It means the device should have the same limitation for plain TSO and
> tunnel TSO.

Hmm. I have doubts how it can work given the overhead.

> >
> > I think this kind of things should be explicit.
> >
> >
> > > +
> > > +The following field, \field{tso_max_segs} only exists if
> > > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > > +It specifies the maximum number of segments that can be produced by
> > > +the device after performing segmentation on TSO packet or a UDP tunnel
> > > +TSO packet (when VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set).
> >
> > I don't get this field at all. the assumption is that all segments
> > are the same size, right? Then it is just based on length?
> 
> It's the device side limitation, for example a device can produce 100
> segments at most, even if the tso_max_size is 256K, when MTU is 1500,
> the driver can't send a TSO packet whose TCP length is greater than
> (1500 - 20 - 20) * 100 = 146K.

then "can be produced" is again confusing. device
transmits packets but it does not "produce" them as such.
maybe you just mean "supported".

> >
> >
> >
> > > +
> > >  \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
> > >
> > >  The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
> > > @@ -326,6 +345,17 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > >  The device SHOULD NOT offer VIRTIO_NET_F_CTRL_RX_EXTRA if it
> > >  does not offer VIRTIO_NET_F_CTRL_VQ.
> > >
> > > +If VIRTIO_NET_F_HOST_TSO_LIMIT and VIRTIO_NET_F_MTU have been
> > > +negotiated, the device SHOULD set \field{tso_max_size} so that a TCP
> > > +segment that fully utilizes the configured MTU can be processed by TSO
> > > +(e.g., for IPv4 without options: at least \field{mtu} - 20; for IPv6
> > > +without extension headers: at least \field{mtu} - 40). This
> > > +recommendation does not account for IPv4 options or IPv6 extension
> > > +headers, which reduce the effective segment size.
> > > +
> > > +If VIRTIO_NET_F_HOST_TSO_LIMIT has been negotiated, the device MUST
> > > +set \field{tso_max_segs} to at least 64.
> >
> > where does this 64 come from? pls document.
> 
> A simple backward compatibility which makes sure the value can make
> sure 64K TSO can be segmented with 1500 MTU.

2^16/64 == 1024  

not ~1500

And we don't know MTU is 1500, either.



-- 
MST

Re: [PATCH] virtio-net: introduce TSO limit feature

[Jason Wang]

On Wed, Oct 15, 2025 at 3:02 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Wed, Oct 15, 2025 at 12:29:13PM +0800, Jason Wang wrote:
> > On Tue, Oct 14, 2025 at 4:59 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > thanks for the patch! yet something to improve.
> > > I note issues only when encountered 1st time but please
> > > do go and check all patch for each issue.
> > >
> > >
> > > On Tue, Oct 14, 2025 at 12:22:43PM +0800, Jason Wang wrote:
> > > > This patch introduces TSO limit feature which allows the device to
> > > > advertise:
> > > >
> > > > - Maximum TCP length of a TSO packet or inner TSO packet when UDP
> > > >   tunnel is support
> > > > - Maximum number of segment that can be produced by the device after
> > > >   segmentation of TSO or inner TSO packet of a UDP tunnel
> > > >
> > > > This is a must to implement TCP jumbogram, as networking stack needs
> > > > to know the limitation of the device in order to produce TSO packet as
> > > > large as possible.
>
>
> Maybe you can provide an overview of how this all is used?

Sure, this feature does not implement BIG TCP (jumbogram) itself. BIG
TCP requires a little bit more to be implemented:

1) new gso_type for jumbogram
2) when TCP length exceeds 64K, we should mandate ip->tot_len to be
zero and device can judge from the length of the buffer
3) other stuffs

I guess this is something that Viet wants to work on.

But before BIG TCP, the driver needs to know the device limitation of
TSO packets. That is what this patch did. BIG TCP is not the only
user, it could be used in the software datapath as well. Consider a
simple datapath:

virtio-net -> TAP -> bridge -> eth0

If eth0's tso_max_size is less than 64K, we need to advertise this to
virtio-net otherwise 64K gso packets will be segmented by software run
xmit in eth0. We've encountered this in collaboration with mana and
virtio-net/vhost.

> I am looking at tcp_fragment and I just do not see
> where it looks at any device limits.
>

It works like a device advertising tso_max_size/segs, and it is capped
by gso_max_size/segs. The gso_max_size/segs were the ones that are
used by the stack to determine the skb->len.

Thanks

> --
> MST
>

Re: [PATCH] virtio-net: introduce TSO limit feature

[Jason Wang]

On Wed, Oct 15, 2025 at 3:27 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> Thanks for the answers. Some more comments:
>
> On Wed, Oct 15, 2025 at 12:29:13PM +0800, Jason Wang wrote:
> > > >  device-types/net/description.tex | 46 ++++++++++++++++++++++++++++++++
> > > >  1 file changed, 46 insertions(+)
> > > >
> > > > diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> > > > index 415c7fd..e56df75 100644
> > > > --- a/device-types/net/description.tex
> > > > +++ b/device-types/net/description.tex
> > > > @@ -146,6 +146,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > > >   when VIRTIO_NET_F_IPSEC is negotiated. When a device offers IPsec feature, it SHOULD
> > > >   also offer the VIRTIO_NET_F_OUT_NET_HEADER feature.
> > > >
> > > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT(71)] Device limits the maximum TCP
> > > > +  length and the number of segments when performing TCP segmentation.
> > > > +
> > > >  \end{description}
> > > >
> > > >  \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
> > > > @@ -184,6 +187,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> > > >  \item[VIRTIO_NET_F_VQ_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> > > >  \item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ along with VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> > > >  \item[VIRTIO_NET_F_RSS_CONTEXT] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> > > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6
> > > >  \end{description}
> > > >
> > > >  \begin{note}
> > > > @@ -220,6 +224,8 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > >          le16 rss_max_indirection_table_length;
> > > >          le32 supported_hash_types;
> > > >          le32 supported_tunnel_types;
> > > > +        le32 tso_max_size;
> > > > +        le32 tso_max_segs;
> > > >  };
> > > >  \end{lstlisting}
> > > >
> > > > @@ -276,6 +282,19 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > >  Encapsulation types are defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets /
> > > >  Hash calculation for incoming packets / Encapsulation types supported/enabled for inner header hash}.
> > > >
> > > > +The following field, \field{tso_max_size} only exists if
> > > > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > > > +It specifies the maximum TCP length
> > >
> > > what is TCP length?
> >
> > It's defined in the rfc793:
> >
> > """
> >       The TCP Length is the TCP header length plus the data length in
> >       octets (this is not an explicitly transmitted quantity, but is
> >       computed), and it does not count the 12 octets of the pseudo
> >       header.
> > """
>
>
> But that one is 16 bit so can not exceed 65535.

I just reuse the terminology instead of defining something new. Note
that it is only used in pseudo header for csum after device has
performed TSO. The value in the pseudo header is capped by MTU/MSS.

As replied in another thread, BIG TCP requires more work or features.
Driver needs to set ip->tot_len 0 with a new gso type to let the
device know about BIG TCP packet.

>
> > >
> > > > of a TSO packet
> > >
> > > what is a TSO packet?
> >
> > Packet for device to perform TCP segmentation offload.
>
> pls define terms before use.

I may miss something but TSO has been widely used in the spec before
this feature:

"""
\item[VIRTIO_NET_F_GUEST_ECN (9)] Driver can receive TSO with ECN.
...
\item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can receive TSOv4.
...
"""

>
> > > > that the
> > > > +device can process.
> > >
> > > process in which direction? you mean device can receive?
> >
> > It works only for TX (as TSO works only for TX).
>
>
> rest of spec says "device receives from driver"  for this.
> process is ambiguous

A quick grep doesn't show me things like this, maybe you can point out
the location. Not a native speaker, but using "device receives from
driver" is indeed ambiguous for TX.

>
>
>
>
> > >
> > > > When VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set,
> > > > +it specifies the maximum inner TCP length of a UDP tunnel TSO packet
> > > > +that the device can process.
> > >
> > > Rest of spec talks of " GSO over UDP tunnels packets" is this the same?
> >
> > Not exactly the same, this is only for TSO not genreal GSO.
>
> rest of spec mostly talks of GSO. in fact virtio tso is a kind of
> accelerated gso.

This only applies for some specific software datapath like vhost-net.
But it doesn't apply to others especially the hardware device who will
do real TSO.

> either do the same or add a lot of text
> explaining tso as opposed to gso.

Is this ok to say "UDP tunnel VIRTIO_NET_HDR_GSO_TCPV4 or
VIRTIO_NET_HDR_GSO_TCPV6 packet"?

>
> > >
> > > even if it's actually unused?
> > >
> > > this, on the assumption that the length for tunnel is smaller?
> >
> > It means the device should have the same limitation for plain TSO and
> > tunnel TSO.
>
> Hmm. I have doubts how it can work given the overhead.

If a device can't afford the same limitation, it can simply not
advertise this feature. The reason I don't introduce a dedicated
limitation for tunnel is that there could be more tunnel supported in
the future, it would be a burden to have a per tunnel type limitation.

>
> > >
> > > I think this kind of things should be explicit.
> > >
> > >
> > > > +
> > > > +The following field, \field{tso_max_segs} only exists if
> > > > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > > > +It specifies the maximum number of segments that can be produced by
> > > > +the device after performing segmentation on TSO packet or a UDP tunnel
> > > > +TSO packet (when VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set).
> > >
> > > I don't get this field at all. the assumption is that all segments
> > > are the same size, right? Then it is just based on length?
> >
> > It's the device side limitation, for example a device can produce 100
> > segments at most, even if the tso_max_size is 256K, when MTU is 1500,
> > the driver can't send a TSO packet whose TCP length is greater than
> > (1500 - 20 - 20) * 100 = 146K.
>
> then "can be produced" is again confusing. device
> transmits packets but it does not "produce" them as such.
> maybe you just mean "supported".

I think yes.

>
> > >
> > >
> > >
> > > > +
> > > >  \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
> > > >
> > > >  The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
> > > > @@ -326,6 +345,17 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > >  The device SHOULD NOT offer VIRTIO_NET_F_CTRL_RX_EXTRA if it
> > > >  does not offer VIRTIO_NET_F_CTRL_VQ.
> > > >
> > > > +If VIRTIO_NET_F_HOST_TSO_LIMIT and VIRTIO_NET_F_MTU have been
> > > > +negotiated, the device SHOULD set \field{tso_max_size} so that a TCP
> > > > +segment that fully utilizes the configured MTU can be processed by TSO
> > > > +(e.g., for IPv4 without options: at least \field{mtu} - 20; for IPv6
> > > > +without extension headers: at least \field{mtu} - 40). This
> > > > +recommendation does not account for IPv4 options or IPv6 extension
> > > > +headers, which reduce the effective segment size.
> > > > +
> > > > +If VIRTIO_NET_F_HOST_TSO_LIMIT has been negotiated, the device MUST
> > > > +set \field{tso_max_segs} to at least 64.
> > >
> > > where does this 64 come from? pls document.
> >
> > A simple backward compatibility which makes sure the value can make
> > sure 64K TSO can be segmented with 1500 MTU.
>
> 2^16/64 == 1024
>
> not ~1500

Yes, I just choose one that is sufficient.

>
> And we don't know MTU is 1500, either.

A typical configuration but I can remove this part.

Thanks

>
>
>
> --
> MST
>

Re: [PATCH] virtio-net: introduce TSO limit feature

[Michael S. Tsirkin]

On Thu, Oct 16, 2025 at 01:57:41PM +0800, Jason Wang wrote:
> On Wed, Oct 15, 2025 at 3:27 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > Thanks for the answers. Some more comments:
> >
> > On Wed, Oct 15, 2025 at 12:29:13PM +0800, Jason Wang wrote:
> > > > >  device-types/net/description.tex | 46 ++++++++++++++++++++++++++++++++
> > > > >  1 file changed, 46 insertions(+)
> > > > >
> > > > > diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> > > > > index 415c7fd..e56df75 100644
> > > > > --- a/device-types/net/description.tex
> > > > > +++ b/device-types/net/description.tex
> > > > > @@ -146,6 +146,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > > > >   when VIRTIO_NET_F_IPSEC is negotiated. When a device offers IPsec feature, it SHOULD
> > > > >   also offer the VIRTIO_NET_F_OUT_NET_HEADER feature.
> > > > >
> > > > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT(71)] Device limits the maximum TCP
> > > > > +  length and the number of segments when performing TCP segmentation.
> > > > > +
> > > > >  \end{description}
> > > > >
> > > > >  \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
> > > > > @@ -184,6 +187,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> > > > >  \item[VIRTIO_NET_F_VQ_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> > > > >  \item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ along with VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> > > > >  \item[VIRTIO_NET_F_RSS_CONTEXT] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> > > > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6
> > > > >  \end{description}
> > > > >
> > > > >  \begin{note}
> > > > > @@ -220,6 +224,8 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > >          le16 rss_max_indirection_table_length;
> > > > >          le32 supported_hash_types;
> > > > >          le32 supported_tunnel_types;
> > > > > +        le32 tso_max_size;
> > > > > +        le32 tso_max_segs;
> > > > >  };
> > > > >  \end{lstlisting}
> > > > >
> > > > > @@ -276,6 +282,19 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > >  Encapsulation types are defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets /
> > > > >  Hash calculation for incoming packets / Encapsulation types supported/enabled for inner header hash}.
> > > > >
> > > > > +The following field, \field{tso_max_size} only exists if
> > > > > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > > > > +It specifies the maximum TCP length
> > > >
> > > > what is TCP length?
> > >
> > > It's defined in the rfc793:
> > >
> > > """
> > >       The TCP Length is the TCP header length plus the data length in
> > >       octets (this is not an explicitly transmitted quantity, but is
> > >       computed), and it does not count the 12 octets of the pseudo
> > >       header.
> > > """
> >
> >
> > But that one is 16 bit so can not exceed 65535.
> 
> I just reuse the terminology instead of defining something new.

Let's use a generic term that will work with big tcp.

> Note
> that it is only used in pseudo header for csum after device has
> performed TSO. The value in the pseudo header is capped by MTU/MSS.
> 
> As replied in another thread, BIG TCP requires more work or features.
> Driver needs to set ip->tot_len 0 with a new gso type to let the
> device know about BIG TCP packet.
> 
> >
> > > >
> > > > > of a TSO packet
> > > >
> > > > what is a TSO packet?
> > >
> > > Packet for device to perform TCP segmentation offload.
> >
> > pls define terms before use.
> 
> I may miss something but TSO has been widely used in the spec before
> this feature:
> 
> """
> \item[VIRTIO_NET_F_GUEST_ECN (9)] Driver can receive TSO with ECN.
> ...
> \item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can receive TSOv4.
> ...
> """


Yes but that does not define a "TSO packet".


> >
> > > > > that the
> > > > > +device can process.
> > > >
> > > > process in which direction? you mean device can receive?
> > >
> > > It works only for TX (as TSO works only for TX).
> >
> >
> > rest of spec says "device receives from driver"  for this.
> > process is ambiguous
> 
> A quick grep doesn't show me things like this, maybe you can point out
> the location. Not a native speaker, but using "device receives from
> driver" is indeed ambiguous for TX.

Well right near the text you quoted:

\item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can receive TSOv4.

\item[VIRTIO_NET_F_HOST_TSO6 (12)] Device can receive TSOv6.




> >
> >
> >
> >
> > > >
> > > > > When VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set,
> > > > > +it specifies the maximum inner TCP length of a UDP tunnel TSO packet
> > > > > +that the device can process.
> > > >
> > > > Rest of spec talks of " GSO over UDP tunnels packets" is this the same?
> > >
> > > Not exactly the same, this is only for TSO not genreal GSO.
> >
> > rest of spec mostly talks of GSO. in fact virtio tso is a kind of
> > accelerated gso.
> 
> This only applies for some specific software datapath like vhost-net.
> But it doesn't apply to others especially the hardware device who will
> do real TSO.

My point is that once you have said both GSO and TSO in the same
sentence, any reader's eyes have glazed over.




> > either do the same or add a lot of text
> > explaining tso as opposed to gso.
> 
> Is this ok to say "UDP tunnel VIRTIO_NET_HDR_GSO_TCPV4 or
> VIRTIO_NET_HDR_GSO_TCPV6 packet"?

Do you maybe mean: \field{gso_type} set to VIRTIO_NET_HDR_GSO_TCPV4
or VIRTIO_NET_HDR_GSO_TCPV6 



> >
> > > >
> > > > even if it's actually unused?
> > > >
> > > > this, on the assumption that the length for tunnel is smaller?
> > >
> > > It means the device should have the same limitation for plain TSO and
> > > tunnel TSO.
> >
> > Hmm. I have doubts how it can work given the overhead.
> 
> If a device can't afford the same limitation, it can simply not
> advertise this feature. The reason I don't introduce a dedicated
> limitation for tunnel is that there could be more tunnel supported in
> the future, it would be a burden to have a per tunnel type limitation.

Well presumably the feature is needed no?



> >
> > > >
> > > > I think this kind of things should be explicit.
> > > >
> > > >
> > > > > +
> > > > > +The following field, \field{tso_max_segs} only exists if
> > > > > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > > > > +It specifies the maximum number of segments that can be produced by
> > > > > +the device after performing segmentation on TSO packet or a UDP tunnel
> > > > > +TSO packet (when VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set).
> > > >
> > > > I don't get this field at all. the assumption is that all segments
> > > > are the same size, right? Then it is just based on length?
> > >
> > > It's the device side limitation, for example a device can produce 100
> > > segments at most, even if the tso_max_size is 256K, when MTU is 1500,
> > > the driver can't send a TSO packet whose TCP length is greater than
> > > (1500 - 20 - 20) * 100 = 146K.
> >
> > then "can be produced" is again confusing. device
> > transmits packets but it does not "produce" them as such.
> > maybe you just mean "supported".
> 
> I think yes.
> 
> >
> > > >
> > > >
> > > >
> > > > > +
> > > > >  \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
> > > > >
> > > > >  The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
> > > > > @@ -326,6 +345,17 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > >  The device SHOULD NOT offer VIRTIO_NET_F_CTRL_RX_EXTRA if it
> > > > >  does not offer VIRTIO_NET_F_CTRL_VQ.
> > > > >
> > > > > +If VIRTIO_NET_F_HOST_TSO_LIMIT and VIRTIO_NET_F_MTU have been
> > > > > +negotiated, the device SHOULD set \field{tso_max_size} so that a TCP
> > > > > +segment that fully utilizes the configured MTU can be processed by TSO
> > > > > +(e.g., for IPv4 without options: at least \field{mtu} - 20; for IPv6
> > > > > +without extension headers: at least \field{mtu} - 40). This
> > > > > +recommendation does not account for IPv4 options or IPv6 extension
> > > > > +headers, which reduce the effective segment size.
> > > > > +
> > > > > +If VIRTIO_NET_F_HOST_TSO_LIMIT has been negotiated, the device MUST
> > > > > +set \field{tso_max_segs} to at least 64.
> > > >
> > > > where does this 64 come from? pls document.
> > >
> > > A simple backward compatibility which makes sure the value can make
> > > sure 64K TSO can be segmented with 1500 MTU.
> >
> > 2^16/64 == 1024
> >
> > not ~1500
> 
> Yes, I just choose one that is sufficient.
> 
> >
> > And we don't know MTU is 1500, either.
> 
> A typical configuration but I can remove this part.
> 
> Thanks
> 
> >
> >
> >
> > --
> > MST
> >

Re: [PATCH] virtio-net: introduce TSO limit feature

[Michael S. Tsirkin]

On Thu, Oct 16, 2025 at 01:46:58PM +0800, Jason Wang wrote:
> On Wed, Oct 15, 2025 at 3:02 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Wed, Oct 15, 2025 at 12:29:13PM +0800, Jason Wang wrote:
> > > On Tue, Oct 14, 2025 at 4:59 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > >
> > > > thanks for the patch! yet something to improve.
> > > > I note issues only when encountered 1st time but please
> > > > do go and check all patch for each issue.
> > > >
> > > >
> > > > On Tue, Oct 14, 2025 at 12:22:43PM +0800, Jason Wang wrote:
> > > > > This patch introduces TSO limit feature which allows the device to
> > > > > advertise:
> > > > >
> > > > > - Maximum TCP length of a TSO packet or inner TSO packet when UDP
> > > > >   tunnel is support
> > > > > - Maximum number of segment that can be produced by the device after
> > > > >   segmentation of TSO or inner TSO packet of a UDP tunnel
> > > > >
> > > > > This is a must to implement TCP jumbogram, as networking stack needs
> > > > > to know the limitation of the device in order to produce TSO packet as
> > > > > large as possible.
> >
> >
> > Maybe you can provide an overview of how this all is used?
> 
> Sure, this feature does not implement BIG TCP (jumbogram) itself. BIG
> TCP requires a little bit more to be implemented:
> 
> 1) new gso_type for jumbogram
> 2) when TCP length exceeds 64K, we should mandate ip->tot_len to be
> zero and device can judge from the length of the buffer
> 3) other stuffs
> 
> I guess this is something that Viet wants to work on.
> 
> But before BIG TCP, the driver needs to know the device limitation of
> TSO packets. That is what this patch did. BIG TCP is not the only
> user, it could be used in the software datapath as well. Consider a
> simple datapath:
> 
> virtio-net -> TAP -> bridge -> eth0
> 
> If eth0's tso_max_size is less than 64K, we need to advertise this to
> virtio-net otherwise 64K gso packets will be segmented by software run
> xmit in eth0. We've encountered this in collaboration with mana and
> virtio-net/vhost.


Well then pls update with the real motivation.


> > I am looking at tcp_fragment and I just do not see
> > where it looks at any device limits.
> >
> 
> It works like a device advertising tso_max_size/segs, and it is capped
> by gso_max_size/segs. The gso_max_size/segs were the ones that are
> used by the stack to determine the skb->len.
> 
> Thanks
> 
> > --
> > MST
> >

Re: [PATCH] virtio-net: introduce TSO limit feature

[Michael S. Tsirkin]

On Thu, Oct 16, 2025 at 01:46:58PM +0800, Jason Wang wrote:
> On Wed, Oct 15, 2025 at 3:02 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Wed, Oct 15, 2025 at 12:29:13PM +0800, Jason Wang wrote:
> > > On Tue, Oct 14, 2025 at 4:59 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > >
> > > > thanks for the patch! yet something to improve.
> > > > I note issues only when encountered 1st time but please
> > > > do go and check all patch for each issue.
> > > >
> > > >
> > > > On Tue, Oct 14, 2025 at 12:22:43PM +0800, Jason Wang wrote:
> > > > > This patch introduces TSO limit feature which allows the device to
> > > > > advertise:
> > > > >
> > > > > - Maximum TCP length of a TSO packet or inner TSO packet when UDP
> > > > >   tunnel is support
> > > > > - Maximum number of segment that can be produced by the device after
> > > > >   segmentation of TSO or inner TSO packet of a UDP tunnel
> > > > >
> > > > > This is a must to implement TCP jumbogram, as networking stack needs
> > > > > to know the limitation of the device in order to produce TSO packet as
> > > > > large as possible.
> >
> >
> > Maybe you can provide an overview of how this all is used?
> 
> Sure, this feature does not implement BIG TCP (jumbogram) itself. BIG
> TCP requires a little bit more to be implemented:
> 
> 1) new gso_type for jumbogram
> 2) when TCP length exceeds 64K, we should mandate ip->tot_len to be
> zero and device can judge from the length of the buffer
> 3) other stuffs
> 
> I guess this is something that Viet wants to work on.
> 
> But before BIG TCP, the driver needs to know the device limitation of
> TSO packets. That is what this patch did. BIG TCP is not the only
> user, it could be used in the software datapath as well. Consider a
> simple datapath:
> 
> virtio-net -> TAP -> bridge -> eth0
> 
> If eth0's tso_max_size is less than 64K, we need to advertise this to
> virtio-net otherwise 64K gso packets will be segmented by software run
> xmit in eth0. We've encountered this in collaboration with mana and
> virtio-net/vhost.


Wait a second this does not make any sense. Bridge disables tso,
it only works with gso.


So what is the actual motivation?

> > I am looking at tcp_fragment and I just do not see
> > where it looks at any device limits.
> >
> 
> It works like a device advertising tso_max_size/segs, and it is capped
> by gso_max_size/segs. The gso_max_size/segs were the ones that are
> used by the stack to determine the skb->len.
> 
> Thanks
> 
> > --
> > MST
> >

Re: [PATCH] virtio-net: introduce TSO limit feature

[Jason Wang]

On Thu, Oct 16, 2025 at 2:19 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Thu, Oct 16, 2025 at 01:46:58PM +0800, Jason Wang wrote:
> > On Wed, Oct 15, 2025 at 3:02 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > On Wed, Oct 15, 2025 at 12:29:13PM +0800, Jason Wang wrote:
> > > > On Tue, Oct 14, 2025 at 4:59 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > > >
> > > > > thanks for the patch! yet something to improve.
> > > > > I note issues only when encountered 1st time but please
> > > > > do go and check all patch for each issue.
> > > > >
> > > > >
> > > > > On Tue, Oct 14, 2025 at 12:22:43PM +0800, Jason Wang wrote:
> > > > > > This patch introduces TSO limit feature which allows the device to
> > > > > > advertise:
> > > > > >
> > > > > > - Maximum TCP length of a TSO packet or inner TSO packet when UDP
> > > > > >   tunnel is support
> > > > > > - Maximum number of segment that can be produced by the device after
> > > > > >   segmentation of TSO or inner TSO packet of a UDP tunnel
> > > > > >
> > > > > > This is a must to implement TCP jumbogram, as networking stack needs
> > > > > > to know the limitation of the device in order to produce TSO packet as
> > > > > > large as possible.
> > >
> > >
> > > Maybe you can provide an overview of how this all is used?
> >
> > Sure, this feature does not implement BIG TCP (jumbogram) itself. BIG
> > TCP requires a little bit more to be implemented:
> >
> > 1) new gso_type for jumbogram
> > 2) when TCP length exceeds 64K, we should mandate ip->tot_len to be
> > zero and device can judge from the length of the buffer
> > 3) other stuffs
> >
> > I guess this is something that Viet wants to work on.
> >
> > But before BIG TCP, the driver needs to know the device limitation of
> > TSO packets. That is what this patch did. BIG TCP is not the only
> > user, it could be used in the software datapath as well. Consider a
> > simple datapath:
> >
> > virtio-net -> TAP -> bridge -> eth0
> >
> > If eth0's tso_max_size is less than 64K, we need to advertise this to
> > virtio-net otherwise 64K gso packets will be segmented by software run
> > xmit in eth0. We've encountered this in collaboration with mana and
> > virtio-net/vhost.
>
>
> Wait a second this does not make any sense. Bridge disables tso,
> it only works with gso.
>

I don't understand here, we have used similar setups for a very long
time and TSO works well?

Maybe my statement is not clear enough, I meant the forwarding path of
the bridge.

>
> So what is the actual motivation?

Thanks


>
> > > I am looking at tcp_fragment and I just do not see
> > > where it looks at any device limits.
> > >
> >
> > It works like a device advertising tso_max_size/segs, and it is capped
> > by gso_max_size/segs. The gso_max_size/segs were the ones that are
> > used by the stack to determine the skb->len.
> >
> > Thanks
> >
> > > --
> > > MST
> > >
>

Re: [PATCH] virtio-net: introduce TSO limit feature

[Jason Wang]

On Thu, Oct 16, 2025 at 2:17 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Thu, Oct 16, 2025 at 01:57:41PM +0800, Jason Wang wrote:
> > On Wed, Oct 15, 2025 at 3:27 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > Thanks for the answers. Some more comments:
> > >
> > > On Wed, Oct 15, 2025 at 12:29:13PM +0800, Jason Wang wrote:
> > > > > >  device-types/net/description.tex | 46 ++++++++++++++++++++++++++++++++
> > > > > >  1 file changed, 46 insertions(+)
> > > > > >
> > > > > > diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> > > > > > index 415c7fd..e56df75 100644
> > > > > > --- a/device-types/net/description.tex
> > > > > > +++ b/device-types/net/description.tex
> > > > > > @@ -146,6 +146,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > > > > >   when VIRTIO_NET_F_IPSEC is negotiated. When a device offers IPsec feature, it SHOULD
> > > > > >   also offer the VIRTIO_NET_F_OUT_NET_HEADER feature.
> > > > > >
> > > > > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT(71)] Device limits the maximum TCP
> > > > > > +  length and the number of segments when performing TCP segmentation.
> > > > > > +
> > > > > >  \end{description}
> > > > > >
> > > > > >  \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
> > > > > > @@ -184,6 +187,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> > > > > >  \item[VIRTIO_NET_F_VQ_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> > > > > >  \item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ along with VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> > > > > >  \item[VIRTIO_NET_F_RSS_CONTEXT] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> > > > > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6
> > > > > >  \end{description}
> > > > > >
> > > > > >  \begin{note}
> > > > > > @@ -220,6 +224,8 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > > >          le16 rss_max_indirection_table_length;
> > > > > >          le32 supported_hash_types;
> > > > > >          le32 supported_tunnel_types;
> > > > > > +        le32 tso_max_size;
> > > > > > +        le32 tso_max_segs;
> > > > > >  };
> > > > > >  \end{lstlisting}
> > > > > >
> > > > > > @@ -276,6 +282,19 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > > >  Encapsulation types are defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets /
> > > > > >  Hash calculation for incoming packets / Encapsulation types supported/enabled for inner header hash}.
> > > > > >
> > > > > > +The following field, \field{tso_max_size} only exists if
> > > > > > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > > > > > +It specifies the maximum TCP length
> > > > >
> > > > > what is TCP length?
> > > >
> > > > It's defined in the rfc793:
> > > >
> > > > """
> > > >       The TCP Length is the TCP header length plus the data length in
> > > >       octets (this is not an explicitly transmitted quantity, but is
> > > >       computed), and it does not count the 12 octets of the pseudo
> > > >       header.
> > > > """
> > >
> > >
> > > But that one is 16 bit so can not exceed 65535.
> >
> > I just reuse the terminology instead of defining something new.
>
> Let's use a generic term that will work with big tcp.

After glancing at RFC TCP length seems to be the best as TCP header
doesn't have a length.

Maybe you have something better in mind?

>
> > Note
> > that it is only used in pseudo header for csum after device has
> > performed TSO. The value in the pseudo header is capped by MTU/MSS.
> >
> > As replied in another thread, BIG TCP requires more work or features.
> > Driver needs to set ip->tot_len 0 with a new gso type to let the
> > device know about BIG TCP packet.
> >
> > >
> > > > >
> > > > > > of a TSO packet
> > > > >
> > > > > what is a TSO packet?
> > > >
> > > > Packet for device to perform TCP segmentation offload.
> > >
> > > pls define terms before use.
> >
> > I may miss something but TSO has been widely used in the spec before
> > this feature:
> >
> > """
> > \item[VIRTIO_NET_F_GUEST_ECN (9)] Driver can receive TSO with ECN.
> > ...
> > \item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can receive TSOv4.
> > ...
> > """
>
>
> Yes but that does not define a "TSO packet".

How about a patch beforehand to explain thing like:

\item[VIRTIO_NET_F_GUEST_ECN (9)] Driver can receive TSO (TCP
Segmentation Offload) with ECN.

>
>
> > >
> > > > > > that the
> > > > > > +device can process.
> > > > >
> > > > > process in which direction? you mean device can receive?
> > > >
> > > > It works only for TX (as TSO works only for TX).
> > >
> > >
> > > rest of spec says "device receives from driver"  for this.
> > > process is ambiguous
> >
> > A quick grep doesn't show me things like this, maybe you can point out
> > the location. Not a native speaker, but using "device receives from
> > driver" is indeed ambiguous for TX.
>
> Well right near the text you quoted:
>
> \item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can receive TSOv4.
>
> \item[VIRTIO_NET_F_HOST_TSO6 (12)] Device can receive TSOv6.

Okay, I think you want another to to replace it with

\item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can transmit TSOv4.

?

>
>
>
>
> > >
> > >
> > >
> > >
> > > > >
> > > > > > When VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set,
> > > > > > +it specifies the maximum inner TCP length of a UDP tunnel TSO packet
> > > > > > +that the device can process.
> > > > >
> > > > > Rest of spec talks of " GSO over UDP tunnels packets" is this the same?
> > > >
> > > > Not exactly the same, this is only for TSO not genreal GSO.
> > >
> > > rest of spec mostly talks of GSO. in fact virtio tso is a kind of
> > > accelerated gso.
> >
> > This only applies for some specific software datapath like vhost-net.
> > But it doesn't apply to others especially the hardware device who will
> > do real TSO.
>
> My point is that once you have said both GSO and TSO in the same
> sentence, any reader's eyes have glazed over.
>
>
>
>
> > > either do the same or add a lot of text
> > > explaining tso as opposed to gso.
> >
> > Is this ok to say "UDP tunnel VIRTIO_NET_HDR_GSO_TCPV4 or
> > VIRTIO_NET_HDR_GSO_TCPV6 packet"?
>
> Do you maybe mean: \field{gso_type} set to VIRTIO_NET_HDR_GSO_TCPV4
> or VIRTIO_NET_HDR_GSO_TCPV6

Yes.

>
>
>
> > >
> > > > >
> > > > > even if it's actually unused?
> > > > >
> > > > > this, on the assumption that the length for tunnel is smaller?
> > > >
> > > > It means the device should have the same limitation for plain TSO and
> > > > tunnel TSO.
> > >
> > > Hmm. I have doubts how it can work given the overhead.
> >
> > If a device can't afford the same limitation, it can simply not
> > advertise this feature. The reason I don't introduce a dedicated
> > limitation for tunnel is that there could be more tunnel supported in
> > the future, it would be a burden to have a per tunnel type limitation.
>
> Well presumably the feature is needed no?

Yes, but the kernel assumes the same limitation for tunnel and non
tunnel one. Having a single limitation simply things or if you stick I
can introduce a dedicated one for UDP tunnels.

>
>
>
> > >
> > > > >
> > > > > I think this kind of things should be explicit.
> > > > >
> > > > >
> > > > > > +
> > > > > > +The following field, \field{tso_max_segs} only exists if
> > > > > > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > > > > > +It specifies the maximum number of segments that can be produced by
> > > > > > +the device after performing segmentation on TSO packet or a UDP tunnel
> > > > > > +TSO packet (when VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set).
> > > > >
> > > > > I don't get this field at all. the assumption is that all segments
> > > > > are the same size, right? Then it is just based on length?
> > > >
> > > > It's the device side limitation, for example a device can produce 100
> > > > segments at most, even if the tso_max_size is 256K, when MTU is 1500,
> > > > the driver can't send a TSO packet whose TCP length is greater than
> > > > (1500 - 20 - 20) * 100 = 146K.
> > >
> > > then "can be produced" is again confusing. device
> > > transmits packets but it does not "produce" them as such.
> > > maybe you just mean "supported".
> >
> > I think yes.
> >
> > >
> > > > >
> > > > >
> > > > >
> > > > > > +
> > > > > >  \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
> > > > > >
> > > > > >  The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
> > > > > > @@ -326,6 +345,17 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > > >  The device SHOULD NOT offer VIRTIO_NET_F_CTRL_RX_EXTRA if it
> > > > > >  does not offer VIRTIO_NET_F_CTRL_VQ.
> > > > > >
> > > > > > +If VIRTIO_NET_F_HOST_TSO_LIMIT and VIRTIO_NET_F_MTU have been
> > > > > > +negotiated, the device SHOULD set \field{tso_max_size} so that a TCP
> > > > > > +segment that fully utilizes the configured MTU can be processed by TSO
> > > > > > +(e.g., for IPv4 without options: at least \field{mtu} - 20; for IPv6
> > > > > > +without extension headers: at least \field{mtu} - 40). This
> > > > > > +recommendation does not account for IPv4 options or IPv6 extension
> > > > > > +headers, which reduce the effective segment size.
> > > > > > +
> > > > > > +If VIRTIO_NET_F_HOST_TSO_LIMIT has been negotiated, the device MUST
> > > > > > +set \field{tso_max_segs} to at least 64.
> > > > >
> > > > > where does this 64 come from? pls document.
> > > >
> > > > A simple backward compatibility which makes sure the value can make
> > > > sure 64K TSO can be segmented with 1500 MTU.
> > >
> > > 2^16/64 == 1024
> > >
> > > not ~1500
> >
> > Yes, I just choose one that is sufficient.
> >
> > >
> > > And we don't know MTU is 1500, either.
> >
> > A typical configuration but I can remove this part.
> >
> > Thanks
> >
> > >
> > >
> > >
> > > --
> > > MST
> > >
>

Thanks

Re: [PATCH] virtio-net: introduce TSO limit feature

[Michael S. Tsirkin]

On Thu, Oct 16, 2025 at 02:31:46PM +0800, Jason Wang wrote:
> On Thu, Oct 16, 2025 at 2:17 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Thu, Oct 16, 2025 at 01:57:41PM +0800, Jason Wang wrote:
> > > On Wed, Oct 15, 2025 at 3:27 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > >
> > > > Thanks for the answers. Some more comments:
> > > >
> > > > On Wed, Oct 15, 2025 at 12:29:13PM +0800, Jason Wang wrote:
> > > > > > >  device-types/net/description.tex | 46 ++++++++++++++++++++++++++++++++
> > > > > > >  1 file changed, 46 insertions(+)
> > > > > > >
> > > > > > > diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> > > > > > > index 415c7fd..e56df75 100644
> > > > > > > --- a/device-types/net/description.tex
> > > > > > > +++ b/device-types/net/description.tex
> > > > > > > @@ -146,6 +146,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > > > > > >   when VIRTIO_NET_F_IPSEC is negotiated. When a device offers IPsec feature, it SHOULD
> > > > > > >   also offer the VIRTIO_NET_F_OUT_NET_HEADER feature.
> > > > > > >
> > > > > > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT(71)] Device limits the maximum TCP
> > > > > > > +  length and the number of segments when performing TCP segmentation.
> > > > > > > +
> > > > > > >  \end{description}
> > > > > > >
> > > > > > >  \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
> > > > > > > @@ -184,6 +187,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> > > > > > >  \item[VIRTIO_NET_F_VQ_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> > > > > > >  \item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ along with VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> > > > > > >  \item[VIRTIO_NET_F_RSS_CONTEXT] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> > > > > > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6
> > > > > > >  \end{description}
> > > > > > >
> > > > > > >  \begin{note}
> > > > > > > @@ -220,6 +224,8 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > > > >          le16 rss_max_indirection_table_length;
> > > > > > >          le32 supported_hash_types;
> > > > > > >          le32 supported_tunnel_types;
> > > > > > > +        le32 tso_max_size;
> > > > > > > +        le32 tso_max_segs;
> > > > > > >  };
> > > > > > >  \end{lstlisting}
> > > > > > >
> > > > > > > @@ -276,6 +282,19 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > > > >  Encapsulation types are defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets /
> > > > > > >  Hash calculation for incoming packets / Encapsulation types supported/enabled for inner header hash}.
> > > > > > >
> > > > > > > +The following field, \field{tso_max_size} only exists if
> > > > > > > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > > > > > > +It specifies the maximum TCP length
> > > > > >
> > > > > > what is TCP length?
> > > > >
> > > > > It's defined in the rfc793:
> > > > >
> > > > > """
> > > > >       The TCP Length is the TCP header length plus the data length in
> > > > >       octets (this is not an explicitly transmitted quantity, but is
> > > > >       computed), and it does not count the 12 octets of the pseudo
> > > > >       header.
> > > > > """
> > > >
> > > >
> > > > But that one is 16 bit so can not exceed 65535.
> > >
> > > I just reuse the terminology instead of defining something new.
> >
> > Let's use a generic term that will work with big tcp.
> 
> After glancing at RFC TCP length seems to be the best as TCP header
> doesn't have a length.
> 
> Maybe you have something better in mind?


Depends on what do you mean by it, exactly.

> >
> > > Note
> > > that it is only used in pseudo header for csum after device has
> > > performed TSO. The value in the pseudo header is capped by MTU/MSS.
> > >
> > > As replied in another thread, BIG TCP requires more work or features.
> > > Driver needs to set ip->tot_len 0 with a new gso type to let the
> > > device know about BIG TCP packet.
> > >
> > > >
> > > > > >
> > > > > > > of a TSO packet
> > > > > >
> > > > > > what is a TSO packet?
> > > > >
> > > > > Packet for device to perform TCP segmentation offload.
> > > >
> > > > pls define terms before use.
> > >
> > > I may miss something but TSO has been widely used in the spec before
> > > this feature:
> > >
> > > """
> > > \item[VIRTIO_NET_F_GUEST_ECN (9)] Driver can receive TSO with ECN.
> > > ...
> > > \item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can receive TSOv4.
> > > ...
> > > """
> >
> >
> > Yes but that does not define a "TSO packet".
> 
> How about a patch beforehand to explain thing like:
> 
> \item[VIRTIO_NET_F_GUEST_ECN (9)] Driver can receive TSO (TCP
> Segmentation Offload) with ECN.

That does not define is either?

virtio$ git grep 'TSO packet'
virtio$ 





> >
> >
> > > >
> > > > > > > that the
> > > > > > > +device can process.
> > > > > >
> > > > > > process in which direction? you mean device can receive?
> > > > >
> > > > > It works only for TX (as TSO works only for TX).
> > > >
> > > >
> > > > rest of spec says "device receives from driver"  for this.
> > > > process is ambiguous
> > >
> > > A quick grep doesn't show me things like this, maybe you can point out
> > > the location. Not a native speaker, but using "device receives from
> > > driver" is indeed ambiguous for TX.
> >
> > Well right near the text you quoted:
> >
> > \item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can receive TSOv4.
> >
> > \item[VIRTIO_NET_F_HOST_TSO6 (12)] Device can receive TSOv6.
> 
> Okay, I think you want another to to replace it with
> 
> \item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can transmit TSOv4.
> 
> ?


"receive" in this context is "receive from driver".


TSO is not something that is transmitted on the wire.




> >
> >
> >
> >
> > > >
> > > >
> > > >
> > > >
> > > > > >
> > > > > > > When VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set,
> > > > > > > +it specifies the maximum inner TCP length of a UDP tunnel TSO packet
> > > > > > > +that the device can process.
> > > > > >
> > > > > > Rest of spec talks of " GSO over UDP tunnels packets" is this the same?
> > > > >
> > > > > Not exactly the same, this is only for TSO not genreal GSO.
> > > >
> > > > rest of spec mostly talks of GSO. in fact virtio tso is a kind of
> > > > accelerated gso.
> > >
> > > This only applies for some specific software datapath like vhost-net.
> > > But it doesn't apply to others especially the hardware device who will
> > > do real TSO.
> >
> > My point is that once you have said both GSO and TSO in the same
> > sentence, any reader's eyes have glazed over.
> >
> >
> >
> >
> > > > either do the same or add a lot of text
> > > > explaining tso as opposed to gso.
> > >
> > > Is this ok to say "UDP tunnel VIRTIO_NET_HDR_GSO_TCPV4 or
> > > VIRTIO_NET_HDR_GSO_TCPV6 packet"?
> >
> > Do you maybe mean: \field{gso_type} set to VIRTIO_NET_HDR_GSO_TCPV4
> > or VIRTIO_NET_HDR_GSO_TCPV6
> 
> Yes.
> 
> >
> >
> >
> > > >
> > > > > >
> > > > > > even if it's actually unused?
> > > > > >
> > > > > > this, on the assumption that the length for tunnel is smaller?
> > > > >
> > > > > It means the device should have the same limitation for plain TSO and
> > > > > tunnel TSO.
> > > >
> > > > Hmm. I have doubts how it can work given the overhead.
> > >
> > > If a device can't afford the same limitation, it can simply not
> > > advertise this feature. The reason I don't introduce a dedicated
> > > limitation for tunnel is that there could be more tunnel supported in
> > > the future, it would be a burden to have a per tunnel type limitation.
> >
> > Well presumably the feature is needed no?
> 
> Yes, but the kernel assumes the same limitation for tunnel and non
> tunnel one. Having a single limitation simply things or if you stick I
> can introduce a dedicated one for UDP tunnels.

I don't insist, I think it is worth clarifying though.

> >
> >
> >
> > > >
> > > > > >
> > > > > > I think this kind of things should be explicit.
> > > > > >
> > > > > >
> > > > > > > +
> > > > > > > +The following field, \field{tso_max_segs} only exists if
> > > > > > > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > > > > > > +It specifies the maximum number of segments that can be produced by
> > > > > > > +the device after performing segmentation on TSO packet or a UDP tunnel
> > > > > > > +TSO packet (when VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set).
> > > > > >
> > > > > > I don't get this field at all. the assumption is that all segments
> > > > > > are the same size, right? Then it is just based on length?
> > > > >
> > > > > It's the device side limitation, for example a device can produce 100
> > > > > segments at most, even if the tso_max_size is 256K, when MTU is 1500,
> > > > > the driver can't send a TSO packet whose TCP length is greater than
> > > > > (1500 - 20 - 20) * 100 = 146K.
> > > >
> > > > then "can be produced" is again confusing. device
> > > > transmits packets but it does not "produce" them as such.
> > > > maybe you just mean "supported".
> > >
> > > I think yes.
> > >
> > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > > +
> > > > > > >  \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
> > > > > > >
> > > > > > >  The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
> > > > > > > @@ -326,6 +345,17 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > > > >  The device SHOULD NOT offer VIRTIO_NET_F_CTRL_RX_EXTRA if it
> > > > > > >  does not offer VIRTIO_NET_F_CTRL_VQ.
> > > > > > >
> > > > > > > +If VIRTIO_NET_F_HOST_TSO_LIMIT and VIRTIO_NET_F_MTU have been
> > > > > > > +negotiated, the device SHOULD set \field{tso_max_size} so that a TCP
> > > > > > > +segment that fully utilizes the configured MTU can be processed by TSO
> > > > > > > +(e.g., for IPv4 without options: at least \field{mtu} - 20; for IPv6
> > > > > > > +without extension headers: at least \field{mtu} - 40). This
> > > > > > > +recommendation does not account for IPv4 options or IPv6 extension
> > > > > > > +headers, which reduce the effective segment size.
> > > > > > > +
> > > > > > > +If VIRTIO_NET_F_HOST_TSO_LIMIT has been negotiated, the device MUST
> > > > > > > +set \field{tso_max_segs} to at least 64.
> > > > > >
> > > > > > where does this 64 come from? pls document.
> > > > >
> > > > > A simple backward compatibility which makes sure the value can make
> > > > > sure 64K TSO can be segmented with 1500 MTU.
> > > >
> > > > 2^16/64 == 1024
> > > >
> > > > not ~1500
> > >
> > > Yes, I just choose one that is sufficient.
> > >
> > > >
> > > > And we don't know MTU is 1500, either.
> > >
> > > A typical configuration but I can remove this part.
> > >
> > > Thanks
> > >
> > > >
> > > >
> > > >
> > > > --
> > > > MST
> > > >
> >
> 
> Thanks

Re: [PATCH] virtio-net: introduce TSO limit feature

[Michael S. Tsirkin]

On Thu, Oct 16, 2025 at 02:22:53PM +0800, Jason Wang wrote:
> > Wait a second this does not make any sense. Bridge disables tso,
> > it only works with gso.
> >
> 
> I don't understand here, we have used similar setups for a very long
> time and TSO works well?


My bad, I was thinking about LRO/GRO.


-- 
MST

Re: [PATCH] virtio-net: introduce TSO limit feature

[Jason Wang]

On Thu, Oct 16, 2025 at 6:03 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Thu, Oct 16, 2025 at 02:31:46PM +0800, Jason Wang wrote:
> > On Thu, Oct 16, 2025 at 2:17 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > On Thu, Oct 16, 2025 at 01:57:41PM +0800, Jason Wang wrote:
> > > > On Wed, Oct 15, 2025 at 3:27 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > > >
> > > > > Thanks for the answers. Some more comments:
> > > > >
> > > > > On Wed, Oct 15, 2025 at 12:29:13PM +0800, Jason Wang wrote:
> > > > > > > >  device-types/net/description.tex | 46 ++++++++++++++++++++++++++++++++
> > > > > > > >  1 file changed, 46 insertions(+)
> > > > > > > >
> > > > > > > > diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> > > > > > > > index 415c7fd..e56df75 100644
> > > > > > > > --- a/device-types/net/description.tex
> > > > > > > > +++ b/device-types/net/description.tex
> > > > > > > > @@ -146,6 +146,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > > > > > > >   when VIRTIO_NET_F_IPSEC is negotiated. When a device offers IPsec feature, it SHOULD
> > > > > > > >   also offer the VIRTIO_NET_F_OUT_NET_HEADER feature.
> > > > > > > >
> > > > > > > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT(71)] Device limits the maximum TCP
> > > > > > > > +  length and the number of segments when performing TCP segmentation.
> > > > > > > > +
> > > > > > > >  \end{description}
> > > > > > > >
> > > > > > > >  \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
> > > > > > > > @@ -184,6 +187,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> > > > > > > >  \item[VIRTIO_NET_F_VQ_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> > > > > > > >  \item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ along with VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> > > > > > > >  \item[VIRTIO_NET_F_RSS_CONTEXT] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> > > > > > > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6
> > > > > > > >  \end{description}
> > > > > > > >
> > > > > > > >  \begin{note}
> > > > > > > > @@ -220,6 +224,8 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > > > > >          le16 rss_max_indirection_table_length;
> > > > > > > >          le32 supported_hash_types;
> > > > > > > >          le32 supported_tunnel_types;
> > > > > > > > +        le32 tso_max_size;
> > > > > > > > +        le32 tso_max_segs;
> > > > > > > >  };
> > > > > > > >  \end{lstlisting}
> > > > > > > >
> > > > > > > > @@ -276,6 +282,19 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > > > > >  Encapsulation types are defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets /
> > > > > > > >  Hash calculation for incoming packets / Encapsulation types supported/enabled for inner header hash}.
> > > > > > > >
> > > > > > > > +The following field, \field{tso_max_size} only exists if
> > > > > > > > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > > > > > > > +It specifies the maximum TCP length
> > > > > > >
> > > > > > > what is TCP length?
> > > > > >
> > > > > > It's defined in the rfc793:
> > > > > >
> > > > > > """
> > > > > >       The TCP Length is the TCP header length plus the data length in
> > > > > >       octets (this is not an explicitly transmitted quantity, but is
> > > > > >       computed), and it does not count the 12 octets of the pseudo
> > > > > >       header.
> > > > > > """
> > > > >
> > > > >
> > > > > But that one is 16 bit so can not exceed 65535.
> > > >
> > > > I just reuse the terminology instead of defining something new.
> > >
> > > Let's use a generic term that will work with big tcp.
> >
> > After glancing at RFC TCP length seems to be the best as TCP header
> > doesn't have a length.
> >
> > Maybe you have something better in mind?
>
>
> Depends on what do you mean by it, exactly.

TCP header length plus the data length. Maybe TCP packet length?

>
> > >
> > > > Note
> > > > that it is only used in pseudo header for csum after device has
> > > > performed TSO. The value in the pseudo header is capped by MTU/MSS.
> > > >
> > > > As replied in another thread, BIG TCP requires more work or features.
> > > > Driver needs to set ip->tot_len 0 with a new gso type to let the
> > > > device know about BIG TCP packet.
> > > >
> > > > >
> > > > > > >
> > > > > > > > of a TSO packet
> > > > > > >
> > > > > > > what is a TSO packet?
> > > > > >
> > > > > > Packet for device to perform TCP segmentation offload.
> > > > >
> > > > > pls define terms before use.
> > > >
> > > > I may miss something but TSO has been widely used in the spec before
> > > > this feature:
> > > >
> > > > """
> > > > \item[VIRTIO_NET_F_GUEST_ECN (9)] Driver can receive TSO with ECN.
> > > > ...
> > > > \item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can receive TSOv4.
> > > > ...
> > > > """
> > >
> > >
> > > Yes but that does not define a "TSO packet".
> >
> > How about a patch beforehand to explain thing like:
> >
> > \item[VIRTIO_NET_F_GUEST_ECN (9)] Driver can receive TSO (TCP
> > Segmentation Offload) with ECN.
>
> That does not define is either?
>
> virtio$ git grep 'TSO packet'
> virtio$

Something like this?

Maybe "TSO packet is the packet that device may segment it into
multiple frames"?


>
>
>
>
>
> > >
> > >
> > > > >
> > > > > > > > that the
> > > > > > > > +device can process.
> > > > > > >
> > > > > > > process in which direction? you mean device can receive?
> > > > > >
> > > > > > It works only for TX (as TSO works only for TX).
> > > > >
> > > > >
> > > > > rest of spec says "device receives from driver"  for this.
> > > > > process is ambiguous
> > > >
> > > > A quick grep doesn't show me things like this, maybe you can point out
> > > > the location. Not a native speaker, but using "device receives from
> > > > driver" is indeed ambiguous for TX.
> > >
> > > Well right near the text you quoted:
> > >
> > > \item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can receive TSOv4.
> > >
> > > \item[VIRTIO_NET_F_HOST_TSO6 (12)] Device can receive TSOv6.
> >
> > Okay, I think you want another to to replace it with
> >
> > \item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can transmit TSOv4.
> >
> > ?
>
>
> "receive" in this context is "receive from driver".
>
>
> TSO is not something that is transmitted on the wire.

Ok.

>
>
>
>
> > >
> > >
> > >
> > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > > >
> > > > > > > > When VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set,
> > > > > > > > +it specifies the maximum inner TCP length of a UDP tunnel TSO packet
> > > > > > > > +that the device can process.
> > > > > > >
> > > > > > > Rest of spec talks of " GSO over UDP tunnels packets" is this the same?
> > > > > >
> > > > > > Not exactly the same, this is only for TSO not genreal GSO.
> > > > >
> > > > > rest of spec mostly talks of GSO. in fact virtio tso is a kind of
> > > > > accelerated gso.
> > > >
> > > > This only applies for some specific software datapath like vhost-net.
> > > > But it doesn't apply to others especially the hardware device who will
> > > > do real TSO.
> > >
> > > My point is that once you have said both GSO and TSO in the same
> > > sentence, any reader's eyes have glazed over.
> > >
> > >
> > >
> > >
> > > > > either do the same or add a lot of text
> > > > > explaining tso as opposed to gso.
> > > >
> > > > Is this ok to say "UDP tunnel VIRTIO_NET_HDR_GSO_TCPV4 or
> > > > VIRTIO_NET_HDR_GSO_TCPV6 packet"?
> > >
> > > Do you maybe mean: \field{gso_type} set to VIRTIO_NET_HDR_GSO_TCPV4
> > > or VIRTIO_NET_HDR_GSO_TCPV6
> >
> > Yes.
> >
> > >
> > >
> > >
> > > > >
> > > > > > >
> > > > > > > even if it's actually unused?
> > > > > > >
> > > > > > > this, on the assumption that the length for tunnel is smaller?
> > > > > >
> > > > > > It means the device should have the same limitation for plain TSO and
> > > > > > tunnel TSO.
> > > > >
> > > > > Hmm. I have doubts how it can work given the overhead.
> > > >
> > > > If a device can't afford the same limitation, it can simply not
> > > > advertise this feature. The reason I don't introduce a dedicated
> > > > limitation for tunnel is that there could be more tunnel supported in
> > > > the future, it would be a burden to have a per tunnel type limitation.
> > >
> > > Well presumably the feature is needed no?
> >
> > Yes, but the kernel assumes the same limitation for tunnel and non
> > tunnel one. Having a single limitation simply things or if you stick I
> > can introduce a dedicated one for UDP tunnels.
>
> I don't insist, I think it is worth clarifying though.

Ok how about this:

"""
If a device can not have a single limitation for both plain GSO and
tunnel GSO, it MUST NOT advertise VIRTIO_NET_F_TSO_LIMIT feature.
"""

>
> > >
> > >
> > >
> > > > >
> > > > > > >
> > > > > > > I think this kind of things should be explicit.
> > > > > > >
> > > > > > >
> > > > > > > > +
> > > > > > > > +The following field, \field{tso_max_segs} only exists if
> > > > > > > > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > > > > > > > +It specifies the maximum number of segments that can be produced by
> > > > > > > > +the device after performing segmentation on TSO packet or a UDP tunnel
> > > > > > > > +TSO packet (when VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set).
> > > > > > >
> > > > > > > I don't get this field at all. the assumption is that all segments
> > > > > > > are the same size, right? Then it is just based on length?
> > > > > >
> > > > > > It's the device side limitation, for example a device can produce 100
> > > > > > segments at most, even if the tso_max_size is 256K, when MTU is 1500,
> > > > > > the driver can't send a TSO packet whose TCP length is greater than
> > > > > > (1500 - 20 - 20) * 100 = 146K.
> > > > >
> > > > > then "can be produced" is again confusing. device
> > > > > transmits packets but it does not "produce" them as such.
> > > > > maybe you just mean "supported".
> > > >
> > > > I think yes.
> > > >
> > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > +
> > > > > > > >  \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
> > > > > > > >
> > > > > > > >  The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
> > > > > > > > @@ -326,6 +345,17 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > > > > >  The device SHOULD NOT offer VIRTIO_NET_F_CTRL_RX_EXTRA if it
> > > > > > > >  does not offer VIRTIO_NET_F_CTRL_VQ.
> > > > > > > >
> > > > > > > > +If VIRTIO_NET_F_HOST_TSO_LIMIT and VIRTIO_NET_F_MTU have been
> > > > > > > > +negotiated, the device SHOULD set \field{tso_max_size} so that a TCP
> > > > > > > > +segment that fully utilizes the configured MTU can be processed by TSO
> > > > > > > > +(e.g., for IPv4 without options: at least \field{mtu} - 20; for IPv6
> > > > > > > > +without extension headers: at least \field{mtu} - 40). This
> > > > > > > > +recommendation does not account for IPv4 options or IPv6 extension
> > > > > > > > +headers, which reduce the effective segment size.
> > > > > > > > +
> > > > > > > > +If VIRTIO_NET_F_HOST_TSO_LIMIT has been negotiated, the device MUST
> > > > > > > > +set \field{tso_max_segs} to at least 64.
> > > > > > >
> > > > > > > where does this 64 come from? pls document.
> > > > > >
> > > > > > A simple backward compatibility which makes sure the value can make
> > > > > > sure 64K TSO can be segmented with 1500 MTU.
> > > > >
> > > > > 2^16/64 == 1024
> > > > >
> > > > > not ~1500
> > > >
> > > > Yes, I just choose one that is sufficient.
> > > >
> > > > >
> > > > > And we don't know MTU is 1500, either.
> > > >
> > > > A typical configuration but I can remove this part.
> > > >
> > > > Thanks
> > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > MST
> > > > >
> > >
> >
> > Thanks
>

Thanks

Re: [PATCH] virtio-net: introduce TSO limit feature

[Michael S. Tsirkin]

On Mon, Oct 20, 2025 at 02:25:56PM +0800, Jason Wang wrote:
> On Thu, Oct 16, 2025 at 6:03 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Thu, Oct 16, 2025 at 02:31:46PM +0800, Jason Wang wrote:
> > > On Thu, Oct 16, 2025 at 2:17 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > >
> > > > On Thu, Oct 16, 2025 at 01:57:41PM +0800, Jason Wang wrote:
> > > > > On Wed, Oct 15, 2025 at 3:27 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > > > >
> > > > > > Thanks for the answers. Some more comments:
> > > > > >
> > > > > > On Wed, Oct 15, 2025 at 12:29:13PM +0800, Jason Wang wrote:
> > > > > > > > >  device-types/net/description.tex | 46 ++++++++++++++++++++++++++++++++
> > > > > > > > >  1 file changed, 46 insertions(+)
> > > > > > > > >
> > > > > > > > > diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> > > > > > > > > index 415c7fd..e56df75 100644
> > > > > > > > > --- a/device-types/net/description.tex
> > > > > > > > > +++ b/device-types/net/description.tex
> > > > > > > > > @@ -146,6 +146,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > > > > > > > >   when VIRTIO_NET_F_IPSEC is negotiated. When a device offers IPsec feature, it SHOULD
> > > > > > > > >   also offer the VIRTIO_NET_F_OUT_NET_HEADER feature.
> > > > > > > > >
> > > > > > > > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT(71)] Device limits the maximum TCP
> > > > > > > > > +  length and the number of segments when performing TCP segmentation.
> > > > > > > > > +
> > > > > > > > >  \end{description}
> > > > > > > > >
> > > > > > > > >  \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
> > > > > > > > > @@ -184,6 +187,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> > > > > > > > >  \item[VIRTIO_NET_F_VQ_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> > > > > > > > >  \item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ along with VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> > > > > > > > >  \item[VIRTIO_NET_F_RSS_CONTEXT] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> > > > > > > > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6
> > > > > > > > >  \end{description}
> > > > > > > > >
> > > > > > > > >  \begin{note}
> > > > > > > > > @@ -220,6 +224,8 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > > > > > >          le16 rss_max_indirection_table_length;
> > > > > > > > >          le32 supported_hash_types;
> > > > > > > > >          le32 supported_tunnel_types;
> > > > > > > > > +        le32 tso_max_size;
> > > > > > > > > +        le32 tso_max_segs;
> > > > > > > > >  };
> > > > > > > > >  \end{lstlisting}
> > > > > > > > >
> > > > > > > > > @@ -276,6 +282,19 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > > > > > >  Encapsulation types are defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets /
> > > > > > > > >  Hash calculation for incoming packets / Encapsulation types supported/enabled for inner header hash}.
> > > > > > > > >
> > > > > > > > > +The following field, \field{tso_max_size} only exists if
> > > > > > > > > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > > > > > > > > +It specifies the maximum TCP length
> > > > > > > >
> > > > > > > > what is TCP length?
> > > > > > >
> > > > > > > It's defined in the rfc793:
> > > > > > >
> > > > > > > """
> > > > > > >       The TCP Length is the TCP header length plus the data length in
> > > > > > >       octets (this is not an explicitly transmitted quantity, but is
> > > > > > >       computed), and it does not count the 12 octets of the pseudo
> > > > > > >       header.
> > > > > > > """
> > > > > >
> > > > > >
> > > > > > But that one is 16 bit so can not exceed 65535.
> > > > >
> > > > > I just reuse the terminology instead of defining something new.
> > > >
> > > > Let's use a generic term that will work with big tcp.
> > >
> > > After glancing at RFC TCP length seems to be the best as TCP header
> > > doesn't have a length.
> > >
> > > Maybe you have something better in mind?
> >
> >
> > Depends on what do you mean by it, exactly.
> 
> TCP header length plus the data length. Maybe TCP packet length?

Hmm. It's not even a packet length is it?
Linux has this:
	Device (as in HW) limit on the max TSO request size

but switching to "request" is more work.

for now, we have:




\item If the driver negotiated
  VIRTIO_NET_F_HOST_TSO4, TSO6, USO or UFO, and the packet requires
  TCP segmentation, UDP segmentation or fragmentation, then \field{gso_type}
  is set to VIRTIO_NET_HDR_GSO_TCPV4, TCPV6, UDP_L4 or UDP.
  (Otherwise, it is set to VIRTIO_NET_HDR_GSO_NONE). In this
  case, packets larger than 1514 bytes can be transmitted: the
  metadata indicates how to replicate the packet header to cut it
  into smaller packets. The other gso fields are set:


So if we just go with "length of a packet that requires TCP
segmentation" then I think it's at least not making things
any worse.


> >
> > > >
> > > > > Note
> > > > > that it is only used in pseudo header for csum after device has
> > > > > performed TSO. The value in the pseudo header is capped by MTU/MSS.
> > > > >
> > > > > As replied in another thread, BIG TCP requires more work or features.
> > > > > Driver needs to set ip->tot_len 0 with a new gso type to let the
> > > > > device know about BIG TCP packet.
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > > > of a TSO packet
> > > > > > > >
> > > > > > > > what is a TSO packet?
> > > > > > >
> > > > > > > Packet for device to perform TCP segmentation offload.
> > > > > >
> > > > > > pls define terms before use.
> > > > >
> > > > > I may miss something but TSO has been widely used in the spec before
> > > > > this feature:
> > > > >
> > > > > """
> > > > > \item[VIRTIO_NET_F_GUEST_ECN (9)] Driver can receive TSO with ECN.
> > > > > ...
> > > > > \item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can receive TSOv4.
> > > > > ...
> > > > > """
> > > >
> > > >
> > > > Yes but that does not define a "TSO packet".
> > >
> > > How about a patch beforehand to explain thing like:
> > >
> > > \item[VIRTIO_NET_F_GUEST_ECN (9)] Driver can receive TSO (TCP
> > > Segmentation Offload) with ECN.
> >
> > That does not define is either?
> >
> > virtio$ git grep 'TSO packet'
> > virtio$
> 
> Something like this?
> 
> Maybe "TSO packet is the packet that device may segment it into
> multiple frames"?


The current text is a bit of a mess where "packet" interchangeably means
the result of segmentation or the request to segment.
We should fix that but for your patch I think it is reasonable
to just spell it out at each instance: "packet that requires TCP
segmentation".


> 
> >
> >
> >
> >
> >
> > > >
> > > >
> > > > > >
> > > > > > > > > that the
> > > > > > > > > +device can process.
> > > > > > > >
> > > > > > > > process in which direction? you mean device can receive?
> > > > > > >
> > > > > > > It works only for TX (as TSO works only for TX).
> > > > > >
> > > > > >
> > > > > > rest of spec says "device receives from driver"  for this.
> > > > > > process is ambiguous
> > > > >
> > > > > A quick grep doesn't show me things like this, maybe you can point out
> > > > > the location. Not a native speaker, but using "device receives from
> > > > > driver" is indeed ambiguous for TX.
> > > >
> > > > Well right near the text you quoted:
> > > >
> > > > \item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can receive TSOv4.
> > > >
> > > > \item[VIRTIO_NET_F_HOST_TSO6 (12)] Device can receive TSOv6.
> > >
> > > Okay, I think you want another to to replace it with
> > >
> > > \item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can transmit TSOv4.
> > >
> > > ?
> >
> >
> > "receive" in this context is "receive from driver".
> >
> >
> > TSO is not something that is transmitted on the wire.
> 
> Ok.
> 
> >
> >
> >
> >
> > > >
> > > >
> > > >
> > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > > >
> > > > > > > > > When VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set,
> > > > > > > > > +it specifies the maximum inner TCP length of a UDP tunnel TSO packet
> > > > > > > > > +that the device can process.
> > > > > > > >
> > > > > > > > Rest of spec talks of " GSO over UDP tunnels packets" is this the same?
> > > > > > >
> > > > > > > Not exactly the same, this is only for TSO not genreal GSO.
> > > > > >
> > > > > > rest of spec mostly talks of GSO. in fact virtio tso is a kind of
> > > > > > accelerated gso.
> > > > >
> > > > > This only applies for some specific software datapath like vhost-net.
> > > > > But it doesn't apply to others especially the hardware device who will
> > > > > do real TSO.
> > > >
> > > > My point is that once you have said both GSO and TSO in the same
> > > > sentence, any reader's eyes have glazed over.
> > > >
> > > >
> > > >
> > > >
> > > > > > either do the same or add a lot of text
> > > > > > explaining tso as opposed to gso.
> > > > >
> > > > > Is this ok to say "UDP tunnel VIRTIO_NET_HDR_GSO_TCPV4 or
> > > > > VIRTIO_NET_HDR_GSO_TCPV6 packet"?
> > > >
> > > > Do you maybe mean: \field{gso_type} set to VIRTIO_NET_HDR_GSO_TCPV4
> > > > or VIRTIO_NET_HDR_GSO_TCPV6
> > >
> > > Yes.
> > >
> > > >
> > > >
> > > >
> > > > > >
> > > > > > > >
> > > > > > > > even if it's actually unused?
> > > > > > > >
> > > > > > > > this, on the assumption that the length for tunnel is smaller?
> > > > > > >
> > > > > > > It means the device should have the same limitation for plain TSO and
> > > > > > > tunnel TSO.
> > > > > >
> > > > > > Hmm. I have doubts how it can work given the overhead.
> > > > >
> > > > > If a device can't afford the same limitation, it can simply not
> > > > > advertise this feature. The reason I don't introduce a dedicated
> > > > > limitation for tunnel is that there could be more tunnel supported in
> > > > > the future, it would be a burden to have a per tunnel type limitation.
> > > >
> > > > Well presumably the feature is needed no?
> > >
> > > Yes, but the kernel assumes the same limitation for tunnel and non
> > > tunnel one. Having a single limitation simply things or if you stick I
> > > can introduce a dedicated one for UDP tunnels.
> >
> > I don't insist, I think it is worth clarifying though.
> 
> Ok how about this:
> 
> """
> If a device can not have a single limitation for both plain GSO and
> tunnel GSO, it MUST NOT advertise VIRTIO_NET_F_TSO_LIMIT feature.
> """
> 
> >
> > > >
> > > >
> > > >
> > > > > >
> > > > > > > >
> > > > > > > > I think this kind of things should be explicit.
> > > > > > > >
> > > > > > > >
> > > > > > > > > +
> > > > > > > > > +The following field, \field{tso_max_segs} only exists if
> > > > > > > > > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > > > > > > > > +It specifies the maximum number of segments that can be produced by
> > > > > > > > > +the device after performing segmentation on TSO packet or a UDP tunnel
> > > > > > > > > +TSO packet (when VIRTIO_NET_F_HOST_UDP_TUNNEL_GSO is set).
> > > > > > > >
> > > > > > > > I don't get this field at all. the assumption is that all segments
> > > > > > > > are the same size, right? Then it is just based on length?
> > > > > > >
> > > > > > > It's the device side limitation, for example a device can produce 100
> > > > > > > segments at most, even if the tso_max_size is 256K, when MTU is 1500,
> > > > > > > the driver can't send a TSO packet whose TCP length is greater than
> > > > > > > (1500 - 20 - 20) * 100 = 146K.
> > > > > >
> > > > > > then "can be produced" is again confusing. device
> > > > > > transmits packets but it does not "produce" them as such.
> > > > > > maybe you just mean "supported".
> > > > >
> > > > > I think yes.
> > > > >
> > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > +
> > > > > > > > >  \devicenormative{\subsubsection}{Device configuration layout}{Device Types / Network Device / Device configuration layout}
> > > > > > > > >
> > > > > > > > >  The device MUST set \field{max_virtqueue_pairs} to between 1 and 0x8000 inclusive,
> > > > > > > > > @@ -326,6 +345,17 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > > > > > >  The device SHOULD NOT offer VIRTIO_NET_F_CTRL_RX_EXTRA if it
> > > > > > > > >  does not offer VIRTIO_NET_F_CTRL_VQ.
> > > > > > > > >
> > > > > > > > > +If VIRTIO_NET_F_HOST_TSO_LIMIT and VIRTIO_NET_F_MTU have been
> > > > > > > > > +negotiated, the device SHOULD set \field{tso_max_size} so that a TCP
> > > > > > > > > +segment that fully utilizes the configured MTU can be processed by TSO
> > > > > > > > > +(e.g., for IPv4 without options: at least \field{mtu} - 20; for IPv6
> > > > > > > > > +without extension headers: at least \field{mtu} - 40). This
> > > > > > > > > +recommendation does not account for IPv4 options or IPv6 extension
> > > > > > > > > +headers, which reduce the effective segment size.
> > > > > > > > > +
> > > > > > > > > +If VIRTIO_NET_F_HOST_TSO_LIMIT has been negotiated, the device MUST
> > > > > > > > > +set \field{tso_max_segs} to at least 64.
> > > > > > > >
> > > > > > > > where does this 64 come from? pls document.
> > > > > > >
> > > > > > > A simple backward compatibility which makes sure the value can make
> > > > > > > sure 64K TSO can be segmented with 1500 MTU.
> > > > > >
> > > > > > 2^16/64 == 1024
> > > > > >
> > > > > > not ~1500
> > > > >
> > > > > Yes, I just choose one that is sufficient.
> > > > >
> > > > > >
> > > > > > And we don't know MTU is 1500, either.
> > > > >
> > > > > A typical configuration but I can remove this part.
> > > > >
> > > > > Thanks
> > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > MST
> > > > > >
> > > >
> > >
> > > Thanks
> >
> 
> Thanks

Re: [PATCH] virtio-net: introduce TSO limit feature

[Jason Wang]

On Mon, Oct 20, 2025 at 4:19 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Mon, Oct 20, 2025 at 02:25:56PM +0800, Jason Wang wrote:
> > On Thu, Oct 16, 2025 at 6:03 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > On Thu, Oct 16, 2025 at 02:31:46PM +0800, Jason Wang wrote:
> > > > On Thu, Oct 16, 2025 at 2:17 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > > >
> > > > > On Thu, Oct 16, 2025 at 01:57:41PM +0800, Jason Wang wrote:
> > > > > > On Wed, Oct 15, 2025 at 3:27 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > > > > >
> > > > > > > Thanks for the answers. Some more comments:
> > > > > > >
> > > > > > > On Wed, Oct 15, 2025 at 12:29:13PM +0800, Jason Wang wrote:
> > > > > > > > > >  device-types/net/description.tex | 46 ++++++++++++++++++++++++++++++++
> > > > > > > > > >  1 file changed, 46 insertions(+)
> > > > > > > > > >
> > > > > > > > > > diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> > > > > > > > > > index 415c7fd..e56df75 100644
> > > > > > > > > > --- a/device-types/net/description.tex
> > > > > > > > > > +++ b/device-types/net/description.tex
> > > > > > > > > > @@ -146,6 +146,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> > > > > > > > > >   when VIRTIO_NET_F_IPSEC is negotiated. When a device offers IPsec feature, it SHOULD
> > > > > > > > > >   also offer the VIRTIO_NET_F_OUT_NET_HEADER feature.
> > > > > > > > > >
> > > > > > > > > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT(71)] Device limits the maximum TCP
> > > > > > > > > > +  length and the number of segments when performing TCP segmentation.
> > > > > > > > > > +
> > > > > > > > > >  \end{description}
> > > > > > > > > >
> > > > > > > > > >  \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
> > > > > > > > > > @@ -184,6 +187,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> > > > > > > > > >  \item[VIRTIO_NET_F_VQ_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
> > > > > > > > > >  \item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ along with VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
> > > > > > > > > >  \item[VIRTIO_NET_F_RSS_CONTEXT] Requires VIRTIO_NET_F_CTRL_VQ and VIRTIO_NET_F_RSS.
> > > > > > > > > > +\item[VIRTIO_NET_F_HOST_TSO_LIMIT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6
> > > > > > > > > >  \end{description}
> > > > > > > > > >
> > > > > > > > > >  \begin{note}
> > > > > > > > > > @@ -220,6 +224,8 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > > > > > > >          le16 rss_max_indirection_table_length;
> > > > > > > > > >          le32 supported_hash_types;
> > > > > > > > > >          le32 supported_tunnel_types;
> > > > > > > > > > +        le32 tso_max_size;
> > > > > > > > > > +        le32 tso_max_segs;
> > > > > > > > > >  };
> > > > > > > > > >  \end{lstlisting}
> > > > > > > > > >
> > > > > > > > > > @@ -276,6 +282,19 @@ \subsection{Device configuration layout}\label{sec:Device Types / Network Device
> > > > > > > > > >  Encapsulation types are defined in \ref{sec:Device Types / Network Device / Device Operation / Processing of Incoming Packets /
> > > > > > > > > >  Hash calculation for incoming packets / Encapsulation types supported/enabled for inner header hash}.
> > > > > > > > > >
> > > > > > > > > > +The following field, \field{tso_max_size} only exists if
> > > > > > > > > > +VIRTIO_NET_F_HOST_TSO_LIMIT is set.
> > > > > > > > > > +It specifies the maximum TCP length
> > > > > > > > >
> > > > > > > > > what is TCP length?
> > > > > > > >
> > > > > > > > It's defined in the rfc793:
> > > > > > > >
> > > > > > > > """
> > > > > > > >       The TCP Length is the TCP header length plus the data length in
> > > > > > > >       octets (this is not an explicitly transmitted quantity, but is
> > > > > > > >       computed), and it does not count the 12 octets of the pseudo
> > > > > > > >       header.
> > > > > > > > """
> > > > > > >
> > > > > > >
> > > > > > > But that one is 16 bit so can not exceed 65535.
> > > > > >
> > > > > > I just reuse the terminology instead of defining something new.
> > > > >
> > > > > Let's use a generic term that will work with big tcp.
> > > >
> > > > After glancing at RFC TCP length seems to be the best as TCP header
> > > > doesn't have a length.
> > > >
> > > > Maybe you have something better in mind?
> > >
> > >
> > > Depends on what do you mean by it, exactly.
> >
> > TCP header length plus the data length. Maybe TCP packet length?
>
> Hmm. It's not even a packet length is it?
> Linux has this:
>         Device (as in HW) limit on the max TSO request size
>
> but switching to "request" is more work.
>
> for now, we have:
>
>
>
>
> \item If the driver negotiated
>   VIRTIO_NET_F_HOST_TSO4, TSO6, USO or UFO, and the packet requires
>   TCP segmentation, UDP segmentation or fragmentation, then \field{gso_type}
>   is set to VIRTIO_NET_HDR_GSO_TCPV4, TCPV6, UDP_L4 or UDP.
>   (Otherwise, it is set to VIRTIO_NET_HDR_GSO_NONE). In this
>   case, packets larger than 1514 bytes can be transmitted: the
>   metadata indicates how to replicate the packet header to cut it
>   into smaller packets. The other gso fields are set:
>
>
> So if we just go with "length of a packet that requires TCP
> segmentation" then I think it's at least not making things
> any worse.

Ok, I will do it.

>
>
> > >
> > > > >
> > > > > > Note
> > > > > > that it is only used in pseudo header for csum after device has
> > > > > > performed TSO. The value in the pseudo header is capped by MTU/MSS.
> > > > > >
> > > > > > As replied in another thread, BIG TCP requires more work or features.
> > > > > > Driver needs to set ip->tot_len 0 with a new gso type to let the
> > > > > > device know about BIG TCP packet.
> > > > > >
> > > > > > >
> > > > > > > > >
> > > > > > > > > > of a TSO packet
> > > > > > > > >
> > > > > > > > > what is a TSO packet?
> > > > > > > >
> > > > > > > > Packet for device to perform TCP segmentation offload.
> > > > > > >
> > > > > > > pls define terms before use.
> > > > > >
> > > > > > I may miss something but TSO has been widely used in the spec before
> > > > > > this feature:
> > > > > >
> > > > > > """
> > > > > > \item[VIRTIO_NET_F_GUEST_ECN (9)] Driver can receive TSO with ECN.
> > > > > > ...
> > > > > > \item[VIRTIO_NET_F_HOST_TSO4 (11)] Device can receive TSOv4.
> > > > > > ...
> > > > > > """
> > > > >
> > > > >
> > > > > Yes but that does not define a "TSO packet".
> > > >
> > > > How about a patch beforehand to explain thing like:
> > > >
> > > > \item[VIRTIO_NET_F_GUEST_ECN (9)] Driver can receive TSO (TCP
> > > > Segmentation Offload) with ECN.
> > >
> > > That does not define is either?
> > >
> > > virtio$ git grep 'TSO packet'
> > > virtio$
> >
> > Something like this?
> >
> > Maybe "TSO packet is the packet that device may segment it into
> > multiple frames"?
>
>
> The current text is a bit of a mess where "packet" interchangeably means
> the result of segmentation or the request to segment.
> We should fix that but for your patch I think it is reasonable
> to just spell it out at each instance: "packet that requires TCP
> segmentation".

Fine.

Thanks