[virtio-comment] [PATCH] virtio-net: define support for receive-side scaling

[virtio-comment] [PATCH] virtio-net: define support for receive-side scaling

[Yuri Benditovich]

Added support for RSS receive steering mode.

Signed-off-by: Yuri Benditovich <yuri.benditovich@daynix.com>
---
 conformance.tex |   2 +
 content.tex     | 170 ++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 172 insertions(+)

diff --git a/conformance.tex b/conformance.tex
index 0ac58aa..bd31ba3 100644
--- a/conformance.tex
+++ b/conformance.tex
@@ -101,6 +101,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
 \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Gratuitous Packet Sending}
 \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode}
 \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Offloads State Configuration / Setting Offloads State}
+\item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Using RSS commands}
 \end{itemize}
 
 \conformance{\subsection}{Block Driver Conformance}\label{sec:Conformance / Driver Conformance / Block Driver Conformance}
@@ -257,6 +258,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
 \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Setting MAC Address Filtering}
 \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Gratuitous Packet Sending}
 \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode}
+\item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
 \end{itemize}
 
 \conformance{\subsection}{Block Device Conformance}\label{sec:Conformance / Device Conformance / Block Device Conformance}
diff --git a/content.tex b/content.tex
index 679391e..5672299 100644
--- a/content.tex
+++ b/content.tex
@@ -2811,6 +2811,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
+\item[VIRTIO_NET_F_RSS(60)] Device supports RSS (receive-side scaling)
+    with Toeplitz hash calculation and configurable hash parameters for receive steering
+
 \item[VIRTIO_NET_F_RSC_EXT(61)] Device can process duplicated ACKs
     and report number of coalesced segments and duplicated ACKs
 
@@ -2840,6 +2843,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_MQ] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
+\item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_MQ
 \end{description}
 
 \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -3785,6 +3789,172 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 according to the native endian of the guest rather than
 (necessarily when not using the legacy interface) little-endian.
 
+\paragraph{Receive-side scaling (RSS)}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}
+The device indicates presence of this feature if in addition to automatic receive steering it supports RSS steering with Toeplitz hash calculation and configurable parameters.
+
+Driver uses following value as \field{class} for RSS-related commands.
+\begin{lstlisting}
+#define VIRTIO_NET_CTRL_RSS              6
+\end{lstlisting}
+Driver uses following values as \field{command} for RSS-related commands.
+\begin{lstlisting}
+#define VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES              0
+#define VIRTIO_NET_CTRL_RSS_SET_PARAMETERS                  1
+\end{lstlisting}
+
+\subparagraph{Querying RSS capabilities}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Querying RSS capabilities}
+
+Driver sends VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES command to query device capabilities related to RSS.
+Device responds the command using following format for \field{command-specific-data}:
+\begin{lstlisting}
+struct virtio_net_rss_capabilities {
+    le16 hash_types_v4;  (bitmask of supported hash types for IPv4 packets)
+    le16 hash_types_v6;  (bitmask of supported hash types for IPv6 packets)
+    le16 max_indirection_table_len; (maximal number of queue indices in indirection table)
+    u8 max_key_length; (maximal length of RSS key in bytes)
+};
+\end{lstlisting}
+To describe supported/enabled hash types device and driver use following definitions:
+
+Hash types applicable for IPv4 packets and for IPv6 packets without extension headers
+\begin{lstlisting}
+#define VIRTIO_NET_RSS_HASH_TYPE_IP              (1 << 0)
+#define VIRTIO_NET_RSS_HASH_TYPE_TCP             (1 << 1)
+#define VIRTIO_NET_RSS_HASH_TYPE_UDP             (1 << 2)
+\end{lstlisting}
+Hash types applicable for IPv6 packets with extension headers
+\begin{lstlisting}
+#define VIRTIO_NET_RSS_HASH_TYPE_IP_EX           (1 << 4) (only for IPv6)
+#define VIRTIO_NET_RSS_HASH_TYPE_TCP_EX          (1 << 5) (only for IPv6)
+#define VIRTIO_NET_RSS_HASH_TYPE_UDP_EX          (1 << 6) (only for IPv6)
+\end{lstlisting}
+For exact meaning of VIRTIO_NET_RSS_HASH_TYPE_ flags see \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
+
+\subparagraph{Setting RSS parameters}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}
+
+Driver sends VIRTIO_NET_CTRL_RSS_SET_PARAMETERS command using following format for \field{command-specific-data}:
+\begin{lstlisting}
+struct virtio_net_rss_params {
+    u8 enable_rss;
+    u8 hash_key_length;
+    le16 hash_types_v4;  (bitmask of allowed hash types for IPv4 packets)
+    le16 hash_types_v6;  (bitmask of allowed hash types for IPv6 packets)
+    le16 indirection_table_length; (number of queue indices in indirection_table array)
+    le16 unclassified_queue; (queue to place unclassified packets in)
+    le16 indirection_table[indirection_table_length];
+    u8 hash_key_data[hash_key_length];
+};
+
+\end{lstlisting}
+Driver sets \field{enable_rss} to 0 to disable RSS processing by the device and use automatic steering mode instead.
+
+Driver sets \field{enable_rss} to 1 to enable RSS processing by the device using provided parameters.
+
+\subparagraph{RSS hash types}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
+
+The device calculates hash on IPv4 packets according to the field \field{hash_types_v4} of virtio_net_rss_params structure as follows:
+\begin{itemize}
+\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCP header, the hash is calculated over following fields:
+\begin{itemize}
+\item Source IP address
+\item Destination IP address
+\item Source TCP port
+\item Destination TCP port
+\end{itemize}
+\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDP header, the hash is calculated over following fields:
+\begin{itemize}
+\item Source IP address
+\item Destination IP address
+\item Source UDP port
+\item Destination UDP port
+\end{itemize}
+\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
+\begin{itemize}
+\item Source IP address
+\item Destination IP address
+\end{itemize}
+\item Else the device does not calculate the hash
+\end{itemize}
+
+
+The device calculates hash on IPv6 packets without extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
+\begin{itemize}
+\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCPv6 header, the hash is calculated over following fields:
+\begin{itemize}
+\item Source IPv6 address
+\item Destination IPv6 address
+\item Source TCP port
+\item Destination TCP port
+\end{itemize}
+\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDPv6 header, the hash is calculated over following fields:
+\begin{itemize}
+\item Source IPv6 address
+\item Destination IPv6 address
+\item Source UDP port
+\item Destination UDP port
+\end{itemize}
+\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
+\begin{itemize}
+\item Source IPv6 address
+\item Destination IPv6 address
+\end{itemize}
+\item Else the device does not calculate the hash
+\end{itemize}
+
+
+The device calculates hash on IPv6 packets with extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
+\begin{itemize}
+\item If VIRTIO_NET_RSS_HASH_TYPE_TCP_EX is set and the packet has TCPv6 header, the hash is calculated over following fields:
+\begin{itemize}
+\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
+\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
+\item Source TCP port
+\item Destination TCP port
+\end{itemize}
+\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP_EX is set and the packet has UDPv6 header, the hash is calculated over following fields:
+\begin{itemize}
+\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
+\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
+\item Source UDP port
+\item Destination UDP port
+\end{itemize}
+\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP_EX is set, the hash is calculated over following fields:
+\begin{itemize}
+\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
+\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
+\end{itemize}
+\item Else skip IPv6 extension headers and calculate the hash as defined above for IPv6 packet without extension headers
+\end{itemize}
+
+
+\drivernormative{\subparagraph}{Using RSS commands}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Using RSS commands}
+
+A driver MUST NOT use RSS commands if the feature VIRTIO_NET_F_RSS has not been negotiated.
+
+A driver MUST NOT set RSS parameters before it successfully enabled operation with multiple queues.
+
+A driver MUST fill \field{indirection_table} array only with indices of enabled queues.
+
+An \field{indirection_table_length} MUST be power of two.
+
+A driver MUST NOT set any VIRTIO_NET_RSS_HASH_TYPE_ flags that are not supported by device.
+
+A driver MUST NOT set any bits below for \field{hash_types_v4}:
+\begin{itemize}
+\item VIRTIO_NET_RSS_HASH_TYPE_IP_EX
+\item VIRTIO_NET_RSS_HASH_TYPE_TCP_EX
+\item VIRTIO_NET_RSS_HASH_TYPE_UDP_EX
+\end{itemize}
+
+\devicenormative{\subparagraph}{RSS processing}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
+If the device reports support for VIRTIO_NET_F_RSS it MUST support keys of at least 40 bytes and indirection table of at least 128 entries.
+
+The device MUST determine destination queue for network packet as follows:
+\begin{itemize}
+\item Calculate hash of the packet as defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
+\item If the device did not calculate the hash for specific packet, the device directs the packet to the queue specified by \field{unclassified_queue} of virtio_net_rss_params structure
+\item Apply mask of (indirection_table_length - 1) to the calculated hash and use the result as the index in the indirection table to get 0-based number of destination receive queue
+\end{itemize}
 
 \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
 Types / Network Device / Legacy Interface: Framing Requirements}
-- 
2.17.2


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] Re: [PATCH] virtio-net: define support for receive-side scaling

[Michael S. Tsirkin]

On Wed, Oct 02, 2019 at 07:41:39PM +0300, Yuri Benditovich wrote:
> Added support for RSS receive steering mode.
> 
> Signed-off-by: Yuri Benditovich <yuri.benditovich@daynix.com>

could you pls list the changes from the previous versions?

> ---
>  conformance.tex |   2 +
>  content.tex     | 170 ++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 172 insertions(+)
> 
> diff --git a/conformance.tex b/conformance.tex
> index 0ac58aa..bd31ba3 100644
> --- a/conformance.tex
> +++ b/conformance.tex
> @@ -101,6 +101,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
>  \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Gratuitous Packet Sending}
>  \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode}
>  \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Offloads State Configuration / Setting Offloads State}
> +\item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Using RSS commands}
>  \end{itemize}
>  
>  \conformance{\subsection}{Block Driver Conformance}\label{sec:Conformance / Driver Conformance / Block Driver Conformance}
> @@ -257,6 +258,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
>  \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Setting MAC Address Filtering}
>  \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Gratuitous Packet Sending}
>  \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode}
> +\item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
>  \end{itemize}
>  
>  \conformance{\subsection}{Block Device Conformance}\label{sec:Conformance / Device Conformance / Block Device Conformance}
> diff --git a/content.tex b/content.tex
> index 679391e..5672299 100644
> --- a/content.tex
> +++ b/content.tex
> @@ -2811,6 +2811,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
>  \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
>      channel.
>  
> +\item[VIRTIO_NET_F_RSS(60)] Device supports RSS (receive-side scaling)
> +    with Toeplitz hash calculation and configurable hash parameters for receive steering
> +
>  \item[VIRTIO_NET_F_RSC_EXT(61)] Device can process duplicated ACKs
>      and report number of coalesced segments and duplicated ACKs
>  
> @@ -2840,6 +2843,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
>  \item[VIRTIO_NET_F_MQ] Requires VIRTIO_NET_F_CTRL_VQ.
>  \item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ.
>  \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> +\item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_MQ
>  \end{description}
>  
>  \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> @@ -3785,6 +3789,172 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>  according to the native endian of the guest rather than
>  (necessarily when not using the legacy interface) little-endian.
>  
> +\paragraph{Receive-side scaling (RSS)}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}
> +The device indicates presence of this feature if in addition to automatic receive steering it supports RSS steering with Toeplitz hash calculation and configurable parameters.
> +
> +Driver uses following value as \field{class} for RSS-related commands.
> +\begin{lstlisting}
> +#define VIRTIO_NET_CTRL_RSS              6
> +\end{lstlisting}
> +Driver uses following values as \field{command} for RSS-related commands.
> +\begin{lstlisting}
> +#define VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES              0
> +#define VIRTIO_NET_CTRL_RSS_SET_PARAMETERS                  1
> +\end{lstlisting}
> +
> +\subparagraph{Querying RSS capabilities}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Querying RSS capabilities}
> +
> +Driver sends VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES command to query device capabilities related to RSS.
> +Device responds the command using following format for \field{command-specific-data}:
> +\begin{lstlisting}
> +struct virtio_net_rss_capabilities {
> +    le16 hash_types_v4;  (bitmask of supported hash types for IPv4 packets)
> +    le16 hash_types_v6;  (bitmask of supported hash types for IPv6 packets)
> +    le16 max_indirection_table_len; (maximal number of queue indices in indirection table)
> +    u8 max_key_length; (maximal length of RSS key in bytes)
> +};
> +\end{lstlisting}
> +To describe supported/enabled hash types device and driver use following definitions:
> +
> +Hash types applicable for IPv4 packets and for IPv6 packets without extension headers
> +\begin{lstlisting}
> +#define VIRTIO_NET_RSS_HASH_TYPE_IP              (1 << 0)
> +#define VIRTIO_NET_RSS_HASH_TYPE_TCP             (1 << 1)
> +#define VIRTIO_NET_RSS_HASH_TYPE_UDP             (1 << 2)
> +\end{lstlisting}
> +Hash types applicable for IPv6 packets with extension headers
> +\begin{lstlisting}
> +#define VIRTIO_NET_RSS_HASH_TYPE_IP_EX           (1 << 4) (only for IPv6)
> +#define VIRTIO_NET_RSS_HASH_TYPE_TCP_EX          (1 << 5) (only for IPv6)
> +#define VIRTIO_NET_RSS_HASH_TYPE_UDP_EX          (1 << 6) (only for IPv6)
> +\end{lstlisting}
> +For exact meaning of VIRTIO_NET_RSS_HASH_TYPE_ flags see \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
> +
> +\subparagraph{Setting RSS parameters}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}
> +
> +Driver sends VIRTIO_NET_CTRL_RSS_SET_PARAMETERS command using following format for \field{command-specific-data}:
> +\begin{lstlisting}
> +struct virtio_net_rss_params {
> +    u8 enable_rss;
> +    u8 hash_key_length;
> +    le16 hash_types_v4;  (bitmask of allowed hash types for IPv4 packets)
> +    le16 hash_types_v6;  (bitmask of allowed hash types for IPv6 packets)
> +    le16 indirection_table_length; (number of queue indices in indirection_table array)
> +    le16 unclassified_queue; (queue to place unclassified packets in)
> +    le16 indirection_table[indirection_table_length];
> +    u8 hash_key_data[hash_key_length];
> +};
> +
> +\end{lstlisting}
> +Driver sets \field{enable_rss} to 0 to disable RSS processing by the device and use automatic steering mode instead.
> +
> +Driver sets \field{enable_rss} to 1 to enable RSS processing by the device using provided parameters.
> +
> +\subparagraph{RSS hash types}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
> +
> +The device calculates hash on IPv4 packets according to the field \field{hash_types_v4} of virtio_net_rss_params structure as follows:
> +\begin{itemize}
> +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCP header, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Source IP address
> +\item Destination IP address
> +\item Source TCP port
> +\item Destination TCP port
> +\end{itemize}
> +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDP header, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Source IP address
> +\item Destination IP address
> +\item Source UDP port
> +\item Destination UDP port
> +\end{itemize}
> +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Source IP address
> +\item Destination IP address
> +\end{itemize}
> +\item Else the device does not calculate the hash
> +\end{itemize}
> +
> +
> +The device calculates hash on IPv6 packets without extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
> +\begin{itemize}
> +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCPv6 header, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Source IPv6 address
> +\item Destination IPv6 address
> +\item Source TCP port
> +\item Destination TCP port
> +\end{itemize}
> +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDPv6 header, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Source IPv6 address
> +\item Destination IPv6 address
> +\item Source UDP port
> +\item Destination UDP port
> +\end{itemize}
> +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Source IPv6 address
> +\item Destination IPv6 address
> +\end{itemize}
> +\item Else the device does not calculate the hash
> +\end{itemize}
> +
> +
> +The device calculates hash on IPv6 packets with extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
> +\begin{itemize}
> +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP_EX is set and the packet has TCPv6 header, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> +\item Source TCP port
> +\item Destination TCP port
> +\end{itemize}
> +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP_EX is set and the packet has UDPv6 header, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> +\item Source UDP port
> +\item Destination UDP port
> +\end{itemize}
> +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP_EX is set, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> +\end{itemize}
> +\item Else skip IPv6 extension headers and calculate the hash as defined above for IPv6 packet without extension headers
> +\end{itemize}
> +
> +
> +\drivernormative{\subparagraph}{Using RSS commands}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Using RSS commands}
> +
> +A driver MUST NOT use RSS commands if the feature VIRTIO_NET_F_RSS has not been negotiated.
> +
> +A driver MUST NOT set RSS parameters before it successfully enabled operation with multiple queues.
> +
> +A driver MUST fill \field{indirection_table} array only with indices of enabled queues.
> +
> +An \field{indirection_table_length} MUST be power of two.
> +
> +A driver MUST NOT set any VIRTIO_NET_RSS_HASH_TYPE_ flags that are not supported by device.
> +
> +A driver MUST NOT set any bits below for \field{hash_types_v4}:
> +\begin{itemize}
> +\item VIRTIO_NET_RSS_HASH_TYPE_IP_EX
> +\item VIRTIO_NET_RSS_HASH_TYPE_TCP_EX
> +\item VIRTIO_NET_RSS_HASH_TYPE_UDP_EX
> +\end{itemize}
> +
> +\devicenormative{\subparagraph}{RSS processing}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
> +If the device reports support for VIRTIO_NET_F_RSS it MUST support keys of at least 40 bytes and indirection table of at least 128 entries.
> +
> +The device MUST determine destination queue for network packet as follows:
> +\begin{itemize}
> +\item Calculate hash of the packet as defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
> +\item If the device did not calculate the hash for specific packet, the device directs the packet to the queue specified by \field{unclassified_queue} of virtio_net_rss_params structure
> +\item Apply mask of (indirection_table_length - 1) to the calculated hash and use the result as the index in the indirection table to get 0-based number of destination receive queue
> +\end{itemize}
>  
>  \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
>  Types / Network Device / Legacy Interface: Framing Requirements}
> -- 
> 2.17.2


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] Re: [PATCH] virtio-net: define support for receive-side scaling

[Yuri Benditovich]

This patch does not have too much in common with previous ones, this
is a reason why it is not "controlled steering mode v4"
According to the latest discussion about steering mode, complete
rework of the definitions was done to make this feature dedicated to
RSS only.

On Thu, Oct 3, 2019 at 12:27 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Wed, Oct 02, 2019 at 07:41:39PM +0300, Yuri Benditovich wrote:
> > Added support for RSS receive steering mode.
> >
> > Signed-off-by: Yuri Benditovich <yuri.benditovich@daynix.com>
>
> could you pls list the changes from the previous versions?
>
> > ---
> >  conformance.tex |   2 +
> >  content.tex     | 170 ++++++++++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 172 insertions(+)
> >
> > diff --git a/conformance.tex b/conformance.tex
> > index 0ac58aa..bd31ba3 100644
> > --- a/conformance.tex
> > +++ b/conformance.tex
> > @@ -101,6 +101,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
> >  \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Gratuitous Packet Sending}
> >  \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode}
> >  \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Offloads State Configuration / Setting Offloads State}
> > +\item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Using RSS commands}
> >  \end{itemize}
> >
> >  \conformance{\subsection}{Block Driver Conformance}\label{sec:Conformance / Driver Conformance / Block Driver Conformance}
> > @@ -257,6 +258,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
> >  \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Setting MAC Address Filtering}
> >  \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Gratuitous Packet Sending}
> >  \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode}
> > +\item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
> >  \end{itemize}
> >
> >  \conformance{\subsection}{Block Device Conformance}\label{sec:Conformance / Device Conformance / Block Device Conformance}
> > diff --git a/content.tex b/content.tex
> > index 679391e..5672299 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -2811,6 +2811,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> >  \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> >      channel.
> >
> > +\item[VIRTIO_NET_F_RSS(60)] Device supports RSS (receive-side scaling)
> > +    with Toeplitz hash calculation and configurable hash parameters for receive steering
> > +
> >  \item[VIRTIO_NET_F_RSC_EXT(61)] Device can process duplicated ACKs
> >      and report number of coalesced segments and duplicated ACKs
> >
> > @@ -2840,6 +2843,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> >  \item[VIRTIO_NET_F_MQ] Requires VIRTIO_NET_F_CTRL_VQ.
> >  \item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ.
> >  \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> > +\item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_MQ
> >  \end{description}
> >
> >  \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> > @@ -3785,6 +3789,172 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
> >  according to the native endian of the guest rather than
> >  (necessarily when not using the legacy interface) little-endian.
> >
> > +\paragraph{Receive-side scaling (RSS)}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}
> > +The device indicates presence of this feature if in addition to automatic receive steering it supports RSS steering with Toeplitz hash calculation and configurable parameters.
> > +
> > +Driver uses following value as \field{class} for RSS-related commands.
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_CTRL_RSS              6
> > +\end{lstlisting}
> > +Driver uses following values as \field{command} for RSS-related commands.
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES              0
> > +#define VIRTIO_NET_CTRL_RSS_SET_PARAMETERS                  1
> > +\end{lstlisting}
> > +
> > +\subparagraph{Querying RSS capabilities}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Querying RSS capabilities}
> > +
> > +Driver sends VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES command to query device capabilities related to RSS.
> > +Device responds the command using following format for \field{command-specific-data}:
> > +\begin{lstlisting}
> > +struct virtio_net_rss_capabilities {
> > +    le16 hash_types_v4;  (bitmask of supported hash types for IPv4 packets)
> > +    le16 hash_types_v6;  (bitmask of supported hash types for IPv6 packets)
> > +    le16 max_indirection_table_len; (maximal number of queue indices in indirection table)
> > +    u8 max_key_length; (maximal length of RSS key in bytes)
> > +};
> > +\end{lstlisting}
> > +To describe supported/enabled hash types device and driver use following definitions:
> > +
> > +Hash types applicable for IPv4 packets and for IPv6 packets without extension headers
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_RSS_HASH_TYPE_IP              (1 << 0)
> > +#define VIRTIO_NET_RSS_HASH_TYPE_TCP             (1 << 1)
> > +#define VIRTIO_NET_RSS_HASH_TYPE_UDP             (1 << 2)
> > +\end{lstlisting}
> > +Hash types applicable for IPv6 packets with extension headers
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_RSS_HASH_TYPE_IP_EX           (1 << 4) (only for IPv6)
> > +#define VIRTIO_NET_RSS_HASH_TYPE_TCP_EX          (1 << 5) (only for IPv6)
> > +#define VIRTIO_NET_RSS_HASH_TYPE_UDP_EX          (1 << 6) (only for IPv6)
> > +\end{lstlisting}
> > +For exact meaning of VIRTIO_NET_RSS_HASH_TYPE_ flags see \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
> > +
> > +\subparagraph{Setting RSS parameters}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}
> > +
> > +Driver sends VIRTIO_NET_CTRL_RSS_SET_PARAMETERS command using following format for \field{command-specific-data}:
> > +\begin{lstlisting}
> > +struct virtio_net_rss_params {
> > +    u8 enable_rss;
> > +    u8 hash_key_length;
> > +    le16 hash_types_v4;  (bitmask of allowed hash types for IPv4 packets)
> > +    le16 hash_types_v6;  (bitmask of allowed hash types for IPv6 packets)
> > +    le16 indirection_table_length; (number of queue indices in indirection_table array)
> > +    le16 unclassified_queue; (queue to place unclassified packets in)
> > +    le16 indirection_table[indirection_table_length];
> > +    u8 hash_key_data[hash_key_length];
> > +};
> > +
> > +\end{lstlisting}
> > +Driver sets \field{enable_rss} to 0 to disable RSS processing by the device and use automatic steering mode instead.
> > +
> > +Driver sets \field{enable_rss} to 1 to enable RSS processing by the device using provided parameters.
> > +
> > +\subparagraph{RSS hash types}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
> > +
> > +The device calculates hash on IPv4 packets according to the field \field{hash_types_v4} of virtio_net_rss_params structure as follows:
> > +\begin{itemize}
> > +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCP header, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Source IP address
> > +\item Destination IP address
> > +\item Source TCP port
> > +\item Destination TCP port
> > +\end{itemize}
> > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDP header, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Source IP address
> > +\item Destination IP address
> > +\item Source UDP port
> > +\item Destination UDP port
> > +\end{itemize}
> > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Source IP address
> > +\item Destination IP address
> > +\end{itemize}
> > +\item Else the device does not calculate the hash
> > +\end{itemize}
> > +
> > +
> > +The device calculates hash on IPv6 packets without extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
> > +\begin{itemize}
> > +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCPv6 header, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Source IPv6 address
> > +\item Destination IPv6 address
> > +\item Source TCP port
> > +\item Destination TCP port
> > +\end{itemize}
> > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDPv6 header, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Source IPv6 address
> > +\item Destination IPv6 address
> > +\item Source UDP port
> > +\item Destination UDP port
> > +\end{itemize}
> > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Source IPv6 address
> > +\item Destination IPv6 address
> > +\end{itemize}
> > +\item Else the device does not calculate the hash
> > +\end{itemize}
> > +
> > +
> > +The device calculates hash on IPv6 packets with extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
> > +\begin{itemize}
> > +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP_EX is set and the packet has TCPv6 header, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> > +\item Source TCP port
> > +\item Destination TCP port
> > +\end{itemize}
> > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP_EX is set and the packet has UDPv6 header, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> > +\item Source UDP port
> > +\item Destination UDP port
> > +\end{itemize}
> > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP_EX is set, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> > +\end{itemize}
> > +\item Else skip IPv6 extension headers and calculate the hash as defined above for IPv6 packet without extension headers
> > +\end{itemize}
> > +
> > +
> > +\drivernormative{\subparagraph}{Using RSS commands}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Using RSS commands}
> > +
> > +A driver MUST NOT use RSS commands if the feature VIRTIO_NET_F_RSS has not been negotiated.
> > +
> > +A driver MUST NOT set RSS parameters before it successfully enabled operation with multiple queues.
> > +
> > +A driver MUST fill \field{indirection_table} array only with indices of enabled queues.
> > +
> > +An \field{indirection_table_length} MUST be power of two.
> > +
> > +A driver MUST NOT set any VIRTIO_NET_RSS_HASH_TYPE_ flags that are not supported by device.
> > +
> > +A driver MUST NOT set any bits below for \field{hash_types_v4}:
> > +\begin{itemize}
> > +\item VIRTIO_NET_RSS_HASH_TYPE_IP_EX
> > +\item VIRTIO_NET_RSS_HASH_TYPE_TCP_EX
> > +\item VIRTIO_NET_RSS_HASH_TYPE_UDP_EX
> > +\end{itemize}
> > +
> > +\devicenormative{\subparagraph}{RSS processing}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
> > +If the device reports support for VIRTIO_NET_F_RSS it MUST support keys of at least 40 bytes and indirection table of at least 128 entries.
> > +
> > +The device MUST determine destination queue for network packet as follows:
> > +\begin{itemize}
> > +\item Calculate hash of the packet as defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
> > +\item If the device did not calculate the hash for specific packet, the device directs the packet to the queue specified by \field{unclassified_queue} of virtio_net_rss_params structure
> > +\item Apply mask of (indirection_table_length - 1) to the calculated hash and use the result as the index in the indirection table to get 0-based number of destination receive queue
> > +\end{itemize}
> >
> >  \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
> >  Types / Network Device / Legacy Interface: Framing Requirements}
> > --
> > 2.17.2
>

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] [PATCH] virtio-net: define support for receive-side scaling

[Michael S. Tsirkin]

On Wed, Oct 02, 2019 at 07:41:39PM +0300, Yuri Benditovich wrote:
> Added support for RSS receive steering mode.
> 
> Signed-off-by: Yuri Benditovich <yuri.benditovich@daynix.com>


Thanks!
So I would like this to be a bit flatter in the device memory,
such that these are easier to intercept.
See below for detail.



> ---
>  conformance.tex |   2 +
>  content.tex     | 170 ++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 172 insertions(+)

So the description of the new functionality is reasonable, but it looks
like you didn't yet go over the existing text and see that
it makes sense with the new one.

For example, a cursory look gave me statements like

	When multiqueue is enabled, the device MUST use automatic receive steering
	based on packet flow.

or

	Multiqueue is disabled by default. The driver enables multiqueue by
	executing the VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command, specifying
	the number of the transmit and receive queues to be used up to
	\field{max_virtqueue_pairs}; subsequently,
	transmitq1\ldots transmitqn and receiveq1\ldots receiveqn where
	n=\field{virtqueue_pairs} MAY be used.


we need to change the existing text separating multiqueue operation
that applies to both auto steering and to RSS, and auto steering proper.


I also think that it's not wise to always require auto steering
support like your text does: auto steering might be
expensive for hardware to implement, RSS might be easier.




> diff --git a/conformance.tex b/conformance.tex
> index 0ac58aa..bd31ba3 100644
> --- a/conformance.tex
> +++ b/conformance.tex
> @@ -101,6 +101,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
>  \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Gratuitous Packet Sending}
>  \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode}
>  \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Offloads State Configuration / Setting Offloads State}
> +\item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Using RSS commands}
>  \end{itemize}
>  
>  \conformance{\subsection}{Block Driver Conformance}\label{sec:Conformance / Driver Conformance / Block Driver Conformance}
> @@ -257,6 +258,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
>  \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Setting MAC Address Filtering}
>  \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Gratuitous Packet Sending}
>  \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode}
> +\item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
>  \end{itemize}
>  
>  \conformance{\subsection}{Block Device Conformance}\label{sec:Conformance / Device Conformance / Block Device Conformance}
> diff --git a/content.tex b/content.tex
> index 679391e..5672299 100644
> --- a/content.tex
> +++ b/content.tex
> @@ -2811,6 +2811,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
>  \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
>      channel.
>  
> +\item[VIRTIO_NET_F_RSS(60)] Device supports RSS (receive-side scaling)
> +    with Toeplitz hash calculation and configurable hash parameters for receive steering
> +
>  \item[VIRTIO_NET_F_RSC_EXT(61)] Device can process duplicated ACKs
>      and report number of coalesced segments and duplicated ACKs
>  
> @@ -2840,6 +2843,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
>  \item[VIRTIO_NET_F_MQ] Requires VIRTIO_NET_F_CTRL_VQ.
>  \item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ.
>  \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> +\item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_MQ
>  \end{description}
>  
>  \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> @@ -3785,6 +3789,172 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>  according to the native endian of the guest rather than
>  (necessarily when not using the legacy interface) little-endian.
>  
> +\paragraph{Receive-side scaling (RSS)}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}
> +The device indicates presence of this feature if in addition to automatic receive steering it supports RSS steering with Toeplitz hash calculation and configurable parameters.
> +
> +Driver uses following value as \field{class} for RSS-related commands.
> +\begin{lstlisting}
> +#define VIRTIO_NET_CTRL_RSS              6
> +\end{lstlisting}

Why don't we reuse VIRTIO_NET_CTRL_MQ?
It seems appropriate.


> +Driver uses following values as \field{command} for RSS-related commands.
> +\begin{lstlisting}
> +#define VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES              0
> +#define VIRTIO_NET_CTRL_RSS_SET_PARAMETERS                  1
> +\end{lstlisting}
> +
> +\subparagraph{Querying RSS capabilities}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Querying RSS capabilities}
> +
> +Driver sends VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES command to query device capabilities related to RSS.
> +Device responds the command using following format for \field{command-specific-data}:
> +\begin{lstlisting}
> +struct virtio_net_rss_capabilities {
> +    le16 hash_types_v4;  (bitmask of supported hash types for IPv4 packets)
> +    le16 hash_types_v6;  (bitmask of supported hash types for IPv6 packets)


I think hash types should move into feature bits.
There are 8 of them overall, we are not short on bits.

Or if you prefer, we can add a full 32 bit mask
for device specific things.



> +    le16 max_indirection_table_len; (maximal number of queue indices in indirection table)
> +    u8 max_key_length; (maximal length of RSS key in bytes)


I would put the max values in the config space.

> +};
> +\end{lstlisting}
> +To describe supported/enabled hash types device and driver use following definitions:
> +
> +Hash types applicable for IPv4 packets and for IPv6 packets without extension headers
> +\begin{lstlisting}
> +#define VIRTIO_NET_RSS_HASH_TYPE_IP              (1 << 0)
> +#define VIRTIO_NET_RSS_HASH_TYPE_TCP             (1 << 1)
> +#define VIRTIO_NET_RSS_HASH_TYPE_UDP             (1 << 2)
> +\end{lstlisting}
> +Hash types applicable for IPv6 packets with extension headers
> +\begin{lstlisting}
> +#define VIRTIO_NET_RSS_HASH_TYPE_IP_EX           (1 << 4) (only for IPv6)
> +#define VIRTIO_NET_RSS_HASH_TYPE_TCP_EX          (1 << 5) (only for IPv6)
> +#define VIRTIO_NET_RSS_HASH_TYPE_UDP_EX          (1 << 6) (only for IPv6)
> +\end{lstlisting}
> +For exact meaning of VIRTIO_NET_RSS_HASH_TYPE_ flags see \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
> +
> +\subparagraph{Setting RSS parameters}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}
> +
> +Driver sends VIRTIO_NET_CTRL_RSS_SET_PARAMETERS command using following format for \field{command-specific-data}:
> +\begin{lstlisting}
> +struct virtio_net_rss_params {
> +    u8 enable_rss;
> +    u8 hash_key_length;
> +    le16 hash_types_v4;  (bitmask of allowed hash types for IPv4 packets)
> +    le16 hash_types_v6;  (bitmask of allowed hash types for IPv6 packets)
> +    le16 indirection_table_length; (number of queue indices in indirection_table array)
> +    le16 unclassified_queue; (queue to place unclassified packets in)
> +    le16 indirection_table[indirection_table_length];
> +    u8 hash_key_data[hash_key_length];
> +};
> +
> +\end{lstlisting}
> +Driver sets \field{enable_rss} to 0 to disable RSS processing by the device and use automatic steering mode instead.

I am not sure it's a good idea to have !RSS mean automatic steering.
We might have more options in the future.
Why not just enable RSS when the command is given?

ATM we don't have a command to force a single queue mode.
Maybe we want a feature bit and a command for that.

> +
> +Driver sets \field{enable_rss} to 1 to enable RSS processing by the device using provided parameters.
> +
> +\subparagraph{RSS hash types}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
> +
> +The device calculates hash on IPv4 packets according to the field \field{hash_types_v4} of virtio_net_rss_params structure as follows:
> +\begin{itemize}
> +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCP header, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Source IP address
> +\item Destination IP address
> +\item Source TCP port
> +\item Destination TCP port
> +\end{itemize}
> +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDP header, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Source IP address
> +\item Destination IP address
> +\item Source UDP port
> +\item Destination UDP port
> +\end{itemize}
> +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Source IP address
> +\item Destination IP address
> +\end{itemize}
> +\item Else the device does not calculate the hash
> +\end{itemize}
> +
> +
> +The device calculates hash on IPv6 packets without extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
> +\begin{itemize}
> +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCPv6 header, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Source IPv6 address
> +\item Destination IPv6 address
> +\item Source TCP port
> +\item Destination TCP port
> +\end{itemize}
> +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDPv6 header, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Source IPv6 address
> +\item Destination IPv6 address
> +\item Source UDP port
> +\item Destination UDP port
> +\end{itemize}
> +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Source IPv6 address
> +\item Destination IPv6 address
> +\end{itemize}
> +\item Else the device does not calculate the hash
> +\end{itemize}
> +
> +
> +The device calculates hash on IPv6 packets with extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
> +\begin{itemize}
> +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP_EX is set and the packet has TCPv6 header, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> +\item Source TCP port
> +\item Destination TCP port
> +\end{itemize}
> +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP_EX is set and the packet has UDPv6 header, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> +\item Source UDP port
> +\item Destination UDP port
> +\end{itemize}
> +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP_EX is set, the hash is calculated over following fields:
> +\begin{itemize}
> +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> +\end{itemize}
> +\item Else skip IPv6 extension headers and calculate the hash as defined above for IPv6 packet without extension headers
> +\end{itemize}
> +
> +
> +\drivernormative{\subparagraph}{Using RSS commands}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Using RSS commands}
> +
> +A driver MUST NOT use RSS commands if the feature VIRTIO_NET_F_RSS has not been negotiated.
> +
> +A driver MUST NOT set RSS parameters before it successfully enabled operation with multiple queues.
> +
> +A driver MUST fill \field{indirection_table} array only with indices of enabled queues.
> +
> +An \field{indirection_table_length} MUST be power of two.
> +
> +A driver MUST NOT set any VIRTIO_NET_RSS_HASH_TYPE_ flags that are not supported by device.
> +
> +A driver MUST NOT set any bits below for \field{hash_types_v4}:
> +\begin{itemize}
> +\item VIRTIO_NET_RSS_HASH_TYPE_IP_EX
> +\item VIRTIO_NET_RSS_HASH_TYPE_TCP_EX
> +\item VIRTIO_NET_RSS_HASH_TYPE_UDP_EX
> +\end{itemize}
> +
> +\devicenormative{\subparagraph}{RSS processing}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
> +If the device reports support for VIRTIO_NET_F_RSS it MUST support keys of at least 40 bytes and indirection table of at least 128 entries.
> +
> +The device MUST determine destination queue for network packet as follows:
> +\begin{itemize}
> +\item Calculate hash of the packet as defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
> +\item If the device did not calculate the hash for specific packet, the device directs the packet to the queue specified by \field{unclassified_queue} of virtio_net_rss_params structure
> +\item Apply mask of (indirection_table_length - 1) to the calculated hash and use the result as the index in the indirection table to get 0-based number of destination receive queue
> +\end{itemize}
>  
>  \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
>  Types / Network Device / Legacy Interface: Framing Requirements}
> -- 
> 2.17.2
> 
> 
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
> 
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
> 
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] [PATCH] virtio-net: define support for receive-side scaling

[Yuri Benditovich]

On Sat, Oct 12, 2019 at 7:55 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Wed, Oct 02, 2019 at 07:41:39PM +0300, Yuri Benditovich wrote:
> > Added support for RSS receive steering mode.
> >
> > Signed-off-by: Yuri Benditovich <yuri.benditovich@daynix.com>
>
>
> Thanks!
> So I would like this to be a bit flatter in the device memory,
> such that these are easier to intercept.
> See below for detail.
>
>
>
> > ---
> >  conformance.tex |   2 +
> >  content.tex     | 170 ++++++++++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 172 insertions(+)
>
> So the description of the new functionality is reasonable, but it looks
> like you didn't yet go over the existing text and see that
> it makes sense with the new one.
>
> For example, a cursory look gave me statements like
>
>         When multiqueue is enabled, the device MUST use automatic receive steering
>         based on packet flow.
>
> or
>
>         Multiqueue is disabled by default. The driver enables multiqueue by
>         executing the VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command, specifying
>         the number of the transmit and receive queues to be used up to
>         \field{max_virtqueue_pairs}; subsequently,
>         transmitq1\ldots transmitqn and receiveq1\ldots receiveqn where
>         n=\field{virtqueue_pairs} MAY be used.
>
>
> we need to change the existing text separating multiqueue operation
> that applies to both auto steering and to RSS, and auto steering proper.

I agree. I'll make respective changes in v2

>
> I also think that it's not wise to always require auto steering
> support like your text does: auto steering might be
> expensive for hardware to implement, RSS might be easier.
>

I agree that exact definition of how auto streering works should be changed.
I'll try to make it less imperative, probably using existing
definition as an example of implementation and not a mandatory
requirement.

>
>
>
> > diff --git a/conformance.tex b/conformance.tex
> > index 0ac58aa..bd31ba3 100644
> > --- a/conformance.tex
> > +++ b/conformance.tex
> > @@ -101,6 +101,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
> >  \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Gratuitous Packet Sending}
> >  \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode}
> >  \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Offloads State Configuration / Setting Offloads State}
> > +\item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Using RSS commands}
> >  \end{itemize}
> >
> >  \conformance{\subsection}{Block Driver Conformance}\label{sec:Conformance / Driver Conformance / Block Driver Conformance}
> > @@ -257,6 +258,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets}
> >  \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Setting MAC Address Filtering}
> >  \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Gratuitous Packet Sending}
> >  \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode}
> > +\item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
> >  \end{itemize}
> >
> >  \conformance{\subsection}{Block Device Conformance}\label{sec:Conformance / Device Conformance / Block Device Conformance}
> > diff --git a/content.tex b/content.tex
> > index 679391e..5672299 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -2811,6 +2811,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> >  \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> >      channel.
> >
> > +\item[VIRTIO_NET_F_RSS(60)] Device supports RSS (receive-side scaling)
> > +    with Toeplitz hash calculation and configurable hash parameters for receive steering
> > +
> >  \item[VIRTIO_NET_F_RSC_EXT(61)] Device can process duplicated ACKs
> >      and report number of coalesced segments and duplicated ACKs
> >
> > @@ -2840,6 +2843,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> >  \item[VIRTIO_NET_F_MQ] Requires VIRTIO_NET_F_CTRL_VQ.
> >  \item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ.
> >  \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> > +\item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_MQ
> >  \end{description}
> >
> >  \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> > @@ -3785,6 +3789,172 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
> >  according to the native endian of the guest rather than
> >  (necessarily when not using the legacy interface) little-endian.
> >
> > +\paragraph{Receive-side scaling (RSS)}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS)}
> > +The device indicates presence of this feature if in addition to automatic receive steering it supports RSS steering with Toeplitz hash calculation and configurable parameters.
> > +
> > +Driver uses following value as \field{class} for RSS-related commands.
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_CTRL_RSS              6
> > +\end{lstlisting}
>
> Why don't we reuse VIRTIO_NET_CTRL_MQ?
> It seems appropriate.

No problem, we can reuse VIRTIO_NET_CTRL_MQ for RSS functionality.

>
>
> > +Driver uses following values as \field{command} for RSS-related commands.
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES              0
> > +#define VIRTIO_NET_CTRL_RSS_SET_PARAMETERS                  1
> > +\end{lstlisting}
> > +
> > +\subparagraph{Querying RSS capabilities}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Querying RSS capabilities}
> > +
> > +Driver sends VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES command to query device capabilities related to RSS.
> > +Device responds the command using following format for \field{command-specific-data}:
> > +\begin{lstlisting}
> > +struct virtio_net_rss_capabilities {
> > +    le16 hash_types_v4;  (bitmask of supported hash types for IPv4 packets)
> > +    le16 hash_types_v6;  (bitmask of supported hash types for IPv6 packets)
>
>
> I think hash types should move into feature bits.
> There are 8 of them overall, we are not short on bits.
>
At the moment we have almost 30 feature bits for network devices (the
range 0..23 defined for specific devices is fully used)
The range 38..63 defined for futher extensions is partially used
(there is also some feature bits that aren't documented yet).
I think we do not have too much unused feature bits and I would prefer
to indicate support for different types of hashes via dedicated
command.
There are also other RSS-related capabilities that the driver should
know (see below)

> Or if you prefer, we can add a full 32 bit mask
> for device specific things.
>
>
>
> > +    le16 max_indirection_table_len; (maximal number of queue indices in indirection table)
> > +    u8 max_key_length; (maximal length of RSS key in bytes)
>
>
> I would put the max values in the config space.

Indeed, we can extend the config layout of virtio_net device
(virtio_net_config) and discard VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES
command.

Unfortunately the definition of config space (virtio_net_config) is
not synchronized between the spec and kernel headers.
In kernel/qemu headers (used also by Win drivers) it was extended to
accomodate speed and duplex and this definition is not present in the
spec.
Although there was some discussion about respective virtio_net feature
I do not see any spec patch related to this change.

Do you prefer to add 'reserved' fields for speed/duplex and then add
fields for RSS related capabilities (max_key_size,
max_indirection_table_len)?
In this case also bitmask of supported hashes can be placed here, I think.

>
> > +};
> > +\end{lstlisting}
> > +To describe supported/enabled hash types device and driver use following definitions:
> > +
> > +Hash types applicable for IPv4 packets and for IPv6 packets without extension headers
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_RSS_HASH_TYPE_IP              (1 << 0)
> > +#define VIRTIO_NET_RSS_HASH_TYPE_TCP             (1 << 1)
> > +#define VIRTIO_NET_RSS_HASH_TYPE_UDP             (1 << 2)
> > +\end{lstlisting}
> > +Hash types applicable for IPv6 packets with extension headers
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_RSS_HASH_TYPE_IP_EX           (1 << 4) (only for IPv6)
> > +#define VIRTIO_NET_RSS_HASH_TYPE_TCP_EX          (1 << 5) (only for IPv6)
> > +#define VIRTIO_NET_RSS_HASH_TYPE_UDP_EX          (1 << 6) (only for IPv6)
> > +\end{lstlisting}
> > +For exact meaning of VIRTIO_NET_RSS_HASH_TYPE_ flags see \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
> > +
> > +\subparagraph{Setting RSS parameters}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}
> > +
> > +Driver sends VIRTIO_NET_CTRL_RSS_SET_PARAMETERS command using following format for \field{command-specific-data}:
> > +\begin{lstlisting}
> > +struct virtio_net_rss_params {
> > +    u8 enable_rss;
> > +    u8 hash_key_length;
> > +    le16 hash_types_v4;  (bitmask of allowed hash types for IPv4 packets)
> > +    le16 hash_types_v6;  (bitmask of allowed hash types for IPv6 packets)
> > +    le16 indirection_table_length; (number of queue indices in indirection_table array)
> > +    le16 unclassified_queue; (queue to place unclassified packets in)
> > +    le16 indirection_table[indirection_table_length];
> > +    u8 hash_key_data[hash_key_length];
> > +};
> > +
> > +\end{lstlisting}
> > +Driver sets \field{enable_rss} to 0 to disable RSS processing by the device and use automatic steering mode instead.
>
> I am not sure it's a good idea to have !RSS mean automatic steering.
> We might have more options in the future.
> Why not just enable RSS when the command is given?

The driver (Win for sure) can receive a command to disable RSS (also
during run-time).
In this case it makes sense that the device does its default steering.
If in future we have some additional options this is responsibility of
the driver to configure the steering accordingly.

>
> ATM we don't have a command to force a single queue mode.
> Maybe we want a feature bit and a command for that.

According to the spec (5.1.6.5.5):
Multiqueue is disabled by setting virtqueue_pairs to 1 (this is the
default) and waiting for the device to use the command buffer.

>
> > +
> > +Driver sets \field{enable_rss} to 1 to enable RSS processing by the device using provided parameters.
> > +
> > +\subparagraph{RSS hash types}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
> > +
> > +The device calculates hash on IPv4 packets according to the field \field{hash_types_v4} of virtio_net_rss_params structure as follows:
> > +\begin{itemize}
> > +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCP header, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Source IP address
> > +\item Destination IP address
> > +\item Source TCP port
> > +\item Destination TCP port
> > +\end{itemize}
> > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDP header, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Source IP address
> > +\item Destination IP address
> > +\item Source UDP port
> > +\item Destination UDP port
> > +\end{itemize}
> > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Source IP address
> > +\item Destination IP address
> > +\end{itemize}
> > +\item Else the device does not calculate the hash
> > +\end{itemize}
> > +
> > +
> > +The device calculates hash on IPv6 packets without extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
> > +\begin{itemize}
> > +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCPv6 header, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Source IPv6 address
> > +\item Destination IPv6 address
> > +\item Source TCP port
> > +\item Destination TCP port
> > +\end{itemize}
> > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDPv6 header, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Source IPv6 address
> > +\item Destination IPv6 address
> > +\item Source UDP port
> > +\item Destination UDP port
> > +\end{itemize}
> > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Source IPv6 address
> > +\item Destination IPv6 address
> > +\end{itemize}
> > +\item Else the device does not calculate the hash
> > +\end{itemize}
> > +
> > +
> > +The device calculates hash on IPv6 packets with extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
> > +\begin{itemize}
> > +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP_EX is set and the packet has TCPv6 header, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> > +\item Source TCP port
> > +\item Destination TCP port
> > +\end{itemize}
> > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP_EX is set and the packet has UDPv6 header, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> > +\item Source UDP port
> > +\item Destination UDP port
> > +\end{itemize}
> > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP_EX is set, the hash is calculated over following fields:
> > +\begin{itemize}
> > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> > +\end{itemize}
> > +\item Else skip IPv6 extension headers and calculate the hash as defined above for IPv6 packet without extension headers
> > +\end{itemize}
> > +
> > +
> > +\drivernormative{\subparagraph}{Using RSS commands}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Using RSS commands}
> > +
> > +A driver MUST NOT use RSS commands if the feature VIRTIO_NET_F_RSS has not been negotiated.
> > +
> > +A driver MUST NOT set RSS parameters before it successfully enabled operation with multiple queues.
> > +
> > +A driver MUST fill \field{indirection_table} array only with indices of enabled queues.
> > +
> > +An \field{indirection_table_length} MUST be power of two.
> > +
> > +A driver MUST NOT set any VIRTIO_NET_RSS_HASH_TYPE_ flags that are not supported by device.
> > +
> > +A driver MUST NOT set any bits below for \field{hash_types_v4}:
> > +\begin{itemize}
> > +\item VIRTIO_NET_RSS_HASH_TYPE_IP_EX
> > +\item VIRTIO_NET_RSS_HASH_TYPE_TCP_EX
> > +\item VIRTIO_NET_RSS_HASH_TYPE_UDP_EX
> > +\end{itemize}
> > +
> > +\devicenormative{\subparagraph}{RSS processing}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
> > +If the device reports support for VIRTIO_NET_F_RSS it MUST support keys of at least 40 bytes and indirection table of at least 128 entries.
> > +
> > +The device MUST determine destination queue for network packet as follows:
> > +\begin{itemize}
> > +\item Calculate hash of the packet as defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
> > +\item If the device did not calculate the hash for specific packet, the device directs the packet to the queue specified by \field{unclassified_queue} of virtio_net_rss_params structure
> > +\item Apply mask of (indirection_table_length - 1) to the calculated hash and use the result as the index in the indirection table to get 0-based number of destination receive queue
> > +\end{itemize}
> >
> >  \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
> >  Types / Network Device / Legacy Interface: Framing Requirements}
> > --
> > 2.17.2
> >
> >
> > This publicly archived list offers a means to provide input to the
> > OASIS Virtual I/O Device (VIRTIO) TC.
> >
> > In order to verify user consent to the Feedback License terms and
> > to minimize spam in the list archive, subscription is required
> > before posting.
> >
> > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > List help: virtio-comment-help@lists.oasis-open.org
> > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > Committee: https://www.oasis-open.org/committees/virtio/
> > Join OASIS: https://www.oasis-open.org/join/
>
>
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
>

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] [PATCH] virtio-net: define support for receive-side scaling

[Michael S. Tsirkin]

On Sun, Oct 13, 2019 at 01:27:59PM +0300, Yuri Benditovich wrote:
> > > +Driver sends VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES command to query device capabilities related to RSS.
> > > +Device responds the command using following format for \field{command-specific-data}:
> > > +\begin{lstlisting}
> > > +struct virtio_net_rss_capabilities {
> > > +    le16 hash_types_v4;  (bitmask of supported hash types for IPv4 packets)
> > > +    le16 hash_types_v6;  (bitmask of supported hash types for IPv6 packets)
> >
> >
> > I think hash types should move into feature bits.
> > There are 8 of them overall, we are not short on bits.
> >
> At the moment we have almost 30 feature bits for network devices (the
> range 0..23 defined for specific devices is fully used)
> The range 38..63 defined for futher extensions is partially used
> (there is also some feature bits that aren't documented yet).
> I think we do not have too much unused feature bits

That's not true.  Used to be a problem pre 1.0 when
pci had a dedicated dword register. Since 1.0 there's
a selector which gives you any number of dwords.

You can start using 64..95 immediately once
we run out of the 32..63 range.


> and I would prefer
> to indicate support for different types of hashes via dedicated
> command.

Big problem with using extra commands for discovery is that they break
passthrough. Tricks like VDPA can intercept and tweak feature
bits easily, and to a lesser extent config space accesses.
Intercepting vq commands is very tricky.


> There are also other RSS-related capabilities that the driver should
> know (see below)
> 
> > Or if you prefer, we can add a full 32 bit mask
> > for device specific things.
> >
> >
> >
> > > +    le16 max_indirection_table_len; (maximal number of queue indices in indirection table)
> > > +    u8 max_key_length; (maximal length of RSS key in bytes)
> >
> >
> > I would put the max values in the config space.
> 
> Indeed, we can extend the config layout of virtio_net device
> (virtio_net_config) and discard VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES
> command.
> 
> Unfortunately the definition of config space (virtio_net_config) is
> not synchronized between the spec and kernel headers.
> In kernel/qemu headers (used also by Win drivers) it was extended to
> accomodate speed and duplex and this definition is not present in the
> spec.
> Although there was some discussion about respective virtio_net feature
> I do not see any spec patch related to this change.
> 
> Do you prefer to add 'reserved' fields for speed/duplex and then add
> fields for RSS related capabilities (max_key_size,
> max_indirection_table_len)?

I'd rather we added the fields, with minimal documentation for now.

> In this case also bitmask of supported hashes can be placed here, I think.
> 
> >
> > > +};
> > > +\end{lstlisting}
> > > +To describe supported/enabled hash types device and driver use following definitions:
> > > +
> > > +Hash types applicable for IPv4 packets and for IPv6 packets without extension headers
> > > +\begin{lstlisting}
> > > +#define VIRTIO_NET_RSS_HASH_TYPE_IP              (1 << 0)
> > > +#define VIRTIO_NET_RSS_HASH_TYPE_TCP             (1 << 1)
> > > +#define VIRTIO_NET_RSS_HASH_TYPE_UDP             (1 << 2)
> > > +\end{lstlisting}
> > > +Hash types applicable for IPv6 packets with extension headers
> > > +\begin{lstlisting}
> > > +#define VIRTIO_NET_RSS_HASH_TYPE_IP_EX           (1 << 4) (only for IPv6)
> > > +#define VIRTIO_NET_RSS_HASH_TYPE_TCP_EX          (1 << 5) (only for IPv6)
> > > +#define VIRTIO_NET_RSS_HASH_TYPE_UDP_EX          (1 << 6) (only for IPv6)
> > > +\end{lstlisting}
> > > +For exact meaning of VIRTIO_NET_RSS_HASH_TYPE_ flags see \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
> > > +
> > > +\subparagraph{Setting RSS parameters}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}
> > > +
> > > +Driver sends VIRTIO_NET_CTRL_RSS_SET_PARAMETERS command using following format for \field{command-specific-data}:
> > > +\begin{lstlisting}
> > > +struct virtio_net_rss_params {
> > > +    u8 enable_rss;
> > > +    u8 hash_key_length;
> > > +    le16 hash_types_v4;  (bitmask of allowed hash types for IPv4 packets)
> > > +    le16 hash_types_v6;  (bitmask of allowed hash types for IPv6 packets)
> > > +    le16 indirection_table_length; (number of queue indices in indirection_table array)
> > > +    le16 unclassified_queue; (queue to place unclassified packets in)
> > > +    le16 indirection_table[indirection_table_length];
> > > +    u8 hash_key_data[hash_key_length];
> > > +};
> > > +
> > > +\end{lstlisting}
> > > +Driver sets \field{enable_rss} to 0 to disable RSS processing by the device and use automatic steering mode instead.
> >
> > I am not sure it's a good idea to have !RSS mean automatic steering.
> > We might have more options in the future.
> > Why not just enable RSS when the command is given?
> 
> The driver (Win for sure) can receive a command to disable RSS (also
> during run-time).
> In this case it makes sense that the device does its default steering.

Right, but if a hardware device implements RSS it might not want
to also support automatic steering.
What I'm saying is that it's better to have driver exiplicitly set
the steering, instead of enable/disable multiple options individually,
and then have device figure out what is ment if multiple ones are
enabled, or if no one is enabled.

> If in future we have some additional options this is responsibility of
> the driver to configure the steering accordingly.
> 
> >
> > ATM we don't have a command to force a single queue mode.
> > Maybe we want a feature bit and a command for that.
> 
> According to the spec (5.1.6.5.5):
> Multiqueue is disabled by setting virtqueue_pairs to 1 (this is the
> default) and waiting for the device to use the command buffer.

right, so how does one disable RSS?

> >
> > > +
> > > +Driver sets \field{enable_rss} to 1 to enable RSS processing by the device using provided parameters.
> > > +
> > > +\subparagraph{RSS hash types}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
> > > +
> > > +The device calculates hash on IPv4 packets according to the field \field{hash_types_v4} of virtio_net_rss_params structure as follows:
> > > +\begin{itemize}
> > > +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCP header, the hash is calculated over following fields:
> > > +\begin{itemize}
> > > +\item Source IP address
> > > +\item Destination IP address
> > > +\item Source TCP port
> > > +\item Destination TCP port
> > > +\end{itemize}
> > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDP header, the hash is calculated over following fields:
> > > +\begin{itemize}
> > > +\item Source IP address
> > > +\item Destination IP address
> > > +\item Source UDP port
> > > +\item Destination UDP port
> > > +\end{itemize}
> > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
> > > +\begin{itemize}
> > > +\item Source IP address
> > > +\item Destination IP address
> > > +\end{itemize}
> > > +\item Else the device does not calculate the hash
> > > +\end{itemize}
> > > +
> > > +
> > > +The device calculates hash on IPv6 packets without extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
> > > +\begin{itemize}
> > > +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCPv6 header, the hash is calculated over following fields:
> > > +\begin{itemize}
> > > +\item Source IPv6 address
> > > +\item Destination IPv6 address
> > > +\item Source TCP port
> > > +\item Destination TCP port
> > > +\end{itemize}
> > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDPv6 header, the hash is calculated over following fields:
> > > +\begin{itemize}
> > > +\item Source IPv6 address
> > > +\item Destination IPv6 address
> > > +\item Source UDP port
> > > +\item Destination UDP port
> > > +\end{itemize}
> > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
> > > +\begin{itemize}
> > > +\item Source IPv6 address
> > > +\item Destination IPv6 address
> > > +\end{itemize}
> > > +\item Else the device does not calculate the hash
> > > +\end{itemize}
> > > +
> > > +
> > > +The device calculates hash on IPv6 packets with extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
> > > +\begin{itemize}
> > > +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP_EX is set and the packet has TCPv6 header, the hash is calculated over following fields:
> > > +\begin{itemize}
> > > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> > > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> > > +\item Source TCP port
> > > +\item Destination TCP port
> > > +\end{itemize}
> > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP_EX is set and the packet has UDPv6 header, the hash is calculated over following fields:
> > > +\begin{itemize}
> > > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> > > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> > > +\item Source UDP port
> > > +\item Destination UDP port
> > > +\end{itemize}
> > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP_EX is set, the hash is calculated over following fields:
> > > +\begin{itemize}
> > > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> > > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> > > +\end{itemize}
> > > +\item Else skip IPv6 extension headers and calculate the hash as defined above for IPv6 packet without extension headers
> > > +\end{itemize}
> > > +
> > > +
> > > +\drivernormative{\subparagraph}{Using RSS commands}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Using RSS commands}
> > > +
> > > +A driver MUST NOT use RSS commands if the feature VIRTIO_NET_F_RSS has not been negotiated.
> > > +
> > > +A driver MUST NOT set RSS parameters before it successfully enabled operation with multiple queues.
> > > +
> > > +A driver MUST fill \field{indirection_table} array only with indices of enabled queues.
> > > +
> > > +An \field{indirection_table_length} MUST be power of two.
> > > +
> > > +A driver MUST NOT set any VIRTIO_NET_RSS_HASH_TYPE_ flags that are not supported by device.
> > > +
> > > +A driver MUST NOT set any bits below for \field{hash_types_v4}:
> > > +\begin{itemize}
> > > +\item VIRTIO_NET_RSS_HASH_TYPE_IP_EX
> > > +\item VIRTIO_NET_RSS_HASH_TYPE_TCP_EX
> > > +\item VIRTIO_NET_RSS_HASH_TYPE_UDP_EX
> > > +\end{itemize}
> > > +
> > > +\devicenormative{\subparagraph}{RSS processing}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
> > > +If the device reports support for VIRTIO_NET_F_RSS it MUST support keys of at least 40 bytes and indirection table of at least 128 entries.
> > > +
> > > +The device MUST determine destination queue for network packet as follows:
> > > +\begin{itemize}
> > > +\item Calculate hash of the packet as defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
> > > +\item If the device did not calculate the hash for specific packet, the device directs the packet to the queue specified by \field{unclassified_queue} of virtio_net_rss_params structure
> > > +\item Apply mask of (indirection_table_length - 1) to the calculated hash and use the result as the index in the indirection table to get 0-based number of destination receive queue
> > > +\end{itemize}
> > >
> > >  \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
> > >  Types / Network Device / Legacy Interface: Framing Requirements}
> > > --
> > > 2.17.2
> > >
> > >
> > > This publicly archived list offers a means to provide input to the
> > > OASIS Virtual I/O Device (VIRTIO) TC.
> > >
> > > In order to verify user consent to the Feedback License terms and
> > > to minimize spam in the list archive, subscription is required
> > > before posting.
> > >
> > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > List help: virtio-comment-help@lists.oasis-open.org
> > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > Committee: https://www.oasis-open.org/committees/virtio/
> > > Join OASIS: https://www.oasis-open.org/join/
> >
> >
> > This publicly archived list offers a means to provide input to the
> > OASIS Virtual I/O Device (VIRTIO) TC.
> >
> > In order to verify user consent to the Feedback License terms and
> > to minimize spam in the list archive, subscription is required
> > before posting.
> >
> > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > List help: virtio-comment-help@lists.oasis-open.org
> > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > Committee: https://www.oasis-open.org/committees/virtio/
> > Join OASIS: https://www.oasis-open.org/join/
> >


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] [PATCH] virtio-net: define support for receive-side scaling

[Yuri Benditovich]

On Sun, Oct 13, 2019 at 3:03 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Sun, Oct 13, 2019 at 01:27:59PM +0300, Yuri Benditovich wrote:
> > > > +Driver sends VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES command to query device capabilities related to RSS.
> > > > +Device responds the command using following format for \field{command-specific-data}:
> > > > +\begin{lstlisting}
> > > > +struct virtio_net_rss_capabilities {
> > > > +    le16 hash_types_v4;  (bitmask of supported hash types for IPv4 packets)
> > > > +    le16 hash_types_v6;  (bitmask of supported hash types for IPv6 packets)
> > >
> > >
> > > I think hash types should move into feature bits.
> > > There are 8 of them overall, we are not short on bits.
> > >
> > At the moment we have almost 30 feature bits for network devices (the
> > range 0..23 defined for specific devices is fully used)
> > The range 38..63 defined for futher extensions is partially used
> > (there is also some feature bits that aren't documented yet).
> > I think we do not have too much unused feature bits
>
> That's not true.  Used to be a problem pre 1.0 when
> pci had a dedicated dword register. Since 1.0 there's
> a selector which gives you any number of dwords.
>
> You can start using 64..95 immediately once
> we run out of the 32..63 range.
>

Good to know. But this requires support for such features in QEMU.
Currently virtio-net supports only 64 bits for features (as well as
all the drivers, if I'm not mistaken)

>
> > and I would prefer
> > to indicate support for different types of hashes via dedicated
> > command.
>
> Big problem with using extra commands for discovery is that they break
> passthrough. Tricks like VDPA can intercept and tweak feature
> bits easily, and to a lesser extent config space accesses.
> Intercepting vq commands is very tricky.
>

RSS requires 'SET' command anyway.

>
> > There are also other RSS-related capabilities that the driver should
> > know (see below)
> >
> > > Or if you prefer, we can add a full 32 bit mask
> > > for device specific things.
> > >
> > >
> > >
> > > > +    le16 max_indirection_table_len; (maximal number of queue indices in indirection table)
> > > > +    u8 max_key_length; (maximal length of RSS key in bytes)
> > >
> > >
> > > I would put the max values in the config space.
> >
> > Indeed, we can extend the config layout of virtio_net device
> > (virtio_net_config) and discard VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES
> > command.
> >
> > Unfortunately the definition of config space (virtio_net_config) is
> > not synchronized between the spec and kernel headers.
> > In kernel/qemu headers (used also by Win drivers) it was extended to
> > accomodate speed and duplex and this definition is not present in the
> > spec.
> > Although there was some discussion about respective virtio_net feature
> > I do not see any spec patch related to this change.
> >
> > Do you prefer to add 'reserved' fields for speed/duplex and then add
> > fields for RSS related capabilities (max_key_size,
> > max_indirection_table_len)?
>
> I'd rather we added the fields, with minimal documentation for now.
>
> > In this case also bitmask of supported hashes can be placed here, I think.
> >
> > >
> > > > +};
> > > > +\end{lstlisting}
> > > > +To describe supported/enabled hash types device and driver use following definitions:
> > > > +
> > > > +Hash types applicable for IPv4 packets and for IPv6 packets without extension headers
> > > > +\begin{lstlisting}
> > > > +#define VIRTIO_NET_RSS_HASH_TYPE_IP              (1 << 0)
> > > > +#define VIRTIO_NET_RSS_HASH_TYPE_TCP             (1 << 1)
> > > > +#define VIRTIO_NET_RSS_HASH_TYPE_UDP             (1 << 2)
> > > > +\end{lstlisting}
> > > > +Hash types applicable for IPv6 packets with extension headers
> > > > +\begin{lstlisting}
> > > > +#define VIRTIO_NET_RSS_HASH_TYPE_IP_EX           (1 << 4) (only for IPv6)
> > > > +#define VIRTIO_NET_RSS_HASH_TYPE_TCP_EX          (1 << 5) (only for IPv6)
> > > > +#define VIRTIO_NET_RSS_HASH_TYPE_UDP_EX          (1 << 6) (only for IPv6)
> > > > +\end{lstlisting}
> > > > +For exact meaning of VIRTIO_NET_RSS_HASH_TYPE_ flags see \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
> > > > +
> > > > +\subparagraph{Setting RSS parameters}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}
> > > > +
> > > > +Driver sends VIRTIO_NET_CTRL_RSS_SET_PARAMETERS command using following format for \field{command-specific-data}:
> > > > +\begin{lstlisting}
> > > > +struct virtio_net_rss_params {
> > > > +    u8 enable_rss;
> > > > +    u8 hash_key_length;
> > > > +    le16 hash_types_v4;  (bitmask of allowed hash types for IPv4 packets)
> > > > +    le16 hash_types_v6;  (bitmask of allowed hash types for IPv6 packets)
> > > > +    le16 indirection_table_length; (number of queue indices in indirection_table array)
> > > > +    le16 unclassified_queue; (queue to place unclassified packets in)
> > > > +    le16 indirection_table[indirection_table_length];
> > > > +    u8 hash_key_data[hash_key_length];
> > > > +};
> > > > +
> > > > +\end{lstlisting}
> > > > +Driver sets \field{enable_rss} to 0 to disable RSS processing by the device and use automatic steering mode instead.
> > >
> > > I am not sure it's a good idea to have !RSS mean automatic steering.
> > > We might have more options in the future.
> > > Why not just enable RSS when the command is given?
> >
> > The driver (Win for sure) can receive a command to disable RSS (also
> > during run-time).
> > In this case it makes sense that the device does its default steering.
>
> Right, but if a hardware device implements RSS it might not want
> to also support automatic steering.

I'd suggest to define 'automatic steering' as any default mechanism to
direct packets to RX queus that does not require explicit
configuration.
It is not guaranteed to be optimal.
(This is different from what the spec defines now)

> What I'm saying is that it's better to have driver exiplicitly set
> the steering, instead of enable/disable multiple options individually,
> and then have device figure out what is ment if multiple ones are
> enabled, or if no one is enabled.

What I suggest:
When the driver successfully enabled MQ, the device works with its
default steering.
If the driver enables RSS, the device works with configured settings.
If the driver disables RSS, the device works with again with its
default steering.
If in future we define other steering mechanism, we'll need to state
that enabling other (future) steering effectively discards any
previously set steering settings.

Any objections?
Do you want to state that this is impossible to switch back to default steering?
Do you want to add the requirement that the driver must not disable
RSS if it did not enable it?

>
> > If in future we have some additional options this is responsibility of
> > the driver to configure the steering accordingly.
> >
> > >
> > > ATM we don't have a command to force a single queue mode.
> > > Maybe we want a feature bit and a command for that.
> >
> > According to the spec (5.1.6.5.5):
> > Multiqueue is disabled by setting virtqueue_pairs to 1 (this is the
> > default) and waiting for the device to use the command buffer.
>
> right, so how does one disable RSS?

Can you please clarify the question?
If due to any reason the device uses single queue, it does not need
any steering.

>
> > >
> > > > +
> > > > +Driver sets \field{enable_rss} to 1 to enable RSS processing by the device using provided parameters.
> > > > +
> > > > +\subparagraph{RSS hash types}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
> > > > +
> > > > +The device calculates hash on IPv4 packets according to the field \field{hash_types_v4} of virtio_net_rss_params structure as follows:
> > > > +\begin{itemize}
> > > > +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCP header, the hash is calculated over following fields:
> > > > +\begin{itemize}
> > > > +\item Source IP address
> > > > +\item Destination IP address
> > > > +\item Source TCP port
> > > > +\item Destination TCP port
> > > > +\end{itemize}
> > > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDP header, the hash is calculated over following fields:
> > > > +\begin{itemize}
> > > > +\item Source IP address
> > > > +\item Destination IP address
> > > > +\item Source UDP port
> > > > +\item Destination UDP port
> > > > +\end{itemize}
> > > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
> > > > +\begin{itemize}
> > > > +\item Source IP address
> > > > +\item Destination IP address
> > > > +\end{itemize}
> > > > +\item Else the device does not calculate the hash
> > > > +\end{itemize}
> > > > +
> > > > +
> > > > +The device calculates hash on IPv6 packets without extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
> > > > +\begin{itemize}
> > > > +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCPv6 header, the hash is calculated over following fields:
> > > > +\begin{itemize}
> > > > +\item Source IPv6 address
> > > > +\item Destination IPv6 address
> > > > +\item Source TCP port
> > > > +\item Destination TCP port
> > > > +\end{itemize}
> > > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDPv6 header, the hash is calculated over following fields:
> > > > +\begin{itemize}
> > > > +\item Source IPv6 address
> > > > +\item Destination IPv6 address
> > > > +\item Source UDP port
> > > > +\item Destination UDP port
> > > > +\end{itemize}
> > > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
> > > > +\begin{itemize}
> > > > +\item Source IPv6 address
> > > > +\item Destination IPv6 address
> > > > +\end{itemize}
> > > > +\item Else the device does not calculate the hash
> > > > +\end{itemize}
> > > > +
> > > > +
> > > > +The device calculates hash on IPv6 packets with extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
> > > > +\begin{itemize}
> > > > +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP_EX is set and the packet has TCPv6 header, the hash is calculated over following fields:
> > > > +\begin{itemize}
> > > > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> > > > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> > > > +\item Source TCP port
> > > > +\item Destination TCP port
> > > > +\end{itemize}
> > > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP_EX is set and the packet has UDPv6 header, the hash is calculated over following fields:
> > > > +\begin{itemize}
> > > > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> > > > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> > > > +\item Source UDP port
> > > > +\item Destination UDP port
> > > > +\end{itemize}
> > > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP_EX is set, the hash is calculated over following fields:
> > > > +\begin{itemize}
> > > > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> > > > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> > > > +\end{itemize}
> > > > +\item Else skip IPv6 extension headers and calculate the hash as defined above for IPv6 packet without extension headers
> > > > +\end{itemize}
> > > > +
> > > > +
> > > > +\drivernormative{\subparagraph}{Using RSS commands}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Using RSS commands}
> > > > +
> > > > +A driver MUST NOT use RSS commands if the feature VIRTIO_NET_F_RSS has not been negotiated.
> > > > +
> > > > +A driver MUST NOT set RSS parameters before it successfully enabled operation with multiple queues.
> > > > +
> > > > +A driver MUST fill \field{indirection_table} array only with indices of enabled queues.
> > > > +
> > > > +An \field{indirection_table_length} MUST be power of two.
> > > > +
> > > > +A driver MUST NOT set any VIRTIO_NET_RSS_HASH_TYPE_ flags that are not supported by device.
> > > > +
> > > > +A driver MUST NOT set any bits below for \field{hash_types_v4}:
> > > > +\begin{itemize}
> > > > +\item VIRTIO_NET_RSS_HASH_TYPE_IP_EX
> > > > +\item VIRTIO_NET_RSS_HASH_TYPE_TCP_EX
> > > > +\item VIRTIO_NET_RSS_HASH_TYPE_UDP_EX
> > > > +\end{itemize}
> > > > +
> > > > +\devicenormative{\subparagraph}{RSS processing}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
> > > > +If the device reports support for VIRTIO_NET_F_RSS it MUST support keys of at least 40 bytes and indirection table of at least 128 entries.
> > > > +
> > > > +The device MUST determine destination queue for network packet as follows:
> > > > +\begin{itemize}
> > > > +\item Calculate hash of the packet as defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
> > > > +\item If the device did not calculate the hash for specific packet, the device directs the packet to the queue specified by \field{unclassified_queue} of virtio_net_rss_params structure
> > > > +\item Apply mask of (indirection_table_length - 1) to the calculated hash and use the result as the index in the indirection table to get 0-based number of destination receive queue
> > > > +\end{itemize}
> > > >
> > > >  \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
> > > >  Types / Network Device / Legacy Interface: Framing Requirements}
> > > > --
> > > > 2.17.2
> > > >
> > > >
> > > > This publicly archived list offers a means to provide input to the
> > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > >
> > > > In order to verify user consent to the Feedback License terms and
> > > > to minimize spam in the list archive, subscription is required
> > > > before posting.
> > > >
> > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > Join OASIS: https://www.oasis-open.org/join/
> > >
> > >
> > > This publicly archived list offers a means to provide input to the
> > > OASIS Virtual I/O Device (VIRTIO) TC.
> > >
> > > In order to verify user consent to the Feedback License terms and
> > > to minimize spam in the list archive, subscription is required
> > > before posting.
> > >
> > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > List help: virtio-comment-help@lists.oasis-open.org
> > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > Committee: https://www.oasis-open.org/committees/virtio/
> > > Join OASIS: https://www.oasis-open.org/join/
> > >
>

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] [PATCH] virtio-net: define support for receive-side scaling

[Michael S. Tsirkin]

On Sun, Oct 13, 2019 at 04:07:36PM +0300, Yuri Benditovich wrote:
> On Sun, Oct 13, 2019 at 3:03 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Sun, Oct 13, 2019 at 01:27:59PM +0300, Yuri Benditovich wrote:
> > > > > +Driver sends VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES command to query device capabilities related to RSS.
> > > > > +Device responds the command using following format for \field{command-specific-data}:
> > > > > +\begin{lstlisting}
> > > > > +struct virtio_net_rss_capabilities {
> > > > > +    le16 hash_types_v4;  (bitmask of supported hash types for IPv4 packets)
> > > > > +    le16 hash_types_v6;  (bitmask of supported hash types for IPv6 packets)
> > > >
> > > >
> > > > I think hash types should move into feature bits.
> > > > There are 8 of them overall, we are not short on bits.
> > > >
> > > At the moment we have almost 30 feature bits for network devices (the
> > > range 0..23 defined for specific devices is fully used)
> > > The range 38..63 defined for futher extensions is partially used
> > > (there is also some feature bits that aren't documented yet).
> > > I think we do not have too much unused feature bits
> >
> > That's not true.  Used to be a problem pre 1.0 when
> > pci had a dedicated dword register. Since 1.0 there's
> > a selector which gives you any number of dwords.
> >
> > You can start using 64..95 immediately once
> > we run out of the 32..63 range.
> >
> 
> Good to know. But this requires support for such features in QEMU.
> Currently virtio-net supports only 64 bits for features (as well as
> all the drivers, if I'm not mistaken)

It's pretty easy though.


> >
> > > and I would prefer
> > > to indicate support for different types of hashes via dedicated
> > > command.
> >
> > Big problem with using extra commands for discovery is that they break
> > passthrough. Tricks like VDPA can intercept and tweak feature
> > bits easily, and to a lesser extent config space accesses.
> > Intercepting vq commands is very tricky.
> >
> 
> RSS requires 'SET' command anyway.

Yes but that does not have to be trapped.
The problem is with feature discovery.
SET is not part of that.


> >
> > > There are also other RSS-related capabilities that the driver should
> > > know (see below)
> > >
> > > > Or if you prefer, we can add a full 32 bit mask
> > > > for device specific things.
> > > >
> > > >
> > > >
> > > > > +    le16 max_indirection_table_len; (maximal number of queue indices in indirection table)
> > > > > +    u8 max_key_length; (maximal length of RSS key in bytes)
> > > >
> > > >
> > > > I would put the max values in the config space.
> > >
> > > Indeed, we can extend the config layout of virtio_net device
> > > (virtio_net_config) and discard VIRTIO_NET_CTRL_RSS_QUERY_CAPABILITIES
> > > command.
> > >
> > > Unfortunately the definition of config space (virtio_net_config) is
> > > not synchronized between the spec and kernel headers.
> > > In kernel/qemu headers (used also by Win drivers) it was extended to
> > > accomodate speed and duplex and this definition is not present in the
> > > spec.
> > > Although there was some discussion about respective virtio_net feature
> > > I do not see any spec patch related to this change.
> > >
> > > Do you prefer to add 'reserved' fields for speed/duplex and then add
> > > fields for RSS related capabilities (max_key_size,
> > > max_indirection_table_len)?
> >
> > I'd rather we added the fields, with minimal documentation for now.
> >
> > > In this case also bitmask of supported hashes can be placed here, I think.
> > >
> > > >
> > > > > +};
> > > > > +\end{lstlisting}
> > > > > +To describe supported/enabled hash types device and driver use following definitions:
> > > > > +
> > > > > +Hash types applicable for IPv4 packets and for IPv6 packets without extension headers
> > > > > +\begin{lstlisting}
> > > > > +#define VIRTIO_NET_RSS_HASH_TYPE_IP              (1 << 0)
> > > > > +#define VIRTIO_NET_RSS_HASH_TYPE_TCP             (1 << 1)
> > > > > +#define VIRTIO_NET_RSS_HASH_TYPE_UDP             (1 << 2)
> > > > > +\end{lstlisting}
> > > > > +Hash types applicable for IPv6 packets with extension headers
> > > > > +\begin{lstlisting}
> > > > > +#define VIRTIO_NET_RSS_HASH_TYPE_IP_EX           (1 << 4) (only for IPv6)
> > > > > +#define VIRTIO_NET_RSS_HASH_TYPE_TCP_EX          (1 << 5) (only for IPv6)
> > > > > +#define VIRTIO_NET_RSS_HASH_TYPE_UDP_EX          (1 << 6) (only for IPv6)
> > > > > +\end{lstlisting}
> > > > > +For exact meaning of VIRTIO_NET_RSS_HASH_TYPE_ flags see \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}.
> > > > > +
> > > > > +\subparagraph{Setting RSS parameters}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Setting RSS parameters}
> > > > > +
> > > > > +Driver sends VIRTIO_NET_CTRL_RSS_SET_PARAMETERS command using following format for \field{command-specific-data}:
> > > > > +\begin{lstlisting}
> > > > > +struct virtio_net_rss_params {
> > > > > +    u8 enable_rss;
> > > > > +    u8 hash_key_length;
> > > > > +    le16 hash_types_v4;  (bitmask of allowed hash types for IPv4 packets)
> > > > > +    le16 hash_types_v6;  (bitmask of allowed hash types for IPv6 packets)
> > > > > +    le16 indirection_table_length; (number of queue indices in indirection_table array)
> > > > > +    le16 unclassified_queue; (queue to place unclassified packets in)
> > > > > +    le16 indirection_table[indirection_table_length];
> > > > > +    u8 hash_key_data[hash_key_length];
> > > > > +};
> > > > > +
> > > > > +\end{lstlisting}
> > > > > +Driver sets \field{enable_rss} to 0 to disable RSS processing by the device and use automatic steering mode instead.
> > > >
> > > > I am not sure it's a good idea to have !RSS mean automatic steering.
> > > > We might have more options in the future.
> > > > Why not just enable RSS when the command is given?
> > >
> > > The driver (Win for sure) can receive a command to disable RSS (also
> > > during run-time).
> > > In this case it makes sense that the device does its default steering.
> >
> > Right, but if a hardware device implements RSS it might not want
> > to also support automatic steering.
> 
> I'd suggest to define 'automatic steering' as any default mechanism to
> direct packets to RX queus that does not require explicit
> configuration.
> It is not guaranteed to be optimal.
> (This is different from what the spec defines now)

Is it really useful to have the device send packets
to queues at random?

The bahavior right now is at least well defined, even
though it's best effort.

Is there a motivation to switch existing RFS to
"send whereever device wants to" mode?
If yes pls include the motivation in the proposal.


> > What I'm saying is that it's better to have driver exiplicitly set
> > the steering, instead of enable/disable multiple options individually,
> > and then have device figure out what is ment if multiple ones are
> > enabled, or if no one is enabled.
> 
> What I suggest:
> When the driver successfully enabled MQ, the device works with its
> default steering.
> If the driver enables RSS, the device works with configured settings.
> If the driver disables RSS, the device works with again with its
> default steering.
> If in future we define other steering mechanism, we'll need to state
> that enabling other (future) steering effectively discards any
> previously set steering settings.
> 
> Any objections?

Then what happens when we add a 3rd mode?
Way I see it, at boot device uses a single pair of queues.

If you want automatic RFS, you use a command to do that
(right now it's name is VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET).

If you want a single queue, you again use
VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET with a value of 1.

Why don't we add a new command to enable RSS?

Last command simply takes precedence.

No tricks, no device default steering, device does not need
to maintain state.




> Do you want to state that this is impossible to switch back to default steering?
> Do you want to add the requirement that the driver must not disable
> RSS if it did not enable it?

What I was asking is whether we really should make RSS depend on MQ?
I can see devices with RSS support but with no automatic steering.
No? We could allow VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET with value of 1 only
when RSS && !MQ, for the sole purpose of disabling RSS.
Or we could add a dedicated command that disables RSS and auto RFS.


> >
> > > If in future we have some additional options this is responsibility of
> > > the driver to configure the steering accordingly.
> > >
> > > >
> > > > ATM we don't have a command to force a single queue mode.
> > > > Maybe we want a feature bit and a command for that.
> > >
> > > According to the spec (5.1.6.5.5):
> > > Multiqueue is disabled by setting virtqueue_pairs to 1 (this is the
> > > default) and waiting for the device to use the command buffer.
> >
> > right, so how does one disable RSS?
> 
> Can you please clarify the question?
> If due to any reason the device uses single queue, it does not need
> any steering.

Right. So do we call VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET with a value of 1
to get a single queue?


> >
> > > >
> > > > > +
> > > > > +Driver sets \field{enable_rss} to 1 to enable RSS processing by the device using provided parameters.
> > > > > +
> > > > > +\subparagraph{RSS hash types}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
> > > > > +
> > > > > +The device calculates hash on IPv4 packets according to the field \field{hash_types_v4} of virtio_net_rss_params structure as follows:
> > > > > +\begin{itemize}
> > > > > +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCP header, the hash is calculated over following fields:
> > > > > +\begin{itemize}
> > > > > +\item Source IP address
> > > > > +\item Destination IP address
> > > > > +\item Source TCP port
> > > > > +\item Destination TCP port
> > > > > +\end{itemize}
> > > > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDP header, the hash is calculated over following fields:
> > > > > +\begin{itemize}
> > > > > +\item Source IP address
> > > > > +\item Destination IP address
> > > > > +\item Source UDP port
> > > > > +\item Destination UDP port
> > > > > +\end{itemize}
> > > > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
> > > > > +\begin{itemize}
> > > > > +\item Source IP address
> > > > > +\item Destination IP address
> > > > > +\end{itemize}
> > > > > +\item Else the device does not calculate the hash
> > > > > +\end{itemize}
> > > > > +
> > > > > +
> > > > > +The device calculates hash on IPv6 packets without extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
> > > > > +\begin{itemize}
> > > > > +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP is set and the packet has TCPv6 header, the hash is calculated over following fields:
> > > > > +\begin{itemize}
> > > > > +\item Source IPv6 address
> > > > > +\item Destination IPv6 address
> > > > > +\item Source TCP port
> > > > > +\item Destination TCP port
> > > > > +\end{itemize}
> > > > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP is set and the packet has UDPv6 header, the hash is calculated over following fields:
> > > > > +\begin{itemize}
> > > > > +\item Source IPv6 address
> > > > > +\item Destination IPv6 address
> > > > > +\item Source UDP port
> > > > > +\item Destination UDP port
> > > > > +\end{itemize}
> > > > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP is set, the hash is calculated over following fields:
> > > > > +\begin{itemize}
> > > > > +\item Source IPv6 address
> > > > > +\item Destination IPv6 address
> > > > > +\end{itemize}
> > > > > +\item Else the device does not calculate the hash
> > > > > +\end{itemize}
> > > > > +
> > > > > +
> > > > > +The device calculates hash on IPv6 packets with extension headers according to the field \field{hash_types_v6} of virtio_net_rss_params structure as follows:
> > > > > +\begin{itemize}
> > > > > +\item If VIRTIO_NET_RSS_HASH_TYPE_TCP_EX is set and the packet has TCPv6 header, the hash is calculated over following fields:
> > > > > +\begin{itemize}
> > > > > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> > > > > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> > > > > +\item Source TCP port
> > > > > +\item Destination TCP port
> > > > > +\end{itemize}
> > > > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_UDP_EX is set and the packet has UDPv6 header, the hash is calculated over following fields:
> > > > > +\begin{itemize}
> > > > > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> > > > > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> > > > > +\item Source UDP port
> > > > > +\item Destination UDP port
> > > > > +\end{itemize}
> > > > > +\item Else if VIRTIO_NET_RSS_HASH_TYPE_IP_EX is set, the hash is calculated over following fields:
> > > > > +\begin{itemize}
> > > > > +\item Home address from the home address option in the IPv6 destination options header. If the extension header is not present, use the Source IPv6 address.
> > > > > +\item IPv6 address that is contained in the Routing-Header-Type-2 from the associated extension header. If the extension header is not present, use the Destination IPv6 address.
> > > > > +\end{itemize}
> > > > > +\item Else skip IPv6 extension headers and calculate the hash as defined above for IPv6 packet without extension headers
> > > > > +\end{itemize}
> > > > > +
> > > > > +
> > > > > +\drivernormative{\subparagraph}{Using RSS commands}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / Using RSS commands}
> > > > > +
> > > > > +A driver MUST NOT use RSS commands if the feature VIRTIO_NET_F_RSS has not been negotiated.
> > > > > +
> > > > > +A driver MUST NOT set RSS parameters before it successfully enabled operation with multiple queues.
> > > > > +
> > > > > +A driver MUST fill \field{indirection_table} array only with indices of enabled queues.
> > > > > +
> > > > > +An \field{indirection_table_length} MUST be power of two.
> > > > > +
> > > > > +A driver MUST NOT set any VIRTIO_NET_RSS_HASH_TYPE_ flags that are not supported by device.
> > > > > +
> > > > > +A driver MUST NOT set any bits below for \field{hash_types_v4}:
> > > > > +\begin{itemize}
> > > > > +\item VIRTIO_NET_RSS_HASH_TYPE_IP_EX
> > > > > +\item VIRTIO_NET_RSS_HASH_TYPE_TCP_EX
> > > > > +\item VIRTIO_NET_RSS_HASH_TYPE_UDP_EX
> > > > > +\end{itemize}
> > > > > +
> > > > > +\devicenormative{\subparagraph}{RSS processing}{Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
> > > > > +If the device reports support for VIRTIO_NET_F_RSS it MUST support keys of at least 40 bytes and indirection table of at least 128 entries.
> > > > > +
> > > > > +The device MUST determine destination queue for network packet as follows:
> > > > > +\begin{itemize}
> > > > > +\item Calculate hash of the packet as defined in \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS hash types}
> > > > > +\item If the device did not calculate the hash for specific packet, the device directs the packet to the queue specified by \field{unclassified_queue} of virtio_net_rss_params structure
> > > > > +\item Apply mask of (indirection_table_length - 1) to the calculated hash and use the result as the index in the indirection table to get 0-based number of destination receive queue
> > > > > +\end{itemize}
> > > > >
> > > > >  \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
> > > > >  Types / Network Device / Legacy Interface: Framing Requirements}
> > > > > --
> > > > > 2.17.2
> > > > >
> > > > >
> > > > > This publicly archived list offers a means to provide input to the
> > > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > > >
> > > > > In order to verify user consent to the Feedback License terms and
> > > > > to minimize spam in the list archive, subscription is required
> > > > > before posting.
> > > > >
> > > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > > Join OASIS: https://www.oasis-open.org/join/
> > > >
> > > >
> > > > This publicly archived list offers a means to provide input to the
> > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > >
> > > > In order to verify user consent to the Feedback License terms and
> > > > to minimize spam in the list archive, subscription is required
> > > > before posting.
> > > >
> > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > Join OASIS: https://www.oasis-open.org/join/
> > > >
> >


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/