[virtio-comment] [PATCH v6 0/5] virtio-net: Support flow filter for receive packets

[virtio-comment] [PATCH v6 0/5] virtio-net: Support flow filter for receive packets

[Parav Pandit]

Summary:
========
This series improves virtio net receive packet steering to forward/steer
packets to specific RQ.

This basic functionality will enable Linux ethtool steering, Accelerated
receive flow steering (ARFS) as starting point, and more use cases in
future.

Problem statement:
==================
Currently packet allow/drop interface has few limitations.

1. Driver cannot add or delete an individual entry for mac and vlan.
2. Driver cannot select mac+vlan combination for which
    to allow/drop packet.
3. Driver cannot not set other commonly used packet match fields
    such as IP header fields, TCP, UDP, SCP header fields.
4. Driver cannot steer specific packets based on the match
   fields to specific receiveq.
5. Driver do not have multiple or dedicated virtqueues to
    perform flow filter requests in accelerated manner in
    the device.

Solution:
=========
Flow filter as a generic framework to overcome above limitations.

Overview:
=========
A flow filter defines the flow based on one or more match fields
of the packet, defines an action like drop/forward to RQ.

The flow filters are organized in flow filter groups so that their
processing can be ordered when multiple applications wants to use it.

Flow filters requests can be transported via control vq or dedicated
flow filter virtqueue so that it does not get intermixed with other
slow operations of cvq.

Flow filter requirements addressed by this series is worked by virtio
community at [1].

Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179

It uses updated control vq command format from [2].

[1] https://lists.oasis-open.org/archives/virtio-comment/202308/msg00263.html
[2] https://lists.oasis-open.org/archives/virtio-comment/202310/msg00047.html

Patch summary:
==============
patch-1 adds theory of operation description for flow filter
patch-2 adds device capabilities cvq commands
patch-3 adds group add/delete commands
patch-4 adds flow filter match key, action and requests to transport via vq
patch-5 adds device and driver requirements

Please review.

Changelog:
==========
v5->v6:
- pick next unique bit 65 as to avoid conflict with rss context feature
- fixed missing conformance links
- removed white spaces at end of line
v3->v5:
- removed left over dependencies of flow filter virtqueues
- removed partial sentence
v2->v3:
- removed dependency on dynamic queue infrastucture which is
  not yet ready
v1->v2:
- addressed comments from Satananda
- squashed with match fields definition patch of v1
- added length to the flexible array definition struct to benefit
  from future runtime length bound checkers listed in
  https://people.kernel.org/kees/bounded-flexible-arrays-in-c
- renamed value to key
- addressed comments from Satananda
- merged destination and action to one struct
- added vlan type match field
- kept space for types between l2, l3, l4 header match types
- renamed mask to mask_supported with shorter width
- made more fields reserved for future
- addressed comments from Heng
- grammar correction
- added field to indicate supported number of actions per flow
  filter match entry
- added missing documentation for max_flow_priorities_per_group
- fixed comments from Heng
- grammar corrections
- spelling corrections
- fixed spelling from initializaton to initialization
- added more requirements for multiple actions

v0->v1:
- addressed comments from Satananda
- added device requirement to return non zero value in fields_bmap
- added device requirement to not repeat filter type in response
- added driver requirement to order filter match field as it
  appears in the packet
- added device requirement to fail group delete command on existing
  flow entries
- added mask field in the type to indicate supported mask by device
  and also in later patch to use it to indicate mask on adding
  flow filter. As a result removed the mask_supported capability
  field

Parav Pandit (5):
  virtio-net: Add theory of operation for flow filter
  virtio-net: Add flow filter capabilities read commands
  virtio-net: Add flow filter group life cycle commands
  virtio-net: Add flow filter match entry, action and requests
  virtio-net: Add flow filter device and driver requirements

 device-types/net/description.tex        | 630 ++++++++++++++++++++++++
 device-types/net/device-conformance.tex |   1 +
 device-types/net/driver-conformance.tex |   1 +
 3 files changed, 632 insertions(+)

-- 
2.34.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] [PATCH v6 1/5] virtio-net: Add theory of operation for flow filter

[Parav Pandit]

Currently packet allow/drop interface has following limitations.

1. Driver can either select which MAC and VLANs to consider
for allowing/dropping packets, here, the driver has a
limitation that driver needs to supply full mac
table or full vlan table for each type. Driver cannot add or
delete an individual entry.

2. Driver cannot select mac+vlan combination for which
to allow/drop packet.

3. Driver cannot not set other commonly used packet match fields
such as IP header fields, TCP, UDP, SCP header fields.

4. Driver cannot steer specific packets based on the match
fields to specific receiveq.

5. Driver do not have multiple or dedicated virtqueues to
   perform flow filter requests in accelerated manner in
   the device.

Flow filter as a generic framework overcome above limitations.

As starting point it is useful to support at least two use cases.
a. ARFS
b. ethtool ntuple steering

In future it can be further extended for usecases such as
switching device, connection tracking or may be more.

The flow filter has following properties.

1. It is an extendible object that driver can add or delete.
2. It belongs to a flow filter group (group has priority).
3. Each flow filter is identified using a unique identifier(id),
   has priority, match fields, destination(rq) and action(allow/drop).
4. Flow filter has optionally mask too.

This patch adds theory of operation for flow filter functionality.

Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
Signed-off-by: Parav Pandit <parav@nvidia.com>

---
changelog:
v4->v5:
- to avoid feature bit overlap with rss context patch, pick next
  unique bit 65
v3->v4:
- removed flow filter virtqueue section as dyanmic queues are
  not supported currently
v2->v3:
- removed dependency on the dynamic queue creation as the
  infrastructure is not yet ready
v1->v2:
- fixed comments from Heng
- grammar corrections
- spelling corrections
---
 device-types/net/description.tex | 81 ++++++++++++++++++++++++++++++++
 1 file changed, 81 insertions(+)

diff --git a/device-types/net/description.tex b/device-types/net/description.tex
index aff5e08..30220b5 100644
--- a/device-types/net/description.tex
+++ b/device-types/net/description.tex
@@ -33,6 +33,9 @@ \subsection{Virtqueues}\label{sec:Device Types / Network Device / Virtqueues}
 controlq is optional; it only exists if VIRTIO_NET_F_CTRL_VQ is
 negotiated.
 
+The flow filter virtqueues are optional; it may exists only if VIRTIO_NET_F_FLOW_FILTER
+is negotiated and if the device reports such capability.
+
 \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits}
 
 \begin{description}
@@ -122,6 +125,8 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
     device with the same MAC address.
 
 \item[VIRTIO_NET_F_SPEED_DUPLEX(63)] Device reports speed and duplex.
+
+\item[VIRTIO_NET_F_FLOW_FILTER(65)] Device supports flow filter requests.
 \end{description}
 
 \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device / Feature bits / Feature bit requirements}
@@ -153,6 +158,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
 \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_VQ_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ.
 \item[VIRTIO_NET_F_HASH_TUNNEL] Requires VIRTIO_NET_F_CTRL_VQ along with VIRTIO_NET_F_RSS or VIRTIO_NET_F_HASH_REPORT.
+\item[VIRTIO_NET_F_FLOW_FILTER] Requires VIRTIO_NET_F_CTRL_VQ.
 \end{description}
 
 \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
@@ -1144,6 +1150,81 @@ \subsubsection{Processing of Incoming Packets}\label{sec:Device Types / Network
 #define VIRTIO_NET_HASH_REPORT_UDPv6_EX        9
 \end{lstlisting}
 
+\subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Operation / Flow Filter}
+
+To forward a received packet to a specific receiveq or to drop the packet based
+on one or more fields of the packet, the device supports flow filter
+functionality. The flow filter can match the packet for a flow, take an action
+such as forward the packet to the specific receiveq or drop the packet.
+For example, the driver can request the device to forward received packets
+which match to a specific source and destination IP addresses and
+TCP ports to a specific receiveq.
+
+Each flow filter consists of one or more match key, a flow filter priority,
+a flow filter identifier, an action to forward a packet to the destination
+or to drop the packet.
+
+The match fields also optionally consist of a match mask. When a mask is
+specified for the flow filter, first the packet fields are masked before
+matching with the fields of the flow filter.
+
+Each flow filter is independent of each other. The driver can add, replace
+and delete a flow filter in the device using a flow filter request.
+
+The device indicates the flow filter capabilities to the driver. These
+capabilities include various maximum device limits and
+supported packet match fields.
+
+The flow filters are grouped using a flow filter group. Each flow filter
+group has a priority. The device first applies the flow filters of the highest
+priority group to the received packet. If there is no match for the
+flow filters of such a group for the packet, the flow filters of the next
+priority group are applied, until there is a match for the packet from such
+a group or the last group has reached.
+
+The flow filter group can have one or more flow filters. Within a flow
+filter group, a packet may find a match to multiple flow filters. In such
+scenario, a flow filter with the highest priority is applied first to the
+packet, if there is no match, the next higher priority flow filter is applied.
+
+\paragraph{Packet Processing Order}\label{sec:sec:Device Types / Network Device / Device Operation / Flow Filter / Packet Processing Order}
+
+Whichever filtering and steering functionality is enabled, they are
+applied in the following order for the received packet:
+
+\begin{itemize}
+\item apply device configuration done using control virtqueue commands
+      VIRTIO_NET_CTRL_RX, VIRTIO_NET_CTRL_MAC and VIRTIO_NET_CTRL_VLAN.
+\item apply flow filter configuration done using flow filter requests.
+\item apply device configuration done using command
+      VIRTIO_NET_CTRL_MQ_RSS_CONFIG.
+\end{itemize}
+
+While processing a received packet, at any stage if the packet is dropped,
+the next level of processing is omitted.
+
+When a flow filter is matched for the packet, it also stops the processing
+of the packet for next stage.
+
+Few examples are:
+\begin{itemize}
+\item If the packet is dropped by the flow filter configuration, RSS
+      configuration is not applied to such a packet.
+\item If the packet is forwarded to a specific receiveq using flow filters,
+      RSS configuration is not applied to such a packet due to a match on the
+      flow filter request.
+\item If the packet is dropped due to VIRTIO_NET_CTRL_MAC configuration,
+      flow filters or RSS configuration is not applied to such a packet.
+\item If the packet does not find any match in any of the flow filter groups,
+      next level RSS device configuration is applied if its exists.
+\item If there are three flow filter groups configured as group_A, group_B
+      and group_C with respective priorities as 4, 5, and 6; flow filters of
+      group_C is applied first having highest group priority, if there is a match,
+      flow filters of group_B and group_A are skipped; if there is no match for
+      the flow filters in group_C, the flow filters of next level group_B are applied.
+\end{itemize}
+
+\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Setting Promiscuous Mode}%old label for latexdiff
 \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
 
 The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
-- 
2.34.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

The device responds flow filter capabilities using two commands.
One command indicates generic flow filter device limits such as
number of flow filters, number of flow filter groups, support or
multiple transports etc.

The second command indicates supported match types, and fields
of the packet.

Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
Signed-off-by: Parav Pandit <parav@nvidia.com>

---
changelog:
v2->v3:
- rebased on virtio-1.4 branch
- removed reference for flow filter virtqueue
v1->v2:
- addressed comments from Satananda
- added vlan type match field
- kept space for types between l2, l3, l4 header match types
- renamed mask to mask_supported with shorter width
- made more fields reserved for furture
- addressed comments from Heng
- grammar correction
- added field to indicate supported number of actions per flow
  filter match entry
- added missing documentation for max_flow_priorities_per_group
v0->v1:
- added mask field in the type to indicate supported mask by device
  and also in later patch to use it to indicate mask on adding
  flow filter. As a result removed the mask_supported capability
  field
---
 device-types/net/description.tex | 208 ++++++++++++++++++++++++++++++-
 1 file changed, 206 insertions(+), 2 deletions(-)

diff --git a/device-types/net/description.tex b/device-types/net/description.tex
index 30220b5..eccd8d6 100644
--- a/device-types/net/description.tex
+++ b/device-types/net/description.tex
@@ -1173,7 +1173,11 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
 
 The device indicates the flow filter capabilities to the driver. These
 capabilities include various maximum device limits and
-supported packet match fields.
+supported packet match fields. These control virtqueue
+commands are:
+\ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Capabilities Get}
+and
+\ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}.
 
 The flow filters are grouped using a flow filter group. Each flow filter
 group has a priority. The device first applies the flow filters of the highest
@@ -1224,7 +1228,136 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
       the flow filters in group_C, the flow filters of next level group_B are applied.
 \end{itemize}
 
-\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Setting Promiscuous Mode}%old label for latexdiff
+\paragraph{Match Types and Fields}\label{sec:Device Types / Network Device / Device Operation / Flow Filter / Match Types and Fields}
+
+\begin{lstlisting}
+struct virtio_net_ff_match_type_cap {
+        le16 type;
+        u8 mask_supported;
+        u8 reserved[5];
+        le64 fields_bmap;
+};
+\end{lstlisting}
+
+The \field{type} corresponds to following table:
+
+\begin{tabular}{|l|l|l|}
+\hline
+Type & Name & Description \\
+\hline \hline
+0   & VIRTIO_NET_FF_ETH_HDR & Ethernet header of the packet \\
+\hline
+0x1   & VIRTIO_NET_FF_VLAN_TAG_HDR & VLAN tag of the packet \\
+\hline
+0x200   & VIRTIO_NET_FF_IPV4_HDR & IPv4 header of the packet \\
+\hline
+0x300   & VIRTIO_NET_FF_IPV6_HDR & IPv6 header of the packet \\
+\hline
+0x400   & VIRTIO_NET_FF_TCP_HDR & TCP header of the packet \\
+\hline
+0x500   & VIRTIO_NET_FF_UDP_HDR & UDP header of the packet \\
+\hline
+other   & -    & reserved \\
+\hline
+\end{tabular}
+
+When the \field{mask_supported} is set, for the specific \field{type}, the
+device can perform masking packet fields with the mask supplied in the flow
+filter match entry.
+
+For each \field{type} the \field{fields_bmap} indicates supported fields
+of the packet header which can be matched.
+
+For the \field{type} of VIRTIO_NET_FF_ETH_HDR, header fields
+are represented by a bitmap in \field{fields_bmap} are following:
+
+\begin{tabular}{|l|l|l|}
+\hline
+Bit & Name & Description \\
+\hline \hline
+0   & VIRTIO_NET_FF_DST_MAC & Destination MAC address in the packet \\
+\hline
+1   & VIRTIO_NET_FF_SRC_MAC & Source MAC address in the packet \\
+\hline
+2   & VIRTIO_NET_FF_ETHER_TYPE & Ether type in the packet \\
+\hline
+other   & -    & reserved \\
+\hline
+\end{tabular}
+
+For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN tag fields
+are represented by a bitmap in \field{fields_bmap} are following:
+
+\begin{tabular}{|l|l|l|}
+\hline
+Bit & Name & Description \\
+\hline \hline
+0   & VIRTIO_NET_FF_VLAN_TAG_TCI & Vlan tag TCI 16-bit field \\
+\hline
+other   & -    & reserved \\
+\hline
+\end{tabular}
+
+For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, header fields
+are represented by a bitmap in \field{fields_bmap} are following:
+
+\begin{tabular}{|l|l|l|}
+\hline
+Bit & Name & Description \\
+\hline \hline
+0   & VIRTIO_NET_FF_SRC_IPV4 & Source IPV4 address in the packet \\
+\hline
+1   & VIRTIO_NET_FF_DST_IPV4 & Destination IPV4 address in the packet \\
+\hline
+other   & -    & reserved \\
+\hline
+\end{tabular}
+
+For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header fields
+are represented by a bitmap in \field{fields_bmap} are following:
+
+\begin{tabular}{|l|l|l|}
+\hline
+Bit & Name & Description \\
+\hline \hline
+0   & VIRTIO_NET_FF_SRC_IPV6 & Source IPV6 address in the packet \\
+\hline
+1   & VIRTIO_NET_FF_DST_IPV6 & Destination IPV6 address in the packet \\
+\hline
+other   & -    & reserved \\
+\hline
+\end{tabular}
+
+For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header fields
+are represented by a bitmap in \field{fields_bmap} are following:
+
+\begin{tabular}{|l|l|l|}
+\hline
+Bit & Name & Description \\
+\hline \hline
+0   & VIRTIO_NET_FF_SRC_TCP_PORT & Source TCP port in the packet \\
+\hline
+1   & VIRTIO_NET_FF_DST_TCP_PORT & Destination TCP port in the packet \\
+\hline
+other   & -    & reserved \\
+\hline
+\end{tabular}
+
+For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header fields
+are represented by a bitmap in \field{fields_bmap} are following:
+
+\begin{tabular}{|l|l|l|}
+\hline
+Bit & Name & Description \\
+\hline \hline
+0   & VIRTIO_NET_FF_SRC_UDP_PORT & Source UDP port in the packet \\
+\hline
+1   & VIRTIO_NET_FF_DST_UDP_PORT & Destination UDP port in the packet \\
+\hline
+other   & -    & reserved  \\
+\hline
+\end{tabular}
+
 \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
 
 The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
@@ -2392,6 +2525,77 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 of the driver's records. In such cases, the driver should allocate additional
 space for the \field{command-specific-result} buffer.
 
+\paragraph{Flow Filter}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter}
+
+If the VIRTIO_NET_F_FLOW_FILTER feature is negotiated,
+
+\begin{itemize}
+\item the driver can send commands VIRTIO_NET_CTRL_FF_CAP_GET and
+VIRTIO_NET_CTRL_FF_MATCH_CAP_GET to query the flow filter
+capabilities of the device.
+\end{itemize}
+
+\begin{lstlisting}
+#define VIRTIO_NET_CTRL_FF 7
+ #define VIRTIO_NET_CTRL_FF_CAP_GET 0
+ #define VIRTIO_NET_CTRL_FF_MATCH_CAP_GET 1
+\end{lstlisting}
+
+\subparagraph{Flow Filter Capabilities Get}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Capabilities Get}
+
+The command VIRTIO_NET_CTRL_FF_CAP_GET provides the flow filter device capabilities.
+
+\begin{lstlisting}
+struct virtio_net_ctrl_ff_caps {
+        le16 max_match_fields;
+        le16 max_groups; /* valid group id = max_groups - 1 */
+        le32 max_ff_per_group;
+        le32 max_ff; /* max flow_id in add/del = max_ff - 1 */
+        le16 max_actions;
+        u8 max_flow_priorities_per_group;
+};
+\end{lstlisting}
+
+The \field{max_groups} indicates total number of flow filter groups supported
+by the device whose group identifier can be any value in the range from 0 to
+\field{max_groups - 1}. The flow filter group can have any priority in range
+of 0 to \field{max_groups - 1}.
+
+The \field{max_ff_per_group} indicates maximum number of
+flow filter per flow filter group which can be added by the driver.
+
+The \field{max_ff} indicates maximum number of flow filters across
+all the flow groups which can be added by the driver.
+
+The \field{max_ff_priorities_per_group} indicates maximum priority value
+of a flow filter within a group. A flow filter within a group can have any
+priority in range of zero to \field{max_ff_priorities_per_group - 1}.
+
+The \field{max_match_fields} indicates maximum number of fields of the packet
+which can be matched by the device for a flow filter.
+
+The \field{max_actions} indicates maximum number of actions for a flow filter
+match entry can be supplied by the driver.
+
+The \field{max_flow_priorities_per_group} indicates maximum number of
+priorities supported by the device per flow filter group.
+
+\subparagraph{Flow Filter Match Capabilities Get}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}
+
+The command VIRTIO_NET_CTRL_FF_MATCH_CAP_GET indicates which fields
+from the packet can be matched.
+
+\begin{lstlisting}
+struct virtio_net_ctrl_ff_match_types {
+        le32 num_entries;
+        struct virtio_net_ff_match_type_cap types[];
+};
+\end{lstlisting}
+
+The \field{num_entries} indicates the length of an array \field{types}.
+Each array entry of \field{types} represents supported match fields of
+the packet by the device.
+
 \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
 Types / Network Device / Legacy Interface: Framing Requirements}
 
-- 
2.34.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] [PATCH v6 3/5] virtio-net: Add flow filter group life cycle commands

[Parav Pandit]

All the flow filters are managed in the flow filter group. The
device can have one or more flow filter groups. Each flow filter
group has its own priority. The group priority which
defines the packet processing order in the flow filter domain.

Add commands to add and delete the flow filter group.

Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
Signed-off-by: Parav Pandit <parav@nvidia.com>
Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
---
 device-types/net/description.tex | 36 ++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/device-types/net/description.tex b/device-types/net/description.tex
index eccd8d6..31c8c35 100644
--- a/device-types/net/description.tex
+++ b/device-types/net/description.tex
@@ -1190,6 +1190,11 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
 filter group, a packet may find a match to multiple flow filters. In such
 scenario, a flow filter with the highest priority is applied first to the
 packet, if there is no match, the next higher priority flow filter is applied.
+The driver adds and deletes the flow filter group using a control
+virtqueue commands \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Group Add}
+and
+\ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Group Delete}
+respectively.
 
 \paragraph{Packet Processing Order}\label{sec:sec:Device Types / Network Device / Device Operation / Flow Filter / Packet Processing Order}
 
@@ -2533,12 +2538,17 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 \item the driver can send commands VIRTIO_NET_CTRL_FF_CAP_GET and
 VIRTIO_NET_CTRL_FF_MATCH_CAP_GET to query the flow filter
 capabilities of the device.
+\item the driver can send commands VIRTIO_NET_CTRL_FF_GROUP_ADD
+and VIRTIO_NET_CTRL_FF_GROUP_DEL for adding and deleting the
+flow group.
 \end{itemize}
 
 \begin{lstlisting}
 #define VIRTIO_NET_CTRL_FF 7
  #define VIRTIO_NET_CTRL_FF_CAP_GET 0
  #define VIRTIO_NET_CTRL_FF_MATCH_CAP_GET 1
+ #define VIRTIO_NET_CTRL_FF_GROUP_ADD 2
+ #define VIRTIO_NET_CTRL_FF_GROUP_DEL 3
 \end{lstlisting}
 
 \subparagraph{Flow Filter Capabilities Get}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Capabilities Get}
@@ -2596,6 +2606,32 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 Each array entry of \field{types} represents supported match fields of
 the packet by the device.
 
+\subparagraph{Flow Filter Group Add}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Group Add}
+
+The command VIRTIO_NET_CTRL_FF_GROUP_ADD adds a new flow filter
+group with the supplied group identifier \field{id} with assigned
+priority \field{priority}.
+
+\begin{lstlisting}
+struct virtio_net_ctrl_ff_group_add {
+        le16 priority;	/* higher the value, higher priority */
+        le16 id;
+};
+\end{lstlisting}
+
+\subparagraph{Flow Filter Group Delete}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Group Delete}
+
+The command VIRTIO_NET_CTRL_FF_GROUP_DEL deletes the
+flow filter group that has been previously added using command
+VIRTIO_NET_CTRL_FF_GROUP_ADD. Flow filter group is
+identified using a group identifier \field{id}.
+
+\begin{lstlisting}
+struct virtio_net_ctrl_ff_group_del {
+        le16 id;
+};
+\end{lstlisting}
+
 \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
 Types / Network Device / Legacy Interface: Framing Requirements}
 
-- 
2.34.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] [PATCH v6 4/5] virtio-net: Add flow filter match entry, action and requests

[Parav Pandit]

Define flow filter match key for the defined types.

Currently it covers the most common filter types and value
of Ethernet header, IP addresses, TCP and UDP ports.

Define generic flow filter add and delete requests and its transport
using a control virtqueue command and flow filter virtqueue(s).

Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
Signed-off-by: Parav Pandit <parav@nvidia.com>

---
changelog:
v2->v3:
- removed references to flow filter virtqueues
- removed one partial sentence
- added text for delete request
- aligned request and opcode values to just say request in the defines
v1->v2:
- squashed with match fields definition patch of v1
- added length to the flexible array defintion struct to benefit
  from future runtime length bound checkers listed in
  https://people.kernel.org/kees/bounded-flexible-arrays-in-c
- renamed value to key
- addressed comments from Satananda
- merged destination and action to one struct
v0->v1:
- reworded add flow request text to consider optional mask
- replaced respond with set
- added mask flag to the type
---
 device-types/net/description.tex | 211 ++++++++++++++++++++++++++++++-
 1 file changed, 208 insertions(+), 3 deletions(-)

diff --git a/device-types/net/description.tex b/device-types/net/description.tex
index 31c8c35..c1a08f8 100644
--- a/device-types/net/description.tex
+++ b/device-types/net/description.tex
@@ -33,9 +33,6 @@ \subsection{Virtqueues}\label{sec:Device Types / Network Device / Virtqueues}
 controlq is optional; it only exists if VIRTIO_NET_F_CTRL_VQ is
 negotiated.
 
-The flow filter virtqueues are optional; it may exists only if VIRTIO_NET_F_FLOW_FILTER
-is negotiated and if the device reports such capability.
-
 \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits}
 
 \begin{description}
@@ -1244,6 +1241,50 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
 };
 \end{lstlisting}
 
+\begin{lstlisting}
+struct virtio_ff_match_entry {
+        le16 type;
+        u8 mask_present;
+        u8 key_mask_len; /* sum of length of fields key and mask */
+        le64 fields_bmap;
+        u8 key[];
+        u8 mask[]; /* optional, only present when mask_present is set to 1 */
+};
+
+struct virtio_ff_match {
+       u8 num_entries; /* indicates number of valid entries */
+       u8 reserved[7];
+       struct virtio_ff_match_entry entries[];
+};
+
+#define VIRTIO_NET_FF_DEST_TYPE_RQ 0
+
+struct virtio_ff_action_forward {
+        u8 dest_type;
+        u8 reserved[3];
+        union {
+                le16 vq_index;
+                le32 reserved1;
+        };
+};
+
+#define VIRTIO_NET_FF_ACTION_DROP 0
+#define VIRTIO_NET_FF_ACTION_FORWARD 1
+
+struct virtio_ff_action_entry {
+        u8 action;
+        u8 len; /* indicates the length of value in bytes */
+        u8 value[];
+};
+
+struct virtio_ff_action {
+        u8 num_entries; /* indicates number of valid entries */
+        u8 reserved[7];
+        struct virtio_ff_action_entry entries[];
+};
+
+\end{lstlisting}
+
 The \field{type} corresponds to following table:
 
 \begin{tabular}{|l|l|l|}
@@ -1290,6 +1331,21 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
 \hline
 \end{tabular}
 
+For the \field{type} of VIRTIO_NET_FF_ETH_HDR, the match entry
+\field{key} and \field{mask} are in following format:
+
+\begin{lstlisting}
+struct virtio_net_ff_match_eth_hdr {
+        u8 dmac[6];
+        u8 smac[6];
+        le16 ether_type;
+};
+\end{lstlisting}
+
+The \field{dmac} is valid when VIRTIO_NET_FF_DST_MAC is set.
+The \field{smac} is valid when VIRTIO_NET_FF_SRC_MAC is set.
+The \field{ether_type} is valid when VIRTIO_NET_FF_ETHER_TYPE is set.
+
 For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN tag fields
 are represented by a bitmap in \field{fields_bmap} are following:
 
@@ -1318,6 +1374,20 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
 \hline
 \end{tabular}
 
+For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, match entry
+\field{key} and \field{mask} are in following format:
+
+\begin{lstlisting}
+struct virtio_net_ff_match_ipv4_hdr {
+        le32 reserved[3];
+        le32 sip;
+        le32 dip;
+};
+\end{lstlisting}
+
+The \field{sip} is valid when VIRTIO_NET_FF_SRC_IPV4 is set.
+The \field{dip} is valid when VIRTIO_NET_FF_DST_IPV4 is set.
+
 For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header fields
 are represented by a bitmap in \field{fields_bmap} are following:
 
@@ -1333,6 +1403,20 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
 \hline
 \end{tabular}
 
+For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, match entry
+\field{key} and \field{mask} are in following format:
+
+\begin{lstlisting}
+struct virtio_net_ff_match_ipv6_hdr {
+        le32 reserved[2];
+        u8 sip[16];
+        u8 dip[16];
+};
+\end{lstlisting}
+
+The \field{sip} is valid when VIRTIO_NET_FF_SRC_IPV6 is set.
+The \field{dip} is valid when VIRTIO_NET_FF_DST_IPV6 is set.
+
 For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header fields
 are represented by a bitmap in \field{fields_bmap} are following:
 
@@ -1348,6 +1432,20 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
 \hline
 \end{tabular}
 
+For the \field{type} of VIRTIO_NET_FF_TCP_HDR, match entry
+\field{key} and \field{mask} are in following format:
+
+\begin{lstlisting}
+struct virtio_ndr_ff_match_tcp_hdr {
+        le16 sport;
+        le16 dport;
+        le32 reserved[4];
+};
+\end{lstlisting}
+
+The \field{sport} is valid when VIRTIO_NET_FF_SRC_TCP_PORT is set.
+This \field{dport} is valid when VIRTIO_NET_FF_DST_TCP_PORT is set.
+
 For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header fields
 are represented by a bitmap in \field{fields_bmap} are following:
 
@@ -1363,6 +1461,88 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
 \hline
 \end{tabular}
 
+For the \field{type} of VIRTIO_NET_FF_UDP_HDR, match entry
+\field{key} and \field{mask} are in following format:
+
+\begin{lstlisting}
+struct virtio_ndr_ff_match_udp_hdr {
+        le16 sport;
+        le16 dport;
+        le32 reserved;
+};
+\end{lstlisting}
+
+The \field{sport} is valid when VIRTIO_NET_FF_SRC_UDP_PORT is set.
+This \field{dport} is valid when VIRTIO_NET_FF_DST_UDP_PORT is set.
+
+\paragraph{Flow Filter Request}
+\label{sec:Device Types / Network Device / Device Operation / Flow Filter / Flow Filter Request}
+
+Two flow filter requests are supported by the device.
+
+\begin{itemize}
+\item Add or replace a flow filter using a request \field{struct virtio_net_ff_add}.
+
+\item Delete an existing flow filter using a request \field{struct virtio_net_ff_del}.
+
+\end{itemize}
+
+\begin{lstlisting}
+#define VIRTIO_NET_FF_REQ_ADD 0
+#define VIRTIO_NET_FF_REQ_DEL 1
+
+struct virtio_net_ff_add {
+        u8 request; /* VIRTIO_NET_FF_REQ_ADD */
+        u8 priority;	/* higher the value, higher priority */
+        u16 group_id;
+        le32 id;
+        struct virtio_ff_match match;
+        struct virtio_ff_action action;
+};
+
+struct virtio_net_ff_del {
+        u8 request;  /* VIRTIO_NET_FF_REQ_DEL */
+        u8 reserved[3];
+        le32 id;
+};
+
+struct virtio_net_ff_result {
+        le16 status;
+};
+
+#define VIRTIO_NET_FF_RESULT_OK 0
+#define VIRTIO_NET_FF_RESULT_ERR 1
+
+struct virtio_net_ff_req {
+        /* Device-readable part */
+        union {
+                struct virtio_net_ff_add add;
+                struct virtio_net_ff_del del;
+        };
+        /* Device-writable part */
+        struct virtio_net_ff_result result;
+};
+\end{lstlisting}
+
+When adding a flow filter entry using request \field{struct virtio_net_ff_add},
+the \field{match.match_entries} indidates number of valid array entries \field{match.entries}.
+For each of the valid entry in \field{match.entries}, the field \field{type}
+and \field{key} are in the format described in
+\ref{sec:Device Types / Network Device / Device Operation / Flow Filter / Match Types and Fields}.
+When the \field{mask_present} is set, the field \field{mask} is present and it has
+exact same format as \field{key}.
+
+The field \field{key_mask_len} represents the total length of fields
+\field{key} and \field{mask}.
+
+Previously added flow filter entry can be deleted by the driver using
+VIRTIO_NET_FF_REQ_DEL request.
+
+When the device completes the request, \field{status} is updated
+by the device; when the request is successful, \field{status} is
+set to VIRTIO_NET_FF_RESULT_OK, on error, \field{status} is
+set to VIRTIO_NET_FF_RESULT_ERR.
+
 \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
 
 The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
@@ -2541,6 +2721,8 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 \item the driver can send commands VIRTIO_NET_CTRL_FF_GROUP_ADD
 and VIRTIO_NET_CTRL_FF_GROUP_DEL for adding and deleting the
 flow group.
+\item the driver can send command VIRTIO_NET_CTRL_FF_REQ to
+add or delete flow filter.
 \end{itemize}
 
 \begin{lstlisting}
@@ -2549,6 +2731,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
  #define VIRTIO_NET_CTRL_FF_MATCH_CAP_GET 1
  #define VIRTIO_NET_CTRL_FF_GROUP_ADD 2
  #define VIRTIO_NET_CTRL_FF_GROUP_DEL 3
+ #define VIRTIO_NET_CTRL_FF_REQ 4
 \end{lstlisting}
 
 \subparagraph{Flow Filter Capabilities Get}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Capabilities Get}
@@ -2632,6 +2815,28 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 };
 \end{lstlisting}
 
+\subparagraph{Flow Filter Requests}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Requests}
+
+The flow filter requests are transported using command
+VIRTIO_NET_CTRL_FF_REQ command.
+
+\begin{lstlisting}
+struct virtio_net_ctrl_ff_req {
+        union {
+                struct virtio_net_ff_add add;
+                struct virtio_net_ff_del del;
+        };
+};
+
+\end{lstlisting}
+
+The \field{command-specific-data} is in format of
+\field{struct virtio_net_ctrl_ff_req}.
+
+When the flow filter request command is successful, the
+\field{command-specific-result} is in format of
+\field{struct virtio_net_ff_result}.
+
 \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
 Types / Network Device / Legacy Interface: Framing Requirements}
 
-- 
2.34.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] [PATCH v6 5/5] virtio-net: Add flow filter device and driver requirements

[Parav Pandit]

The flow filter functionality consists of the following
four components.
Add driver and device requirements for it.

1. Device capabilities query for commands VIRTIO_NET_CTRL_FF_CAP_GET,
   VIRTIO_NET_CTRL_FF_MATCH_CAP_GET.
2. Flow filter group operation commands VIRTIO_NET_CTRL_FF_GROUP_ADD
   and VIRTIO_NET_CTRL_FF_GROUP_DEL.
3. Flow filter requests using command VIRTIO_NET_CTRL_FF_REQ and
   the structure virtio_net_ff_op for the flow filter virtqueue.

Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
Signed-off-by: Parav Pandit <parav@nvidia.com>
Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
---
changelog:
v5->v6:
- removed white spaces from end of line
- added missing conformance links
v2->v3:
- removed dependency on the dynamic queue creation as the
  infrastructure is not yet ready
v1->v2:
- fixed comments from Heng
- fixed spelling from initializaton to initialization
- added more requirements for multiple actions
v0->v1:
- addressed comments from Satananda
- added device requirement to return non zero value in fields_bmap
- added device requirement to not repeat filter type in response
- added driver requirement to order filter match field as it
  appears in the packet
- added device requirement to fail group delete command on existing
  flow entries
---
 device-types/net/description.tex        | 104 ++++++++++++++++++++++++
 device-types/net/device-conformance.tex |   1 +
 device-types/net/driver-conformance.tex |   1 +
 3 files changed, 106 insertions(+)

diff --git a/device-types/net/description.tex b/device-types/net/description.tex
index c1a08f8..b8583d4 100644
--- a/device-types/net/description.tex
+++ b/device-types/net/description.tex
@@ -2837,6 +2837,110 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 \field{command-specific-result} is in format of
 \field{struct virtio_net_ff_result}.
 
+\devicenormative{\subparagraph}{Flow Filter}{Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter}
+
+When the VIRTIO_NET_F_FLOW_FILTER is negotiated, the device MUST support
+VIRTIO_NET_CTRL_FF_CAP_GET, VIRTIO_NET_CTRL_FF_MATCH_CAP_GET, VIRTIO_NET_CTRL_FF_GROUP_ADD,
+VIRTIO_NET_CTRL_FF_GROUP_DEL and VIRTIO_NET_CTRL_FF_REQ commands.
+
+When the VIRTIO_NET_F_FLOW_FILTER is not negotiated, the device MUST respond
+with error VIRTIO_NET_ERR for
+VIRTIO_NET_CTRL_FF_CAP_GET, VIRTIO_NET_CTRL_FF_MATCH_CAP_GET, VIRTIO_NET_CTRL_FF_GROUP_ADD,
+VIRTIO_NET_CTRL_FF_GROUP_DEL and VIRTIO_NET_CTRL_FF_REQ commands.
+
+When the command VIRTIO_NET_CTRL_FF_CAP_GET completes successfully, the device
+MUST set non zero value for fields
+\field{max_groups}, \field{max_ff_per_group}, \field{max_ff},
+\field{max_match_fields}, \field{max_flow_priorities_per_group} and
+\field{max_actions}.
+
+When the command VIRTIO_NET_CTRL_FF_MATCH_CAP_GET completes successfully,
+\begin{itemize}
+\item the device MUST set non zero value for fields \field{num_entries}, \field{fields_bmap}
+and set corresponding number of valid entries.
+\item the device MUST NOT repeat \field{type} in the \field{types}.
+\end{itemize}
+
+The device MUST set VIRTIO_NET_ERROR for the command
+VIRTIO_NET_CTRL_FF_GROUP_ADD if there are existing flow filters for the
+supplied group \field{id} or for the supplied \field{priority}.
+
+The device MUST set VIRTIO_NET_ERROR for the command
+VIRTIO_NET_CTRL_FF_GROUP_DEL if the group identified with \field{id}
+does not exist in the device.
+
+The device MUST set VIRTIO_NET_ERROR for the command
+VIRTIO_NET_CTRL_FF_GROUP_DEL if the group identified with \field{id}
+has one or more flow filters present in the group.
+
+The device MUST fail the operation VIRTIO_NET_FF_OP_ADD if the
+\field{match} contains duplicate \field{type}.
+
+The device MUST fail the operation VIRTIO_NET_FF_OP_DEL if the
+requested flow filter of identifier \field{id} do not exist in the
+the device.
+
+The device MUST fail the operation VIRTIO_NET_FF_OP_ADD if the
+\field{vq_index} in the \field{dest} is outside of the range.
+
+When the flow filter forwards the packet to the \field{vq_index} and
+if the receiveq is reset, the device MUST drop such packets.
+
+When the flow filter \field{action} is VIRTIO_NET_FF_ACTION_DROP,
+the device MUST ignore rest of the fields of
+\field{struct virtio_flow_action_entry}.
+
+When the driver has added multiple flow filters with same \field{priority}
+and for a packet if multiple flow filters MAY match such that it MAY result
+in different \field{action} or different \field{dest}, the device
+MAY apply any of the matching flow filters.
+
+The device MUST follow received packet processing ordering chain as following:
+
+The device SHOULD set \field{device_status} to DEVICE_NEEDS_RESET when
+the driver has not set the flow filter transport mode to
+VIRTIO_NET_FF_TRANSPORT_FFVQ and if the driver enables the flow filter
+virtqueue.
+
+The device MUST apply the actions of \field{struct virtio_flow_action} in same
+order as it is supplied by the driver when \field{num_entries} is greater than 1.
+
+\begin{itemize}
+\item Device configuration done using control virtqueue commands VIRTIO_NET_CTRL_RX,
+   VIRTIO_NET_CTRL_MAC and VIRTIO_NET_CTRL_VLAN.
+\item Flow filters programmed using flow filters functionality.
+\item Device configuration done using VIRTIO_NET_CTRL_MQ_RSS_CONFIG command.
+\end{itemize}
+
+When the device drops the packet due to the configuration done using the control
+virtqueue commands VIRTIO_NET_CTRL_RX or VIRTIO_NET_CTRL_MAC or VIRTIO_NET_CTRL_VLAN,
+the device MUST stop processing this packet for flow filters processing.
+
+When the device matches the flow filter for the packet and if the match is successful,
+the filter processing chain MUST stop, i.e. next level of processing MUST not be done.
+
+When the device perform flow filter match operations and if the operation
+result did not have any match, the receive packet processing continues to next level,
+i.e. to apply configuration done using VIRTIO_NET_CTRL_MQ_RSS_CONFIG command.
+
+\drivernormative{\subparagraph}{Flow Filter}{Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter}
+
+The driver MUST NOT send any flow filters specific commands having class code
+of VIRTIO_NET_CTRL_FF when VIRTIO_NET_F_FLOW_FILTER is not negotiated.
+
+The driver SHOULD NOT add multiple flow filters with same \field{priority}
+in a flow filter group, with overlapping match values.
+
+The driver SHOULD use different priority for different flow filters
+if multiple of the flow filters MAY match for a packet.
+
+The driver SHOULD set the \field{type} in \field{match_entries} as that of
+the order appears in the packet.
+
+The driver MUST NOT set \field{num_entries} in \field{struct virtio_ff_action}
+to more than \field{max_actions} supplied by the device in the
+\field{virtio_net_ctrl_ff_caps}.
+
 \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
 Types / Network Device / Legacy Interface: Framing Requirements}
 
diff --git a/device-types/net/device-conformance.tex b/device-types/net/device-conformance.tex
index 52526e4..1228c27 100644
--- a/device-types/net/device-conformance.tex
+++ b/device-types/net/device-conformance.tex
@@ -16,4 +16,5 @@
 \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing}
 \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Inner Header Hash}
 \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Device Statistics}
+\item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter}
 \end{itemize}
diff --git a/device-types/net/driver-conformance.tex b/device-types/net/driver-conformance.tex
index c693c4f..9f9b63e 100644
--- a/device-types/net/driver-conformance.tex
+++ b/device-types/net/driver-conformance.tex
@@ -16,4 +16,5 @@
 \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing}
 \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Inner Header Hash}
 \item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Device Statistics}
+\item \ref{drivernormative:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter}
 \end{itemize}
-- 
2.34.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] RE: [PATCH v6 0/5] virtio-net: Support flow filter for receive packets

[Parav Pandit]

Hi Cornelia, Michael,


> -----Original Message-----
> From: Parav Pandit <parav@nvidia.com>
> Sent: Friday, November 10, 2023 6:09 PM
> To: virtio-comment@lists.oasis-open.org; mst@redhat.com;
> cohuck@redhat.com
> Cc: sburla@marvell.com; Shahaf Shuler <shahafs@nvidia.com>; si-
> wei.liu@oracle.com; xuanzhuo@linux.alibaba.com; Parav Pandit
> <parav@nvidia.com>
> Subject: [PATCH v6 0/5] virtio-net: Support flow filter for receive packets
> 
> Summary:
> ========
> This series improves virtio net receive packet steering to forward/steer packets
> to specific RQ.
> 
> This basic functionality will enable Linux ethtool steering, Accelerated receive
> flow steering (ARFS) as starting point, and more use cases in future.
> 
> Problem statement:
> ==================
> Currently packet allow/drop interface has few limitations.
> 
> 1. Driver cannot add or delete an individual entry for mac and vlan.
> 2. Driver cannot select mac+vlan combination for which
>     to allow/drop packet.
> 3. Driver cannot not set other commonly used packet match fields
>     such as IP header fields, TCP, UDP, SCP header fields.
> 4. Driver cannot steer specific packets based on the match
>    fields to specific receiveq.
> 5. Driver do not have multiple or dedicated virtqueues to
>     perform flow filter requests in accelerated manner in
>     the device.
> 
> Solution:
> =========
> Flow filter as a generic framework to overcome above limitations.
> 
> Overview:
> =========
> A flow filter defines the flow based on one or more match fields of the packet,
> defines an action like drop/forward to RQ.
> 
> The flow filters are organized in flow filter groups so that their processing can be
> ordered when multiple applications wants to use it.
> 
> Flow filters requests can be transported via control vq or dedicated flow filter
> virtqueue so that it does not get intermixed with other slow operations of cvq.
> 
> Flow filter requirements addressed by this series is worked by virtio community
> at [1].
> 
> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
> 

Now that this has been mature phase and on list for several weeks,
I updated the github to point to v6.

Can you please start the voting process?


> It uses updated control vq command format from [2].
> 
> [1] https://lists.oasis-open.org/archives/virtio-
> comment/202308/msg00263.html
> [2] https://lists.oasis-open.org/archives/virtio-
> comment/202310/msg00047.html
> 
> Patch summary:
> ==============
> patch-1 adds theory of operation description for flow filter
> patch-2 adds device capabilities cvq commands
> patch-3 adds group add/delete commands
> patch-4 adds flow filter match key, action and requests to transport via vq
> patch-5 adds device and driver requirements
> 
> Please review.
> 
> Changelog:
> ==========
> v5->v6:
> - pick next unique bit 65 as to avoid conflict with rss context feature
> - fixed missing conformance links
> - removed white spaces at end of line
> v3->v5:
> - removed left over dependencies of flow filter virtqueues
> - removed partial sentence
> v2->v3:
> - removed dependency on dynamic queue infrastucture which is
>   not yet ready
> v1->v2:
> - addressed comments from Satananda
> - squashed with match fields definition patch of v1
> - added length to the flexible array definition struct to benefit
>   from future runtime length bound checkers listed in
>   https://people.kernel.org/kees/bounded-flexible-arrays-in-c
> - renamed value to key
> - addressed comments from Satananda
> - merged destination and action to one struct
> - added vlan type match field
> - kept space for types between l2, l3, l4 header match types
> - renamed mask to mask_supported with shorter width
> - made more fields reserved for future
> - addressed comments from Heng
> - grammar correction
> - added field to indicate supported number of actions per flow
>   filter match entry
> - added missing documentation for max_flow_priorities_per_group
> - fixed comments from Heng
> - grammar corrections
> - spelling corrections
> - fixed spelling from initializaton to initialization
> - added more requirements for multiple actions
> 
> v0->v1:
> - addressed comments from Satananda
> - added device requirement to return non zero value in fields_bmap
> - added device requirement to not repeat filter type in response
> - added driver requirement to order filter match field as it
>   appears in the packet
> - added device requirement to fail group delete command on existing
>   flow entries
> - added mask field in the type to indicate supported mask by device
>   and also in later patch to use it to indicate mask on adding
>   flow filter. As a result removed the mask_supported capability
>   field
> 
> Parav Pandit (5):
>   virtio-net: Add theory of operation for flow filter
>   virtio-net: Add flow filter capabilities read commands
>   virtio-net: Add flow filter group life cycle commands
>   virtio-net: Add flow filter match entry, action and requests
>   virtio-net: Add flow filter device and driver requirements
> 
>  device-types/net/description.tex        | 630 ++++++++++++++++++++++++
>  device-types/net/device-conformance.tex |   1 +
>  device-types/net/driver-conformance.tex |   1 +
>  3 files changed, 632 insertions(+)
> 
> --
> 2.34.1


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] RE: [PATCH v6 0/5] virtio-net: Support flow filter for receive packets

[Parav Pandit]

Hi Cornelia, Michael,


> From: virtio-comment@lists.oasis-open.org <virtio-comment@lists.oasis-
> open.org> On Behalf Of Parav Pandit
> Sent: Monday, November 20, 2023 12:18 PM
> To: virtio-comment@lists.oasis-open.org; mst@redhat.com;
> cohuck@redhat.com
> 
> Hi Cornelia, Michael,
> 
> 
> > From: Parav Pandit <parav@nvidia.com>
> > Sent: Friday, November 10, 2023 6:09 PM
> > To: virtio-comment@lists.oasis-open.org; mst@redhat.com;
> > cohuck@redhat.com
> >
> > Summary:
> > ========
> > This series improves virtio net receive packet steering to
> > forward/steer packets to specific RQ.
> >
> > This basic functionality will enable Linux ethtool steering,
> > Accelerated receive flow steering (ARFS) as starting point, and more use
> cases in future.
> >
> > Problem statement:
> > ==================
> > Currently packet allow/drop interface has few limitations.
> >
> > 1. Driver cannot add or delete an individual entry for mac and vlan.
> > 2. Driver cannot select mac+vlan combination for which
> >     to allow/drop packet.
> > 3. Driver cannot not set other commonly used packet match fields
> >     such as IP header fields, TCP, UDP, SCP header fields.
> > 4. Driver cannot steer specific packets based on the match
> >    fields to specific receiveq.
> > 5. Driver do not have multiple or dedicated virtqueues to
> >     perform flow filter requests in accelerated manner in
> >     the device.
> >
> > Solution:
> > =========
> > Flow filter as a generic framework to overcome above limitations.
> >
> > Overview:
> > =========
> > A flow filter defines the flow based on one or more match fields of
> > the packet, defines an action like drop/forward to RQ.
> >
> > The flow filters are organized in flow filter groups so that their
> > processing can be ordered when multiple applications wants to use it.
> >
> > Flow filters requests can be transported via control vq or dedicated
> > flow filter virtqueue so that it does not get intermixed with other slow
> operations of cvq.
> >
> > Flow filter requirements addressed by this series is worked by virtio
> > community at [1].
> >
> > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
> >
> 
> Now that this has been mature phase and on list for several weeks, I updated
> the github to point to v6.
> 
> Can you please start the voting process?
>

Ping. This is overdue.

Can you please start the voting process?
 
> 
> > It uses updated control vq command format from [2].
> >
> > [1] https://lists.oasis-open.org/archives/virtio-
> > comment/202308/msg00263.html
> > [2] https://lists.oasis-open.org/archives/virtio-
> > comment/202310/msg00047.html
> >
> > Patch summary:
> > ==============
> > patch-1 adds theory of operation description for flow filter
> > patch-2 adds device capabilities cvq commands
> > patch-3 adds group add/delete commands
> > patch-4 adds flow filter match key, action and requests to transport
> > via vq
> > patch-5 adds device and driver requirements
> >
> > Please review.
> >
> > Changelog:
> > ==========
> > v5->v6:
> > - pick next unique bit 65 as to avoid conflict with rss context
> > feature
> > - fixed missing conformance links
> > - removed white spaces at end of line
> > v3->v5:
> > - removed left over dependencies of flow filter virtqueues
> > - removed partial sentence
> > v2->v3:
> > - removed dependency on dynamic queue infrastucture which is
> >   not yet ready
> > v1->v2:
> > - addressed comments from Satananda
> > - squashed with match fields definition patch of v1
> > - added length to the flexible array definition struct to benefit
> >   from future runtime length bound checkers listed in
> >   https://people.kernel.org/kees/bounded-flexible-arrays-in-c
> > - renamed value to key
> > - addressed comments from Satananda
> > - merged destination and action to one struct
> > - added vlan type match field
> > - kept space for types between l2, l3, l4 header match types
> > - renamed mask to mask_supported with shorter width
> > - made more fields reserved for future
> > - addressed comments from Heng
> > - grammar correction
> > - added field to indicate supported number of actions per flow
> >   filter match entry
> > - added missing documentation for max_flow_priorities_per_group
> > - fixed comments from Heng
> > - grammar corrections
> > - spelling corrections
> > - fixed spelling from initializaton to initialization
> > - added more requirements for multiple actions
> >
> > v0->v1:
> > - addressed comments from Satananda
> > - added device requirement to return non zero value in fields_bmap
> > - added device requirement to not repeat filter type in response
> > - added driver requirement to order filter match field as it
> >   appears in the packet
> > - added device requirement to fail group delete command on existing
> >   flow entries
> > - added mask field in the type to indicate supported mask by device
> >   and also in later patch to use it to indicate mask on adding
> >   flow filter. As a result removed the mask_supported capability
> >   field
> >
> > Parav Pandit (5):
> >   virtio-net: Add theory of operation for flow filter
> >   virtio-net: Add flow filter capabilities read commands
> >   virtio-net: Add flow filter group life cycle commands
> >   virtio-net: Add flow filter match entry, action and requests
> >   virtio-net: Add flow filter device and driver requirements
> >
> >  device-types/net/description.tex        | 630 ++++++++++++++++++++++++
> >  device-types/net/device-conformance.tex |   1 +
> >  device-types/net/driver-conformance.tex |   1 +
> >  3 files changed, 632 insertions(+)
> >
> > --
> > 2.34.1
> 
> 
> This publicly archived list offers a means to provide input to theOASIS Virtual
> I/O Device (VIRTIO) TC.In order to verify user consent to the Feedback
> License terms andto minimize spam in the list archive, subscription is
> requiredbefore posting.Subscribe: virtio-comment-subscribe@lists.oasis-
> open.orgUnsubscribe: virtio-comment-unsubscribe@lists.oasis-open.orgList
> help: virtio-comment-help@lists.oasis-open.orgList archive:
> https://lists.oasis-open.org/archives/virtio-comment/Feedback License:
> https://www.oasis-open.org/who/ipr/feedback_license.pdfList Guidelines:
> https://www.oasis-open.org/policies-guidelines/mailing-listsCommittee:
> https://www.oasis-open.org/committees/virtio/Join OASIS:
> https://www.oasis-open.org/join/


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] RE: [PATCH v6 0/5] virtio-net: Support flow filter for receive packets

[Cornelia Huck]

On Mon, Nov 20 2023, Parav Pandit <parav@nvidia.com> wrote:

> Hi Cornelia, Michael,
>
>
>> -----Original Message-----
>> From: Parav Pandit <parav@nvidia.com>
>> Sent: Friday, November 10, 2023 6:09 PM
>> To: virtio-comment@lists.oasis-open.org; mst@redhat.com;
>> cohuck@redhat.com
>> Cc: sburla@marvell.com; Shahaf Shuler <shahafs@nvidia.com>; si-
>> wei.liu@oracle.com; xuanzhuo@linux.alibaba.com; Parav Pandit
>> <parav@nvidia.com>
>> Subject: [PATCH v6 0/5] virtio-net: Support flow filter for receive packets
>> 
>> Summary:
>> ========
>> This series improves virtio net receive packet steering to forward/steer packets
>> to specific RQ.
>> 
>> This basic functionality will enable Linux ethtool steering, Accelerated receive
>> flow steering (ARFS) as starting point, and more use cases in future.
>> 
>> Problem statement:
>> ==================
>> Currently packet allow/drop interface has few limitations.
>> 
>> 1. Driver cannot add or delete an individual entry for mac and vlan.
>> 2. Driver cannot select mac+vlan combination for which
>>     to allow/drop packet.
>> 3. Driver cannot not set other commonly used packet match fields
>>     such as IP header fields, TCP, UDP, SCP header fields.
>> 4. Driver cannot steer specific packets based on the match
>>    fields to specific receiveq.
>> 5. Driver do not have multiple or dedicated virtqueues to
>>     perform flow filter requests in accelerated manner in
>>     the device.
>> 
>> Solution:
>> =========
>> Flow filter as a generic framework to overcome above limitations.
>> 
>> Overview:
>> =========
>> A flow filter defines the flow based on one or more match fields of the packet,
>> defines an action like drop/forward to RQ.
>> 
>> The flow filters are organized in flow filter groups so that their processing can be
>> ordered when multiple applications wants to use it.
>> 
>> Flow filters requests can be transported via control vq or dedicated flow filter
>> virtqueue so that it does not get intermixed with other slow operations of cvq.
>> 
>> Flow filter requirements addressed by this series is worked by virtio community
>> at [1].
>> 
>> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
>> 
>
> Now that this has been mature phase and on list for several weeks,
> I updated the github to point to v6.
>
> Can you please start the voting process?

I don't feel like I'm capable of judging the networking parts, but I
have a couple of wording suggestions (things like definite articles
etc.) Would you prefer a respin, or a vote now and an editorial patch on
top? I can either comment now, or start voting.


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] RE: [PATCH v6 0/5] virtio-net: Support flow filter for receive packets

[Parav Pandit]

Hi Cornelia,

> From: Cornelia Huck <cohuck@redhat.com>
> Sent: Wednesday, November 22, 2023 4:57 PM


> On Mon, Nov 20 2023, Parav Pandit <parav@nvidia.com> wrote:
> 
> > Hi Cornelia, Michael,
> >
> >
> >> -----Original Message-----
> >> From: Parav Pandit <parav@nvidia.com>
> >> Sent: Friday, November 10, 2023 6:09 PM
> >> To: virtio-comment@lists.oasis-open.org; mst@redhat.com;
> >> cohuck@redhat.com
> >> Cc: sburla@marvell.com; Shahaf Shuler <shahafs@nvidia.com>; si-
> >> wei.liu@oracle.com; xuanzhuo@linux.alibaba.com; Parav Pandit
> >> <parav@nvidia.com>
> >> Subject: [PATCH v6 0/5] virtio-net: Support flow filter for receive
> >> packets
> >>
> >> Summary:
> >> ========
> >> This series improves virtio net receive packet steering to
> >> forward/steer packets to specific RQ.
> >>
> >> This basic functionality will enable Linux ethtool steering,
> >> Accelerated receive flow steering (ARFS) as starting point, and more use
> cases in future.
> >>
> >> Problem statement:
> >> ==================
> >> Currently packet allow/drop interface has few limitations.
> >>
> >> 1. Driver cannot add or delete an individual entry for mac and vlan.
> >> 2. Driver cannot select mac+vlan combination for which
> >>     to allow/drop packet.
> >> 3. Driver cannot not set other commonly used packet match fields
> >>     such as IP header fields, TCP, UDP, SCP header fields.
> >> 4. Driver cannot steer specific packets based on the match
> >>    fields to specific receiveq.
> >> 5. Driver do not have multiple or dedicated virtqueues to
> >>     perform flow filter requests in accelerated manner in
> >>     the device.
> >>
> >> Solution:
> >> =========
> >> Flow filter as a generic framework to overcome above limitations.
> >>
> >> Overview:
> >> =========
> >> A flow filter defines the flow based on one or more match fields of
> >> the packet, defines an action like drop/forward to RQ.
> >>
> >> The flow filters are organized in flow filter groups so that their
> >> processing can be ordered when multiple applications wants to use it.
> >>
> >> Flow filters requests can be transported via control vq or dedicated
> >> flow filter virtqueue so that it does not get intermixed with other slow
> operations of cvq.
> >>
> >> Flow filter requirements addressed by this series is worked by virtio
> >> community at [1].
> >>
> >> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
> >>
> >
> > Now that this has been mature phase and on list for several weeks, I
> > updated the github to point to v6.
> >
> > Can you please start the voting process?
> 
> I don't feel like I'm capable of judging the networking parts, but I have a couple
> of wording suggestions (things like definite articles
> etc.) Would you prefer a respin, or a vote now and an editorial patch on top? I
> can either comment now, or start voting.

Your comments are surely useful.
I would like to fix your comments and do a quick respin.
Please share them.

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] Re: [PATCH v6 1/5] virtio-net: Add theory of operation for flow filter

[Cornelia Huck]

On Fri, Nov 10 2023, Parav Pandit <parav@nvidia.com> wrote:

> diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> index aff5e08..30220b5 100644
> --- a/device-types/net/description.tex
> +++ b/device-types/net/description.tex
> @@ -33,6 +33,9 @@ \subsection{Virtqueues}\label{sec:Device Types / Network Device / Virtqueues}
>  controlq is optional; it only exists if VIRTIO_NET_F_CTRL_VQ is
>  negotiated.
>  
> +The flow filter virtqueues are optional; it may exists only if VIRTIO_NET_F_FLOW_FILTER
> +is negotiated and if the device reports such capability.

I first thought we should just reword this... but what does "such
capability" actually mean? If the feature is negotiated, there are M
flow filter virtqueues at positions 2N+1..2N+M, with the value of M
exposed by the device? I think we need to list those virtqueues
explicitly, otherwise it is confusing.

(...)

> +Each flow filter consists of one or more match key, a flow filter priority,

s/key/keys/

> +a flow filter identifier, an action to forward a packet to the destination
> +or to drop the packet.

(...)

> +The flow filter group can have one or more flow filters. Within a flow

s/The/A/

> +filter group, a packet may find a match to multiple flow filters. In such

s/find a match to/match/
s/such/such a/

> +scenario, a flow filter with the highest priority is applied first to the
> +packet, if there is no match, the next higher priority flow filter is applied.

Slightly confusing... maybe

"In such a scenario, of the matching flow filters the one with the
highest priority is applied, skipping any matching filters with a lower
priority."

(At least, that's how I interpret this.)


> +
> +\paragraph{Packet Processing Order}\label{sec:sec:Device Types / Network Device / Device Operation / Flow Filter / Packet Processing Order}
> +
> +Whichever filtering and steering functionality is enabled, they are
> +applied in the following order for the received packet:

"If enabled, filtering and steering functionalities are applied to the
received packet in the following order:" ?

> +
> +\begin{itemize}
> +\item apply device configuration done using control virtqueue commands
> +      VIRTIO_NET_CTRL_RX, VIRTIO_NET_CTRL_MAC and VIRTIO_NET_CTRL_VLAN.
> +\item apply flow filter configuration done using flow filter requests.
> +\item apply device configuration done using command
> +      VIRTIO_NET_CTRL_MQ_RSS_CONFIG.
> +\end{itemize}
> +
> +While processing a received packet, at any stage if the packet is dropped,

"if the packet is dropped at any stage"

> +the next level of processing is omitted.

"the following levels of processing are omitted"

> +
> +When a flow filter is matched for the packet, it also stops the processing
> +of the packet for next stage.

"If a flow filter is matched for the packet, the following levels of
processing are omitted as well." ?

> +
> +Few examples are:

s/Few/A few/

> +\begin{itemize}
> +\item If the packet is dropped by the flow filter configuration, RSS
> +      configuration is not applied to such a packet.
> +\item If the packet is forwarded to a specific receiveq using flow filters,
> +      RSS configuration is not applied to such a packet due to a match on the
> +      flow filter request.
> +\item If the packet is dropped due to VIRTIO_NET_CTRL_MAC configuration,
> +      flow filters or RSS configuration is not applied to such a packet.

"neither flow filters nor RSS configuration are applied"

> +\item If the packet does not find any match in any of the flow filter groups,
> +      next level RSS device configuration is applied if its exists.
> +\item If there are three flow filter groups configured as group_A, group_B
> +      and group_C with respective priorities as 4, 5, and 6; flow filters of

s/flow filters/the flow filters/

> +      group_C is applied first having highest group priority, if there is a match,

s/is applied/are applied/

> +      flow filters of group_B and group_A are skipped; if there is no match for
> +      the flow filters in group_C, the flow filters of next level group_B are applied.
> +\end{itemize}
> +
> +\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Setting Promiscuous Mode}%old label for latexdiff
>  \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>  
>  The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] RE: [PATCH v6 1/5] virtio-net: Add theory of operation for flow filter

[Parav Pandit]

> From: Cornelia Huck <cohuck@redhat.com>
> Sent: Wednesday, November 22, 2023 6:53 PM


> On Fri, Nov 10 2023, Parav Pandit <parav@nvidia.com> wrote:
> 
> > diff --git a/device-types/net/description.tex
> > b/device-types/net/description.tex
> > index aff5e08..30220b5 100644
> > --- a/device-types/net/description.tex
> > +++ b/device-types/net/description.tex
> > @@ -33,6 +33,9 @@ \subsection{Virtqueues}\label{sec:Device Types /
> > Network Device / Virtqueues}  controlq is optional; it only exists if
> > VIRTIO_NET_F_CTRL_VQ is  negotiated.
> >
> > +The flow filter virtqueues are optional; it may exists only if
> > +VIRTIO_NET_F_FLOW_FILTER is negotiated and if the device reports such
> capability.
> 
> I first thought we should just reword this... but what does "such capability"
> actually mean? If the feature is negotiated, there are M flow filter virtqueues
> at positions 2N+1..2N+M, with the value of M exposed by the device? I think
> we need to list those virtqueues explicitly, otherwise it is confusing.
>
I am going to remove this left over of flow filter virtqueues text.
It was removed.
Good catch.

Ack for rest of the below comments.

> (...)
> 
> > +Each flow filter consists of one or more match key, a flow filter
> > +priority,
> 
> s/key/keys/
> 
> > +a flow filter identifier, an action to forward a packet to the
> > +destination or to drop the packet.
> 
> (...)
> 
> > +The flow filter group can have one or more flow filters. Within a
> > +flow
> 
> s/The/A/
> 
> > +filter group, a packet may find a match to multiple flow filters. In
> > +such
> 
> s/find a match to/match/
> s/such/such a/
> 
> > +scenario, a flow filter with the highest priority is applied first to
> > +the packet, if there is no match, the next higher priority flow filter is
> applied.
> 
> Slightly confusing... maybe
> 
> "In such a scenario, of the matching flow filters the one with the highest
> priority is applied, skipping any matching filters with a lower priority."
> 
> (At least, that's how I interpret this.)
> 
> 
> > +
> > +\paragraph{Packet Processing Order}\label{sec:sec:Device Types /
> > +Network Device / Device Operation / Flow Filter / Packet Processing
> > +Order}
> > +
> > +Whichever filtering and steering functionality is enabled, they are
> > +applied in the following order for the received packet:
> 
> "If enabled, filtering and steering functionalities are applied to the received
> packet in the following order:" ?
> 
> > +
> > +\begin{itemize}
> > +\item apply device configuration done using control virtqueue commands
> > +      VIRTIO_NET_CTRL_RX, VIRTIO_NET_CTRL_MAC and
> VIRTIO_NET_CTRL_VLAN.
> > +\item apply flow filter configuration done using flow filter requests.
> > +\item apply device configuration done using command
> > +      VIRTIO_NET_CTRL_MQ_RSS_CONFIG.
> > +\end{itemize}
> > +
> > +While processing a received packet, at any stage if the packet is
> > +dropped,
> 
> "if the packet is dropped at any stage"
> 
> > +the next level of processing is omitted.
> 
> "the following levels of processing are omitted"
> 
> > +
> > +When a flow filter is matched for the packet, it also stops the
> > +processing of the packet for next stage.
> 
> "If a flow filter is matched for the packet, the following levels of processing
> are omitted as well." ?
> 
> > +
> > +Few examples are:
> 
> s/Few/A few/
> 
> > +\begin{itemize}
> > +\item If the packet is dropped by the flow filter configuration, RSS
> > +      configuration is not applied to such a packet.
> > +\item If the packet is forwarded to a specific receiveq using flow filters,
> > +      RSS configuration is not applied to such a packet due to a match on the
> > +      flow filter request.
> > +\item If the packet is dropped due to VIRTIO_NET_CTRL_MAC
> configuration,
> > +      flow filters or RSS configuration is not applied to such a packet.
> 
> "neither flow filters nor RSS configuration are applied"
> 
> > +\item If the packet does not find any match in any of the flow filter
> groups,
> > +      next level RSS device configuration is applied if its exists.
> > +\item If there are three flow filter groups configured as group_A, group_B
> > +      and group_C with respective priorities as 4, 5, and 6; flow
> > +filters of
> 
> s/flow filters/the flow filters/
> 
> > +      group_C is applied first having highest group priority, if
> > + there is a match,
> 
> s/is applied/are applied/
> 
> > +      flow filters of group_B and group_A are skipped; if there is no match
> for
> > +      the flow filters in group_C, the flow filters of next level group_B are
> applied.
> > +\end{itemize}
> > +
> > +\label{sec:Device Types / Network Device / Device Operation / Control
> > +Virtqueue / Setting Promiscuous Mode}%old label for latexdiff
> >  \subsubsection{Control Virtqueue}\label{sec:Device Types / Network
> > Device / Device Operation / Control Virtqueue}
> >
> >  The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Cornelia Huck]

On Fri, Nov 10 2023, Parav Pandit <parav@nvidia.com> wrote:

> diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> index 30220b5..eccd8d6 100644
> --- a/device-types/net/description.tex
> +++ b/device-types/net/description.tex
> @@ -1173,7 +1173,11 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
>  
>  The device indicates the flow filter capabilities to the driver. These
>  capabilities include various maximum device limits and
> -supported packet match fields.
> +supported packet match fields. These control virtqueue
> +commands are:

Wait, if this uses control vq commands, what are the "flow control
virtqueues" mentioned in the previous patch?

> +\ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Capabilities Get}
> +and
> +\ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}.
>  
>  The flow filters are grouped using a flow filter group. Each flow filter
>  group has a priority. The device first applies the flow filters of the highest
> @@ -1224,7 +1228,136 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
>        the flow filters in group_C, the flow filters of next level group_B are applied.
>  \end{itemize}
>  
> -\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Setting Promiscuous Mode}%old label for latexdiff

I think you added this in the last patch? Just drop it?

> +\paragraph{Match Types and Fields}\label{sec:Device Types / Network Device / Device Operation / Flow Filter / Match Types and Fields}
> +
> +\begin{lstlisting}
> +struct virtio_net_ff_match_type_cap {
> +        le16 type;
> +        u8 mask_supported;
> +        u8 reserved[5];
> +        le64 fields_bmap;
> +};
> +\end{lstlisting}
> +
> +The \field{type} corresponds to following table:
> +
> +\begin{tabular}{|l|l|l|}
> +\hline
> +Type & Name & Description \\
> +\hline \hline
> +0   & VIRTIO_NET_FF_ETH_HDR & Ethernet header of the packet \\
> +\hline
> +0x1   & VIRTIO_NET_FF_VLAN_TAG_HDR & VLAN tag of the packet \\
> +\hline
> +0x200   & VIRTIO_NET_FF_IPV4_HDR & IPv4 header of the packet \\
> +\hline
> +0x300   & VIRTIO_NET_FF_IPV6_HDR & IPv6 header of the packet \\
> +\hline
> +0x400   & VIRTIO_NET_FF_TCP_HDR & TCP header of the packet \\
> +\hline
> +0x500   & VIRTIO_NET_FF_UDP_HDR & UDP header of the packet \\
> +\hline
> +other   & -    & reserved \\
> +\hline
> +\end{tabular}
> +
> +When the \field{mask_supported} is set, for the specific \field{type}, the

s/When the/When/

> +device can perform masking packet fields with the mask supplied in the flow

s/perform masking/mask/

> +filter match entry.
> +
> +For each \field{type} the \field{fields_bmap} indicates supported fields
> +of the packet header which can be matched.
> +
> +For the \field{type} of VIRTIO_NET_FF_ETH_HDR, header fields
> +are represented by a bitmap in \field{fields_bmap} are following:

s/are following/as follows/

> +
> +\begin{tabular}{|l|l|l|}
> +\hline
> +Bit & Name & Description \\
> +\hline \hline
> +0   & VIRTIO_NET_FF_DST_MAC & Destination MAC address in the packet \\
> +\hline
> +1   & VIRTIO_NET_FF_SRC_MAC & Source MAC address in the packet \\
> +\hline
> +2   & VIRTIO_NET_FF_ETHER_TYPE & Ether type in the packet \\
> +\hline
> +other   & -    & reserved \\
> +\hline
> +\end{tabular}
> +
> +For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN tag fields
> +are represented by a bitmap in \field{fields_bmap} are following:

s/are following/as follows/

> +
> +\begin{tabular}{|l|l|l|}
> +\hline
> +Bit & Name & Description \\
> +\hline \hline
> +0   & VIRTIO_NET_FF_VLAN_TAG_TCI & Vlan tag TCI 16-bit field \\
> +\hline
> +other   & -    & reserved \\
> +\hline
> +\end{tabular}
> +
> +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, header fields
> +are represented by a bitmap in \field{fields_bmap} are following:

s/are following/as follows/

> +
> +\begin{tabular}{|l|l|l|}
> +\hline
> +Bit & Name & Description \\
> +\hline \hline
> +0   & VIRTIO_NET_FF_SRC_IPV4 & Source IPV4 address in the packet \\
> +\hline
> +1   & VIRTIO_NET_FF_DST_IPV4 & Destination IPV4 address in the packet \\
> +\hline
> +other   & -    & reserved \\
> +\hline
> +\end{tabular}
> +
> +For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header fields
> +are represented by a bitmap in \field{fields_bmap} are following:

s/are following/as follows/

> +
> +\begin{tabular}{|l|l|l|}
> +\hline
> +Bit & Name & Description \\
> +\hline \hline
> +0   & VIRTIO_NET_FF_SRC_IPV6 & Source IPV6 address in the packet \\
> +\hline
> +1   & VIRTIO_NET_FF_DST_IPV6 & Destination IPV6 address in the packet \\
> +\hline
> +other   & -    & reserved \\
> +\hline
> +\end{tabular}
> +
> +For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header fields
> +are represented by a bitmap in \field{fields_bmap} are following:

s/are following/as follows/

> +
> +\begin{tabular}{|l|l|l|}
> +\hline
> +Bit & Name & Description \\
> +\hline \hline
> +0   & VIRTIO_NET_FF_SRC_TCP_PORT & Source TCP port in the packet \\
> +\hline
> +1   & VIRTIO_NET_FF_DST_TCP_PORT & Destination TCP port in the packet \\
> +\hline
> +other   & -    & reserved \\
> +\hline
> +\end{tabular}
> +
> +For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header fields
> +are represented by a bitmap in \field{fields_bmap} are following:

s/are following/as follows/

> +
> +\begin{tabular}{|l|l|l|}
> +\hline
> +Bit & Name & Description \\
> +\hline \hline
> +0   & VIRTIO_NET_FF_SRC_UDP_PORT & Source UDP port in the packet \\
> +\hline
> +1   & VIRTIO_NET_FF_DST_UDP_PORT & Destination UDP port in the packet \\
> +\hline
> +other   & -    & reserved  \\
> +\hline
> +\end{tabular}
> +
>  \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>  
>  The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
> @@ -2392,6 +2525,77 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>  of the driver's records. In such cases, the driver should allocate additional
>  space for the \field{command-specific-result} buffer.
>  
> +\paragraph{Flow Filter}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter}
> +
> +If the VIRTIO_NET_F_FLOW_FILTER feature is negotiated,
> +
> +\begin{itemize}
> +\item the driver can send commands VIRTIO_NET_CTRL_FF_CAP_GET and
> +VIRTIO_NET_CTRL_FF_MATCH_CAP_GET to query the flow filter
> +capabilities of the device.
> +\end{itemize}
> +
> +\begin{lstlisting}
> +#define VIRTIO_NET_CTRL_FF 7
> + #define VIRTIO_NET_CTRL_FF_CAP_GET 0
> + #define VIRTIO_NET_CTRL_FF_MATCH_CAP_GET 1
> +\end{lstlisting}
> +
> +\subparagraph{Flow Filter Capabilities Get}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Capabilities Get}
> +
> +The command VIRTIO_NET_CTRL_FF_CAP_GET provides the flow filter device capabilities.
> +
> +\begin{lstlisting}
> +struct virtio_net_ctrl_ff_caps {
> +        le16 max_match_fields;
> +        le16 max_groups; /* valid group id = max_groups - 1 */
> +        le32 max_ff_per_group;
> +        le32 max_ff; /* max flow_id in add/del = max_ff - 1 */
> +        le16 max_actions;
> +        u8 max_flow_priorities_per_group;
> +};
> +\end{lstlisting}
> +
> +The \field{max_groups} indicates total number of flow filter groups supported

s/The//

> +by the device whose group identifier can be any value in the range from 0 to

s/group identifier/group indentifiers/

> +\field{max_groups - 1}. The flow filter group can have any priority in range
> +of 0 to \field{max_groups - 1}.
> +
> +The \field{max_ff_per_group} indicates maximum number of

s/The//
s/maximum/the maximum/

> +flow filter per flow filter group which can be added by the driver.

"flow filters"

> +
> +The \field{max_ff} indicates maximum number of flow filters across

s/The//
s/maximum/the maximum/

> +all the flow groups which can be added by the driver.
> +
> +The \field{max_ff_priorities_per_group} indicates maximum priority value

s/The//
s/maximum/the maximum/

> +of a flow filter within a group. A flow filter within a group can have any
> +priority in range of zero to \field{max_ff_priorities_per_group - 1}.
> +
> +The \field{max_match_fields} indicates maximum number of fields of the packet

s/The//
s/maximum/the maximum/
s/of the packet/of a packet/

> +which can be matched by the device for a flow filter.
> +
> +The \field{max_actions} indicates maximum number of actions for a flow filter

s/The//
s/maximum/the maximum/

> +match entry can be supplied by the driver.

"that can be supplied"

> +
> +The \field{max_flow_priorities_per_group} indicates maximum number of

s/The//
s/maximum/the maximum/

> +priorities supported by the device per flow filter group.
> +
> +\subparagraph{Flow Filter Match Capabilities Get}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}
> +
> +The command VIRTIO_NET_CTRL_FF_MATCH_CAP_GET indicates which fields
> +from the packet can be matched.

s/from/of/

> +
> +\begin{lstlisting}
> +struct virtio_net_ctrl_ff_match_types {
> +        le32 num_entries;
> +        struct virtio_net_ff_match_type_cap types[];
> +};
> +\end{lstlisting}
> +
> +The \field{num_entries} indicates the length of an array \field{types}.

s/The//
s/an/the/

> +Each array entry of \field{types} represents supported match fields of
> +the packet by the device.

"represents the fields of the packet which are supported for matching by
the device"

> +
>  \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
>  Types / Network Device / Legacy Interface: Framing Requirements}
>  


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] Re: [PATCH v6 3/5] virtio-net: Add flow filter group life cycle commands

[Cornelia Huck]

On Fri, Nov 10 2023, Parav Pandit <parav@nvidia.com> wrote:

> @@ -2596,6 +2606,32 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>  Each array entry of \field{types} represents supported match fields of
>  the packet by the device.
>  
> +\subparagraph{Flow Filter Group Add}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Group Add}
> +
> +The command VIRTIO_NET_CTRL_FF_GROUP_ADD adds a new flow filter
> +group with the supplied group identifier \field{id} with assigned
> +priority \field{priority}.
> +
> +\begin{lstlisting}
> +struct virtio_net_ctrl_ff_group_add {
> +        le16 priority;	/* higher the value, higher priority */
> +        le16 id;
> +};
> +\end{lstlisting}
> +
> +\subparagraph{Flow Filter Group Delete}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Group Delete}
> +
> +The command VIRTIO_NET_CTRL_FF_GROUP_DEL deletes the
> +flow filter group that has been previously added using command
> +VIRTIO_NET_CTRL_FF_GROUP_ADD. Flow filter group is
> +identified using a group identifier \field{id}.

"The command VIRTIO_NET_CTRL_FF_GROUP_DEL deletes the flow filter group
identified by the group identifier \field{id} previously added using the
command VIRTIO_NET_CTRL_FF_GROUP_ADD."

> +
> +\begin{lstlisting}
> +struct virtio_net_ctrl_ff_group_del {
> +        le16 id;
> +};
> +\end{lstlisting}
> +
>  \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
>  Types / Network Device / Legacy Interface: Framing Requirements}


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] RE: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Cornelia Huck <cohuck@redhat.com>
> Sent: Wednesday, November 22, 2023 7:08 PM
> 
> On Fri, Nov 10 2023, Parav Pandit <parav@nvidia.com> wrote:
> 
> > diff --git a/device-types/net/description.tex
> > b/device-types/net/description.tex
> > index 30220b5..eccd8d6 100644
> > --- a/device-types/net/description.tex
> > +++ b/device-types/net/description.tex
> > @@ -1173,7 +1173,11 @@ \subsubsection{Flow Filter}\label{sec:Device
> > Types / Network Device / Device Ope
> >
> >  The device indicates the flow filter capabilities to the driver.
> > These  capabilities include various maximum device limits and
> > -supported packet match fields.
> > +supported packet match fields. These control virtqueue commands are:
> 
> Wait, if this uses control vq commands, what are the "flow control
> virtqueues" mentioned in the previous patch?
>
As I replied, please ignore that text where you gave example of M, M + 1 etc.
I am going to remove that stale left over text.

 
> > +\ref{sec:Device Types / Network Device / Device Operation / Control
> > +Virtqueue / Flow Filter / Flow Filter Capabilities Get} and
> > +\ref{sec:Device Types / Network Device / Device Operation / Control
> Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}.
> >
> >  The flow filters are grouped using a flow filter group. Each flow
> > filter  group has a priority. The device first applies the flow
> > filters of the highest @@ -1224,7 +1228,136 @@ \subsubsection{Flow
> Filter}\label{sec:Device Types / Network Device / Device Ope
> >        the flow filters in group_C, the flow filters of next level group_B are
> applied.
> >  \end{itemize}
> >
> > -\label{sec:Device Types / Network Device / Device Operation / Control
> > Virtqueue / Setting Promiscuous Mode}%old label for latexdiff
> 
> I think you added this in the last patch? Just drop it?
> 
Yes. it was some wrong hunk.

> > +\paragraph{Match Types and Fields}\label{sec:Device Types / Network
> > +Device / Device Operation / Flow Filter / Match Types and Fields}
> > +
> > +\begin{lstlisting}
> > +struct virtio_net_ff_match_type_cap {
> > +        le16 type;
> > +        u8 mask_supported;
> > +        u8 reserved[5];
> > +        le64 fields_bmap;
> > +};
> > +\end{lstlisting}
> > +
> > +The \field{type} corresponds to following table:
> > +
> > +\begin{tabular}{|l|l|l|}
> > +\hline
> > +Type & Name & Description \\
> > +\hline \hline
> > +0   & VIRTIO_NET_FF_ETH_HDR & Ethernet header of the packet \\
> > +\hline
> > +0x1   & VIRTIO_NET_FF_VLAN_TAG_HDR & VLAN tag of the packet \\
> > +\hline
> > +0x200   & VIRTIO_NET_FF_IPV4_HDR & IPv4 header of the packet \\
> > +\hline
> > +0x300   & VIRTIO_NET_FF_IPV6_HDR & IPv6 header of the packet \\
> > +\hline
> > +0x400   & VIRTIO_NET_FF_TCP_HDR & TCP header of the packet \\
> > +\hline
> > +0x500   & VIRTIO_NET_FF_UDP_HDR & UDP header of the packet \\
> > +\hline
> > +other   & -    & reserved \\
> > +\hline
> > +\end{tabular}
> > +
> > +When the \field{mask_supported} is set, for the specific
> > +\field{type}, the
> 
> s/When the/When/
> 
> > +device can perform masking packet fields with the mask supplied in
> > +the flow
> 
> s/perform masking/mask/
> 
> > +filter match entry.
> > +
> > +For each \field{type} the \field{fields_bmap} indicates supported
> > +fields of the packet header which can be matched.
> > +
> > +For the \field{type} of VIRTIO_NET_FF_ETH_HDR, header fields are
> > +represented by a bitmap in \field{fields_bmap} are following:
> 
> s/are following/as follows/
> 
> > +
> > +\begin{tabular}{|l|l|l|}
> > +\hline
> > +Bit & Name & Description \\
> > +\hline \hline
> > +0   & VIRTIO_NET_FF_DST_MAC & Destination MAC address in the packet
> \\
> > +\hline
> > +1   & VIRTIO_NET_FF_SRC_MAC & Source MAC address in the packet \\
> > +\hline
> > +2   & VIRTIO_NET_FF_ETHER_TYPE & Ether type in the packet \\
> > +\hline
> > +other   & -    & reserved \\
> > +\hline
> > +\end{tabular}
> > +
> > +For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN tag fields
> > +are represented by a bitmap in \field{fields_bmap} are following:
> 
> s/are following/as follows/
> 
> > +
> > +\begin{tabular}{|l|l|l|}
> > +\hline
> > +Bit & Name & Description \\
> > +\hline \hline
> > +0   & VIRTIO_NET_FF_VLAN_TAG_TCI & Vlan tag TCI 16-bit field \\
> > +\hline
> > +other   & -    & reserved \\
> > +\hline
> > +\end{tabular}
> > +
> > +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, header fields are
> > +represented by a bitmap in \field{fields_bmap} are following:
> 
> s/are following/as follows/
> 
> > +
> > +\begin{tabular}{|l|l|l|}
> > +\hline
> > +Bit & Name & Description \\
> > +\hline \hline
> > +0   & VIRTIO_NET_FF_SRC_IPV4 & Source IPV4 address in the packet \\
> > +\hline
> > +1   & VIRTIO_NET_FF_DST_IPV4 & Destination IPV4 address in the packet
> \\
> > +\hline
> > +other   & -    & reserved \\
> > +\hline
> > +\end{tabular}
> > +
> > +For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header fields are
> > +represented by a bitmap in \field{fields_bmap} are following:
> 
> s/are following/as follows/
> 
> > +
> > +\begin{tabular}{|l|l|l|}
> > +\hline
> > +Bit & Name & Description \\
> > +\hline \hline
> > +0   & VIRTIO_NET_FF_SRC_IPV6 & Source IPV6 address in the packet \\
> > +\hline
> > +1   & VIRTIO_NET_FF_DST_IPV6 & Destination IPV6 address in the packet
> \\
> > +\hline
> > +other   & -    & reserved \\
> > +\hline
> > +\end{tabular}
> > +
> > +For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header fields are
> > +represented by a bitmap in \field{fields_bmap} are following:
> 
> s/are following/as follows/
> 
> > +
> > +\begin{tabular}{|l|l|l|}
> > +\hline
> > +Bit & Name & Description \\
> > +\hline \hline
> > +0   & VIRTIO_NET_FF_SRC_TCP_PORT & Source TCP port in the packet \\
> > +\hline
> > +1   & VIRTIO_NET_FF_DST_TCP_PORT & Destination TCP port in the packet
> \\
> > +\hline
> > +other   & -    & reserved \\
> > +\hline
> > +\end{tabular}
> > +
> > +For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header fields are
> > +represented by a bitmap in \field{fields_bmap} are following:
> 
> s/are following/as follows/
> 
> > +
> > +\begin{tabular}{|l|l|l|}
> > +\hline
> > +Bit & Name & Description \\
> > +\hline \hline
> > +0   & VIRTIO_NET_FF_SRC_UDP_PORT & Source UDP port in the packet \\
> > +\hline
> > +1   & VIRTIO_NET_FF_DST_UDP_PORT & Destination UDP port in the
> packet \\
> > +\hline
> > +other   & -    & reserved  \\
> > +\hline
> > +\end{tabular}
> > +
> >  \subsubsection{Control Virtqueue}\label{sec:Device Types / Network
> > Device / Device Operation / Control Virtqueue}
> >
> >  The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is @@
> > -2392,6 +2525,77 @@ \subsubsection{Control Virtqueue}\label{sec:Device
> > Types / Network Device / Devi  of the driver's records. In such cases,
> > the driver should allocate additional  space for the \field{command-
> specific-result} buffer.
> >
> > +\paragraph{Flow Filter}\label{sec:Device Types / Network Device /
> > +Device Operation / Control Virtqueue / Flow Filter}
> > +
> > +If the VIRTIO_NET_F_FLOW_FILTER feature is negotiated,
> > +
> > +\begin{itemize}
> > +\item the driver can send commands VIRTIO_NET_CTRL_FF_CAP_GET and
> > +VIRTIO_NET_CTRL_FF_MATCH_CAP_GET to query the flow filter
> > +capabilities of the device.
> > +\end{itemize}
> > +
> > +\begin{lstlisting}
> > +#define VIRTIO_NET_CTRL_FF 7
> > + #define VIRTIO_NET_CTRL_FF_CAP_GET 0  #define
> > +VIRTIO_NET_CTRL_FF_MATCH_CAP_GET 1 \end{lstlisting}
> > +
> > +\subparagraph{Flow Filter Capabilities Get}\label{sec:Device Types /
> > +Network Device / Device Operation / Control Virtqueue / Flow Filter /
> > +Flow Filter Capabilities Get}
> > +
> > +The command VIRTIO_NET_CTRL_FF_CAP_GET provides the flow filter
> device capabilities.
> > +
> > +\begin{lstlisting}
> > +struct virtio_net_ctrl_ff_caps {
> > +        le16 max_match_fields;
> > +        le16 max_groups; /* valid group id = max_groups - 1 */
> > +        le32 max_ff_per_group;
> > +        le32 max_ff; /* max flow_id in add/del = max_ff - 1 */
> > +        le16 max_actions;
> > +        u8 max_flow_priorities_per_group; }; \end{lstlisting}
> > +
> > +The \field{max_groups} indicates total number of flow filter groups
> > +supported
> 
> s/The//
> 
> > +by the device whose group identifier can be any value in the range
> > +from 0 to
> 
> s/group identifier/group indentifiers/
> 
> > +\field{max_groups - 1}. The flow filter group can have any priority
> > +in range of 0 to \field{max_groups - 1}.
> > +
> > +The \field{max_ff_per_group} indicates maximum number of
> 
> s/The//
> s/maximum/the maximum/
> 
> > +flow filter per flow filter group which can be added by the driver.
> 
> "flow filters"
> 
> > +
> > +The \field{max_ff} indicates maximum number of flow filters across
> 
> s/The//
> s/maximum/the maximum/
> 
> > +all the flow groups which can be added by the driver.
> > +
> > +The \field{max_ff_priorities_per_group} indicates maximum priority
> > +value
> 
> s/The//
> s/maximum/the maximum/
> 
> > +of a flow filter within a group. A flow filter within a group can
> > +have any priority in range of zero to \field{max_ff_priorities_per_group -
> 1}.
> > +
> > +The \field{max_match_fields} indicates maximum number of fields of
> > +the packet
> 
> s/The//
> s/maximum/the maximum/
> s/of the packet/of a packet/
> 
> > +which can be matched by the device for a flow filter.
> > +
> > +The \field{max_actions} indicates maximum number of actions for a
> > +flow filter
> 
> s/The//
> s/maximum/the maximum/
> 
> > +match entry can be supplied by the driver.
> 
> "that can be supplied"
> 
> > +
> > +The \field{max_flow_priorities_per_group} indicates maximum number of
> 
> s/The//
> s/maximum/the maximum/
> 
> > +priorities supported by the device per flow filter group.
> > +
> > +\subparagraph{Flow Filter Match Capabilities Get}\label{sec:Device
> > +Types / Network Device / Device Operation / Control Virtqueue / Flow
> > +Filter / Flow Filter Match Capabilities Get}
> > +
> > +The command VIRTIO_NET_CTRL_FF_MATCH_CAP_GET indicates which
> fields
> > +from the packet can be matched.
> 
> s/from/of/
> 
> > +
> > +\begin{lstlisting}
> > +struct virtio_net_ctrl_ff_match_types {
> > +        le32 num_entries;
> > +        struct virtio_net_ff_match_type_cap types[]; };
> > +\end{lstlisting}
> > +
> > +The \field{num_entries} indicates the length of an array \field{types}.
> 
> s/The//
> s/an/the/
> 
> > +Each array entry of \field{types} represents supported match fields
> > +of the packet by the device.
> 
> "represents the fields of the packet which are supported for matching by the
> device"
> 
> > +
> >  \subsubsection{Legacy Interface: Framing
> > Requirements}\label{sec:Device  Types / Network Device / Legacy
> > Interface: Framing Requirements}
> >


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] Re: [PATCH v6 4/5] virtio-net: Add flow filter match entry, action and requests

[Cornelia Huck]

On Fri, Nov 10 2023, Parav Pandit <parav@nvidia.com> wrote:

> diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> index 31c8c35..c1a08f8 100644
> --- a/device-types/net/description.tex
> +++ b/device-types/net/description.tex
> @@ -33,9 +33,6 @@ \subsection{Virtqueues}\label{sec:Device Types / Network Device / Virtqueues}
>  controlq is optional; it only exists if VIRTIO_NET_F_CTRL_VQ is
>  negotiated.
>  
> -The flow filter virtqueues are optional; it may exists only if VIRTIO_NET_F_FLOW_FILTER
> -is negotiated and if the device reports such capability.
> -

Ok, here you remove the confusing text again -- please remove the hunk
from the first patch instead.

(...)

> @@ -1290,6 +1331,21 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
>  \hline
>  \end{tabular}
>  
> +For the \field{type} of VIRTIO_NET_FF_ETH_HDR, the match entry

s/entry/entries/

> +\field{key} and \field{mask} are in following format:
> +
> +\begin{lstlisting}
> +struct virtio_net_ff_match_eth_hdr {
> +        u8 dmac[6];
> +        u8 smac[6];
> +        le16 ether_type;
> +};
> +\end{lstlisting}
> +
> +The \field{dmac} is valid when VIRTIO_NET_FF_DST_MAC is set.
> +The \field{smac} is valid when VIRTIO_NET_FF_SRC_MAC is set.
> +The \field{ether_type} is valid when VIRTIO_NET_FF_ETHER_TYPE is set.

s/The// (x3)

> +
>  For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN tag fields
>  are represented by a bitmap in \field{fields_bmap} are following:
>  
> @@ -1318,6 +1374,20 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
>  \hline
>  \end{tabular}
>  
> +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, match entry

s/match entry/the match entries/

> +\field{key} and \field{mask} are in following format:
> +
> +\begin{lstlisting}
> +struct virtio_net_ff_match_ipv4_hdr {
> +        le32 reserved[3];
> +        le32 sip;
> +        le32 dip;
> +};
> +\end{lstlisting}
> +
> +The \field{sip} is valid when VIRTIO_NET_FF_SRC_IPV4 is set.
> +The \field{dip} is valid when VIRTIO_NET_FF_DST_IPV4 is set.

s/The// (x2)

> +
>  For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header fields
>  are represented by a bitmap in \field{fields_bmap} are following:
>  
> @@ -1333,6 +1403,20 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
>  \hline
>  \end{tabular}
>  
> +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, match entry

s/match entry/the match entries/

> +\field{key} and \field{mask} are in following format:
> +
> +\begin{lstlisting}
> +struct virtio_net_ff_match_ipv6_hdr {
> +        le32 reserved[2];
> +        u8 sip[16];
> +        u8 dip[16];
> +};
> +\end{lstlisting}
> +
> +The \field{sip} is valid when VIRTIO_NET_FF_SRC_IPV6 is set.
> +The \field{dip} is valid when VIRTIO_NET_FF_DST_IPV6 is set.

s/The// (x2)

> +
>  For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header fields
>  are represented by a bitmap in \field{fields_bmap} are following:
>  
> @@ -1348,6 +1432,20 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
>  \hline
>  \end{tabular}
>  
> +For the \field{type} of VIRTIO_NET_FF_TCP_HDR, match entry

s/match entry/the match entries/

> +\field{key} and \field{mask} are in following format:
> +
> +\begin{lstlisting}
> +struct virtio_ndr_ff_match_tcp_hdr {
> +        le16 sport;
> +        le16 dport;
> +        le32 reserved[4];
> +};
> +\end{lstlisting}
> +
> +The \field{sport} is valid when VIRTIO_NET_FF_SRC_TCP_PORT is set.

s/The//

> +This \field{dport} is valid when VIRTIO_NET_FF_DST_TCP_PORT is set.

s/This//

> +
>  For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header fields
>  are represented by a bitmap in \field{fields_bmap} are following:
>  
> @@ -1363,6 +1461,88 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
>  \hline
>  \end{tabular}
>  
> +For the \field{type} of VIRTIO_NET_FF_UDP_HDR, match entry

s/match entry/the match entries/

> +\field{key} and \field{mask} are in following format:
> +
> +\begin{lstlisting}
> +struct virtio_ndr_ff_match_udp_hdr {
> +        le16 sport;
> +        le16 dport;
> +        le32 reserved;
> +};
> +\end{lstlisting}
> +
> +The \field{sport} is valid when VIRTIO_NET_FF_SRC_UDP_PORT is set.

s/The//

> +This \field{dport} is valid when VIRTIO_NET_FF_DST_UDP_PORT is set.

s/This//

> +
> +\paragraph{Flow Filter Request}
> +\label{sec:Device Types / Network Device / Device Operation / Flow Filter / Flow Filter Request}
> +
> +Two flow filter requests are supported by the device.
> +
> +\begin{itemize}
> +\item Add or replace a flow filter using a request \field{struct virtio_net_ff_add}.
> +
> +\item Delete an existing flow filter using a request \field{struct virtio_net_ff_del}.
> +
> +\end{itemize}
> +
> +\begin{lstlisting}
> +#define VIRTIO_NET_FF_REQ_ADD 0
> +#define VIRTIO_NET_FF_REQ_DEL 1
> +
> +struct virtio_net_ff_add {
> +        u8 request; /* VIRTIO_NET_FF_REQ_ADD */
> +        u8 priority;	/* higher the value, higher priority */
> +        u16 group_id;
> +        le32 id;
> +        struct virtio_ff_match match;
> +        struct virtio_ff_action action;
> +};
> +
> +struct virtio_net_ff_del {
> +        u8 request;  /* VIRTIO_NET_FF_REQ_DEL */
> +        u8 reserved[3];
> +        le32 id;
> +};
> +
> +struct virtio_net_ff_result {
> +        le16 status;
> +};
> +
> +#define VIRTIO_NET_FF_RESULT_OK 0
> +#define VIRTIO_NET_FF_RESULT_ERR 1
> +
> +struct virtio_net_ff_req {
> +        /* Device-readable part */
> +        union {
> +                struct virtio_net_ff_add add;
> +                struct virtio_net_ff_del del;
> +        };
> +        /* Device-writable part */
> +        struct virtio_net_ff_result result;
> +};
> +\end{lstlisting}
> +
> +When adding a flow filter entry using request \field{struct virtio_net_ff_add},
> +the \field{match.match_entries} indidates number of valid array entries \field{match.entries}.

s/the//
s/number of/the number of/

> +For each of the valid entry in \field{match.entries}, the field \field{type}

s/entry/entries/
s/field/fields/

> +and \field{key} are in the format described in
> +\ref{sec:Device Types / Network Device / Device Operation / Flow Filter / Match Types and Fields}.
> +When the \field{mask_present} is set, the field \field{mask} is present and it has

s/When the/When/
s/it has/has/

> +exact same format as \field{key}.
> +
> +The field \field{key_mask_len} represents the total length of fields
> +\field{key} and \field{mask}.
> +
> +Previously added flow filter entry can be deleted by the driver using

s/entry/entries/
s/using/using the/

> +VIRTIO_NET_FF_REQ_DEL request.
> +
> +When the device completes the request, \field{status} is updated
> +by the device; when the request is successful, \field{status} is
> +set to VIRTIO_NET_FF_RESULT_OK, on error, \field{status} is
> +set to VIRTIO_NET_FF_RESULT_ERR.
> +
>  \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
>  
>  The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
> @@ -2541,6 +2721,8 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>  \item the driver can send commands VIRTIO_NET_CTRL_FF_GROUP_ADD
>  and VIRTIO_NET_CTRL_FF_GROUP_DEL for adding and deleting the
>  flow group.
> +\item the driver can send command VIRTIO_NET_CTRL_FF_REQ to

s/command/the command/

> +add or delete flow filter.
>  \end{itemize}
>  
>  \begin{lstlisting}
> @@ -2549,6 +2731,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>   #define VIRTIO_NET_CTRL_FF_MATCH_CAP_GET 1
>   #define VIRTIO_NET_CTRL_FF_GROUP_ADD 2
>   #define VIRTIO_NET_CTRL_FF_GROUP_DEL 3
> + #define VIRTIO_NET_CTRL_FF_REQ 4
>  \end{lstlisting}
>  
>  \subparagraph{Flow Filter Capabilities Get}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Capabilities Get}
> @@ -2632,6 +2815,28 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>  };
>  \end{lstlisting}
>  
> +\subparagraph{Flow Filter Requests}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Requests}
> +
> +The flow filter requests are transported using command

s/command/the/

> +VIRTIO_NET_CTRL_FF_REQ command.
> +
> +\begin{lstlisting}
> +struct virtio_net_ctrl_ff_req {
> +        union {
> +                struct virtio_net_ff_add add;
> +                struct virtio_net_ff_del del;
> +        };
> +};
> +
> +\end{lstlisting}
> +
> +The \field{command-specific-data} is in format of
> +\field{struct virtio_net_ctrl_ff_req}.
> +
> +When the flow filter request command is successful, the
> +\field{command-specific-result} is in format of
> +\field{struct virtio_net_ff_result}.
> +
>  \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
>  Types / Network Device / Legacy Interface: Framing Requirements}


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Fri, Nov 10, 2023 at 02:38:50PM +0200, Parav Pandit wrote:
> The device responds flow filter capabilities using two commands.
> One command indicates generic flow filter device limits such as
> number of flow filters, number of flow filter groups, support or
> multiple transports etc.
> 
> The second command indicates supported match types, and fields
> of the packet.
> 
> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> Signed-off-by: Parav Pandit <parav@nvidia.com>
> 
> ---
> changelog:
> v2->v3:
> - rebased on virtio-1.4 branch
> - removed reference for flow filter virtqueue
> v1->v2:
> - addressed comments from Satananda
> - added vlan type match field
> - kept space for types between l2, l3, l4 header match types
> - renamed mask to mask_supported with shorter width
> - made more fields reserved for furture
> - addressed comments from Heng
> - grammar correction
> - added field to indicate supported number of actions per flow
>   filter match entry
> - added missing documentation for max_flow_priorities_per_group
> v0->v1:
> - added mask field in the type to indicate supported mask by device
>   and also in later patch to use it to indicate mask on adding
>   flow filter. As a result removed the mask_supported capability
>   field
> ---
>  device-types/net/description.tex | 208 ++++++++++++++++++++++++++++++-
>  1 file changed, 206 insertions(+), 2 deletions(-)
> 
> diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> index 30220b5..eccd8d6 100644
> --- a/device-types/net/description.tex
> +++ b/device-types/net/description.tex
> @@ -1173,7 +1173,11 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
>  
>  The device indicates the flow filter capabilities to the driver. These
>  capabilities include various maximum device limits and
> -supported packet match fields.
> +supported packet match fields. These control virtqueue
> +commands are:
> +\ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Capabilities Get}
> +and
> +\ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}.
>  
>  The flow filters are grouped using a flow filter group. Each flow filter
>  group has a priority. The device first applies the flow filters of the highest
> @@ -1224,7 +1228,136 @@ \subsubsection{Flow Filter}\label{sec:Device Types / Network Device / Device Ope
>        the flow filters in group_C, the flow filters of next level group_B are applied.
>  \end{itemize}
>  
> -\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Setting Promiscuous Mode}%old label for latexdiff
> +\paragraph{Match Types and Fields}\label{sec:Device Types / Network Device / Device Operation / Flow Filter / Match Types and Fields}
> +
> +\begin{lstlisting}
> +struct virtio_net_ff_match_type_cap {
> +        le16 type;
> +        u8 mask_supported;
> +        u8 reserved[5];
> +        le64 fields_bmap;
> +};
> +\end{lstlisting}
> +
> +The \field{type} corresponds to following table:
> +
> +\begin{tabular}{|l|l|l|}
> +\hline
> +Type & Name & Description \\
> +\hline \hline
> +0   & VIRTIO_NET_FF_ETH_HDR & Ethernet header of the packet \\
> +\hline
> +0x1   & VIRTIO_NET_FF_VLAN_TAG_HDR & VLAN tag of the packet \\
> +\hline
> +0x200   & VIRTIO_NET_FF_IPV4_HDR & IPv4 header of the packet \\
> +\hline
> +0x300   & VIRTIO_NET_FF_IPV6_HDR & IPv6 header of the packet \\
> +\hline
> +0x400   & VIRTIO_NET_FF_TCP_HDR & TCP header of the packet \\
> +\hline
> +0x500   & VIRTIO_NET_FF_UDP_HDR & UDP header of the packet \\
> +\hline
> +other   & -    & reserved \\
> +\hline
> +\end{tabular}
> +
> +When the \field{mask_supported} is set, for the specific \field{type}, the
> +device can perform masking packet fields with the mask supplied in the flow
> +filter match entry.
> +
> +For each \field{type} the \field{fields_bmap} indicates supported fields
> +of the packet header which can be matched.
> +
> +For the \field{type} of VIRTIO_NET_FF_ETH_HDR, header fields
> +are represented by a bitmap in \field{fields_bmap} are following:
> +
> +\begin{tabular}{|l|l|l|}
> +\hline
> +Bit & Name & Description \\
> +\hline \hline
> +0   & VIRTIO_NET_FF_DST_MAC & Destination MAC address in the packet \\
> +\hline
> +1   & VIRTIO_NET_FF_SRC_MAC & Source MAC address in the packet \\
> +\hline
> +2   & VIRTIO_NET_FF_ETHER_TYPE & Ether type in the packet \\
> +\hline
> +other   & -    & reserved \\
> +\hline
> +\end{tabular}
> +
> +For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN tag fields
> +are represented by a bitmap in \field{fields_bmap} are following:
> +
> +\begin{tabular}{|l|l|l|}
> +\hline
> +Bit & Name & Description \\
> +\hline \hline
> +0   & VIRTIO_NET_FF_VLAN_TAG_TCI & Vlan tag TCI 16-bit field \\
> +\hline
> +other   & -    & reserved \\
> +\hline
> +\end{tabular}
> +
> +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, header fields
> +are represented by a bitmap in \field{fields_bmap} are following:
> +
> +\begin{tabular}{|l|l|l|}
> +\hline
> +Bit & Name & Description \\
> +\hline \hline
> +0   & VIRTIO_NET_FF_SRC_IPV4 & Source IPV4 address in the packet \\
> +\hline
> +1   & VIRTIO_NET_FF_DST_IPV4 & Destination IPV4 address in the packet \\
> +\hline
> +other   & -    & reserved \\
> +\hline
> +\end{tabular}
> +
> +For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header fields
> +are represented by a bitmap in \field{fields_bmap} are following:
> +
> +\begin{tabular}{|l|l|l|}
> +\hline
> +Bit & Name & Description \\
> +\hline \hline
> +0   & VIRTIO_NET_FF_SRC_IPV6 & Source IPV6 address in the packet \\
> +\hline
> +1   & VIRTIO_NET_FF_DST_IPV6 & Destination IPV6 address in the packet \\
> +\hline
> +other   & -    & reserved \\
> +\hline
> +\end{tabular}
> +
> +For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header fields
> +are represented by a bitmap in \field{fields_bmap} are following:
> +
> +\begin{tabular}{|l|l|l|}
> +\hline
> +Bit & Name & Description \\
> +\hline \hline
> +0   & VIRTIO_NET_FF_SRC_TCP_PORT & Source TCP port in the packet \\
> +\hline
> +1   & VIRTIO_NET_FF_DST_TCP_PORT & Destination TCP port in the packet \\
> +\hline
> +other   & -    & reserved \\
> +\hline
> +\end{tabular}
> +
> +For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header fields
> +are represented by a bitmap in \field{fields_bmap} are following:
> +
> +\begin{tabular}{|l|l|l|}
> +\hline
> +Bit & Name & Description \\
> +\hline \hline
> +0   & VIRTIO_NET_FF_SRC_UDP_PORT & Source UDP port in the packet \\
> +\hline
> +1   & VIRTIO_NET_FF_DST_UDP_PORT & Destination UDP port in the packet \\
> +\hline
> +other   & -    & reserved  \\
> +\hline
> +\end{tabular}
> +

This is such an elaborate structure to report just 12 read only bits.
Please let's just follow the example of  le32 supported_tunnel_types
and add l32 supported_flow_control.


After you were trying to add kilobytes to megabytes of memory -
I see little reason to save 12 RO bits that can be shared by any
number of VFs.

However, I do think we should create an option to access config space
over DMA (preferably admin commands). Let's add this quickly and then
we don't need to worry about legacy guests accessing flow filter
through MMIO.

-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] RE: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, November 22, 2023 7:32 PM
> 
> On Fri, Nov 10, 2023 at 02:38:50PM +0200, Parav Pandit wrote:
> > The device responds flow filter capabilities using two commands.
> > One command indicates generic flow filter device limits such as number
> > of flow filters, number of flow filter groups, support or multiple
> > transports etc.
> >
> > The second command indicates supported match types, and fields of the
> > packet.
> >
> > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
> > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > Signed-off-by: Parav Pandit <parav@nvidia.com>
> >
> > ---
> > changelog:
> > v2->v3:
> > - rebased on virtio-1.4 branch
> > - removed reference for flow filter virtqueue
> > v1->v2:
> > - addressed comments from Satananda
> > - added vlan type match field
> > - kept space for types between l2, l3, l4 header match types
> > - renamed mask to mask_supported with shorter width
> > - made more fields reserved for furture
> > - addressed comments from Heng
> > - grammar correction
> > - added field to indicate supported number of actions per flow
> >   filter match entry
> > - added missing documentation for max_flow_priorities_per_group
> > v0->v1:
> > - added mask field in the type to indicate supported mask by device
> >   and also in later patch to use it to indicate mask on adding
> >   flow filter. As a result removed the mask_supported capability
> >   field
> > ---
> >  device-types/net/description.tex | 208
> > ++++++++++++++++++++++++++++++-
> >  1 file changed, 206 insertions(+), 2 deletions(-)
> >
> > diff --git a/device-types/net/description.tex
> > b/device-types/net/description.tex
> > index 30220b5..eccd8d6 100644
> > --- a/device-types/net/description.tex
> > +++ b/device-types/net/description.tex
> > @@ -1173,7 +1173,11 @@ \subsubsection{Flow Filter}\label{sec:Device
> > Types / Network Device / Device Ope
> >
> >  The device indicates the flow filter capabilities to the driver.
> > These  capabilities include various maximum device limits and
> > -supported packet match fields.
> > +supported packet match fields. These control virtqueue commands are:
> > +\ref{sec:Device Types / Network Device / Device Operation / Control
> > +Virtqueue / Flow Filter / Flow Filter Capabilities Get} and
> > +\ref{sec:Device Types / Network Device / Device Operation / Control
> Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}.
> >
> >  The flow filters are grouped using a flow filter group. Each flow
> > filter  group has a priority. The device first applies the flow
> > filters of the highest @@ -1224,7 +1228,136 @@ \subsubsection{Flow
> Filter}\label{sec:Device Types / Network Device / Device Ope
> >        the flow filters in group_C, the flow filters of next level group_B are
> applied.
> >  \end{itemize}
> >
> > -\label{sec:Device Types / Network Device / Device Operation / Control
> > Virtqueue / Setting Promiscuous Mode}%old label for latexdiff
> > +\paragraph{Match Types and Fields}\label{sec:Device Types / Network
> > +Device / Device Operation / Flow Filter / Match Types and Fields}
> > +
> > +\begin{lstlisting}
> > +struct virtio_net_ff_match_type_cap {
> > +        le16 type;
> > +        u8 mask_supported;
> > +        u8 reserved[5];
> > +        le64 fields_bmap;
> > +};
> > +\end{lstlisting}
> > +
> > +The \field{type} corresponds to following table:
> > +
> > +\begin{tabular}{|l|l|l|}
> > +\hline
> > +Type & Name & Description \\
> > +\hline \hline
> > +0   & VIRTIO_NET_FF_ETH_HDR & Ethernet header of the packet \\
> > +\hline
> > +0x1   & VIRTIO_NET_FF_VLAN_TAG_HDR & VLAN tag of the packet \\
> > +\hline
> > +0x200   & VIRTIO_NET_FF_IPV4_HDR & IPv4 header of the packet \\
> > +\hline
> > +0x300   & VIRTIO_NET_FF_IPV6_HDR & IPv6 header of the packet \\
> > +\hline
> > +0x400   & VIRTIO_NET_FF_TCP_HDR & TCP header of the packet \\
> > +\hline
> > +0x500   & VIRTIO_NET_FF_UDP_HDR & UDP header of the packet \\
> > +\hline
> > +other   & -    & reserved \\
> > +\hline
> > +\end{tabular}
> > +
> > +When the \field{mask_supported} is set, for the specific
> > +\field{type}, the device can perform masking packet fields with the
> > +mask supplied in the flow filter match entry.
> > +
> > +For each \field{type} the \field{fields_bmap} indicates supported
> > +fields of the packet header which can be matched.
> > +
> > +For the \field{type} of VIRTIO_NET_FF_ETH_HDR, header fields are
> > +represented by a bitmap in \field{fields_bmap} are following:
> > +
> > +\begin{tabular}{|l|l|l|}
> > +\hline
> > +Bit & Name & Description \\
> > +\hline \hline
> > +0   & VIRTIO_NET_FF_DST_MAC & Destination MAC address in the packet \\
> > +\hline
> > +1   & VIRTIO_NET_FF_SRC_MAC & Source MAC address in the packet \\
> > +\hline
> > +2   & VIRTIO_NET_FF_ETHER_TYPE & Ether type in the packet \\
> > +\hline
> > +other   & -    & reserved \\
> > +\hline
> > +\end{tabular}
> > +
> > +For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN tag fields
> > +are represented by a bitmap in \field{fields_bmap} are following:
> > +
> > +\begin{tabular}{|l|l|l|}
> > +\hline
> > +Bit & Name & Description \\
> > +\hline \hline
> > +0   & VIRTIO_NET_FF_VLAN_TAG_TCI & Vlan tag TCI 16-bit field \\
> > +\hline
> > +other   & -    & reserved \\
> > +\hline
> > +\end{tabular}
> > +
> > +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, header fields are
> > +represented by a bitmap in \field{fields_bmap} are following:
> > +
> > +\begin{tabular}{|l|l|l|}
> > +\hline
> > +Bit & Name & Description \\
> > +\hline \hline
> > +0   & VIRTIO_NET_FF_SRC_IPV4 & Source IPV4 address in the packet \\
> > +\hline
> > +1   & VIRTIO_NET_FF_DST_IPV4 & Destination IPV4 address in the packet \\
> > +\hline
> > +other   & -    & reserved \\
> > +\hline
> > +\end{tabular}
> > +
> > +For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header fields are
> > +represented by a bitmap in \field{fields_bmap} are following:
> > +
> > +\begin{tabular}{|l|l|l|}
> > +\hline
> > +Bit & Name & Description \\
> > +\hline \hline
> > +0   & VIRTIO_NET_FF_SRC_IPV6 & Source IPV6 address in the packet \\
> > +\hline
> > +1   & VIRTIO_NET_FF_DST_IPV6 & Destination IPV6 address in the packet \\
> > +\hline
> > +other   & -    & reserved \\
> > +\hline
> > +\end{tabular}
> > +
> > +For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header fields are
> > +represented by a bitmap in \field{fields_bmap} are following:
> > +
> > +\begin{tabular}{|l|l|l|}
> > +\hline
> > +Bit & Name & Description \\
> > +\hline \hline
> > +0   & VIRTIO_NET_FF_SRC_TCP_PORT & Source TCP port in the packet \\
> > +\hline
> > +1   & VIRTIO_NET_FF_DST_TCP_PORT & Destination TCP port in the packet
> \\
> > +\hline
> > +other   & -    & reserved \\
> > +\hline
> > +\end{tabular}
> > +
> > +For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header fields are
> > +represented by a bitmap in \field{fields_bmap} are following:
> > +
> > +\begin{tabular}{|l|l|l|}
> > +\hline
> > +Bit & Name & Description \\
> > +\hline \hline
> > +0   & VIRTIO_NET_FF_SRC_UDP_PORT & Source UDP port in the packet \\
> > +\hline
> > +1   & VIRTIO_NET_FF_DST_UDP_PORT & Destination UDP port in the packet
> \\
> > +\hline
> > +other   & -    & reserved  \\
> > +\hline
> > +\end{tabular}
> > +
> 
> This is such an elaborate structure to report just 12 read only bits.
> Please let's just follow the example of  le32 supported_tunnel_types and add
> l32 supported_flow_control.
> 
supported_tunnel_types was let go because cvq is efficient.
None of these fields are needed for init time configuration of the driver before DRIVER_OK.

It is a very narrow view of 12 bits. It is going to grow and many.
This is far more structured for each type done here.

> 
> After you were trying to add kilobytes to megabytes of memory - I see little
> reason to save 12 RO bits that can be shared by any number of VFs.
> 
Completely wrong reason and very late review and also does not align with every other command we did.

> However, I do think we should create an option to access config space over
> DMA (preferably admin commands). Let's add this quickly and then we don't
> need to worry about legacy guests accessing flow filter through MMIO.

CVQ is already there as single interface forget and set for rss, rss context, vq moderation, statistics, flow filter caps and more.
No reason to bifurcate.
I won't be able to absorb this comment of DMA interface.
If I discuss further, I will repeat the whole document [1] and I will avoid that now.

[1] https://docs.google.com/document/d/1Iyn-l3Nm0yls3pZaul4lZiVj8x1s73Ed6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Wed, Nov 22, 2023 at 02:10:29PM +0000, Parav Pandit wrote:
> 
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Wednesday, November 22, 2023 7:32 PM
> > 
> > On Fri, Nov 10, 2023 at 02:38:50PM +0200, Parav Pandit wrote:
> > > The device responds flow filter capabilities using two commands.
> > > One command indicates generic flow filter device limits such as number
> > > of flow filters, number of flow filter groups, support or multiple
> > > transports etc.
> > >
> > > The second command indicates supported match types, and fields of the
> > > packet.
> > >
> > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
> > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > Signed-off-by: Parav Pandit <parav@nvidia.com>
> > >
> > > ---
> > > changelog:
> > > v2->v3:
> > > - rebased on virtio-1.4 branch
> > > - removed reference for flow filter virtqueue
> > > v1->v2:
> > > - addressed comments from Satananda
> > > - added vlan type match field
> > > - kept space for types between l2, l3, l4 header match types
> > > - renamed mask to mask_supported with shorter width
> > > - made more fields reserved for furture
> > > - addressed comments from Heng
> > > - grammar correction
> > > - added field to indicate supported number of actions per flow
> > >   filter match entry
> > > - added missing documentation for max_flow_priorities_per_group
> > > v0->v1:
> > > - added mask field in the type to indicate supported mask by device
> > >   and also in later patch to use it to indicate mask on adding
> > >   flow filter. As a result removed the mask_supported capability
> > >   field
> > > ---
> > >  device-types/net/description.tex | 208
> > > ++++++++++++++++++++++++++++++-
> > >  1 file changed, 206 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/device-types/net/description.tex
> > > b/device-types/net/description.tex
> > > index 30220b5..eccd8d6 100644
> > > --- a/device-types/net/description.tex
> > > +++ b/device-types/net/description.tex
> > > @@ -1173,7 +1173,11 @@ \subsubsection{Flow Filter}\label{sec:Device
> > > Types / Network Device / Device Ope
> > >
> > >  The device indicates the flow filter capabilities to the driver.
> > > These  capabilities include various maximum device limits and
> > > -supported packet match fields.
> > > +supported packet match fields. These control virtqueue commands are:
> > > +\ref{sec:Device Types / Network Device / Device Operation / Control
> > > +Virtqueue / Flow Filter / Flow Filter Capabilities Get} and
> > > +\ref{sec:Device Types / Network Device / Device Operation / Control
> > Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}.
> > >
> > >  The flow filters are grouped using a flow filter group. Each flow
> > > filter  group has a priority. The device first applies the flow
> > > filters of the highest @@ -1224,7 +1228,136 @@ \subsubsection{Flow
> > Filter}\label{sec:Device Types / Network Device / Device Ope
> > >        the flow filters in group_C, the flow filters of next level group_B are
> > applied.
> > >  \end{itemize}
> > >
> > > -\label{sec:Device Types / Network Device / Device Operation / Control
> > > Virtqueue / Setting Promiscuous Mode}%old label for latexdiff
> > > +\paragraph{Match Types and Fields}\label{sec:Device Types / Network
> > > +Device / Device Operation / Flow Filter / Match Types and Fields}
> > > +
> > > +\begin{lstlisting}
> > > +struct virtio_net_ff_match_type_cap {
> > > +        le16 type;
> > > +        u8 mask_supported;
> > > +        u8 reserved[5];
> > > +        le64 fields_bmap;
> > > +};
> > > +\end{lstlisting}
> > > +
> > > +The \field{type} corresponds to following table:
> > > +
> > > +\begin{tabular}{|l|l|l|}
> > > +\hline
> > > +Type & Name & Description \\
> > > +\hline \hline
> > > +0   & VIRTIO_NET_FF_ETH_HDR & Ethernet header of the packet \\
> > > +\hline
> > > +0x1   & VIRTIO_NET_FF_VLAN_TAG_HDR & VLAN tag of the packet \\
> > > +\hline
> > > +0x200   & VIRTIO_NET_FF_IPV4_HDR & IPv4 header of the packet \\
> > > +\hline
> > > +0x300   & VIRTIO_NET_FF_IPV6_HDR & IPv6 header of the packet \\
> > > +\hline
> > > +0x400   & VIRTIO_NET_FF_TCP_HDR & TCP header of the packet \\
> > > +\hline
> > > +0x500   & VIRTIO_NET_FF_UDP_HDR & UDP header of the packet \\
> > > +\hline
> > > +other   & -    & reserved \\
> > > +\hline
> > > +\end{tabular}
> > > +
> > > +When the \field{mask_supported} is set, for the specific
> > > +\field{type}, the device can perform masking packet fields with the
> > > +mask supplied in the flow filter match entry.
> > > +
> > > +For each \field{type} the \field{fields_bmap} indicates supported
> > > +fields of the packet header which can be matched.
> > > +
> > > +For the \field{type} of VIRTIO_NET_FF_ETH_HDR, header fields are
> > > +represented by a bitmap in \field{fields_bmap} are following:
> > > +
> > > +\begin{tabular}{|l|l|l|}
> > > +\hline
> > > +Bit & Name & Description \\
> > > +\hline \hline
> > > +0   & VIRTIO_NET_FF_DST_MAC & Destination MAC address in the packet \\
> > > +\hline
> > > +1   & VIRTIO_NET_FF_SRC_MAC & Source MAC address in the packet \\
> > > +\hline
> > > +2   & VIRTIO_NET_FF_ETHER_TYPE & Ether type in the packet \\
> > > +\hline
> > > +other   & -    & reserved \\
> > > +\hline
> > > +\end{tabular}
> > > +
> > > +For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN tag fields
> > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > +
> > > +\begin{tabular}{|l|l|l|}
> > > +\hline
> > > +Bit & Name & Description \\
> > > +\hline \hline
> > > +0   & VIRTIO_NET_FF_VLAN_TAG_TCI & Vlan tag TCI 16-bit field \\
> > > +\hline
> > > +other   & -    & reserved \\
> > > +\hline
> > > +\end{tabular}
> > > +
> > > +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, header fields are
> > > +represented by a bitmap in \field{fields_bmap} are following:
> > > +
> > > +\begin{tabular}{|l|l|l|}
> > > +\hline
> > > +Bit & Name & Description \\
> > > +\hline \hline
> > > +0   & VIRTIO_NET_FF_SRC_IPV4 & Source IPV4 address in the packet \\
> > > +\hline
> > > +1   & VIRTIO_NET_FF_DST_IPV4 & Destination IPV4 address in the packet \\
> > > +\hline
> > > +other   & -    & reserved \\
> > > +\hline
> > > +\end{tabular}
> > > +
> > > +For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header fields are
> > > +represented by a bitmap in \field{fields_bmap} are following:
> > > +
> > > +\begin{tabular}{|l|l|l|}
> > > +\hline
> > > +Bit & Name & Description \\
> > > +\hline \hline
> > > +0   & VIRTIO_NET_FF_SRC_IPV6 & Source IPV6 address in the packet \\
> > > +\hline
> > > +1   & VIRTIO_NET_FF_DST_IPV6 & Destination IPV6 address in the packet \\
> > > +\hline
> > > +other   & -    & reserved \\
> > > +\hline
> > > +\end{tabular}
> > > +
> > > +For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header fields are
> > > +represented by a bitmap in \field{fields_bmap} are following:
> > > +
> > > +\begin{tabular}{|l|l|l|}
> > > +\hline
> > > +Bit & Name & Description \\
> > > +\hline \hline
> > > +0   & VIRTIO_NET_FF_SRC_TCP_PORT & Source TCP port in the packet \\
> > > +\hline
> > > +1   & VIRTIO_NET_FF_DST_TCP_PORT & Destination TCP port in the packet
> > \\
> > > +\hline
> > > +other   & -    & reserved \\
> > > +\hline
> > > +\end{tabular}
> > > +
> > > +For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header fields are
> > > +represented by a bitmap in \field{fields_bmap} are following:
> > > +
> > > +\begin{tabular}{|l|l|l|}
> > > +\hline
> > > +Bit & Name & Description \\
> > > +\hline \hline
> > > +0   & VIRTIO_NET_FF_SRC_UDP_PORT & Source UDP port in the packet \\
> > > +\hline
> > > +1   & VIRTIO_NET_FF_DST_UDP_PORT & Destination UDP port in the packet
> > \\
> > > +\hline
> > > +other   & -    & reserved  \\
> > > +\hline
> > > +\end{tabular}
> > > +
> > 
> > This is such an elaborate structure to report just 12 read only bits.
> > Please let's just follow the example of  le32 supported_tunnel_types and add
> > l32 supported_flow_control.
> > 
> supported_tunnel_types was let go because cvq is efficient.
> None of these fields are needed for init time configuration of the driver before DRIVER_OK.

I really basically disagree. Whether control flow is supported can
easily influence how many VQs are needed.


> 
> It is a very narrow view of 12 bits. It is going to grow and many.
> This is far more structured for each type done here.
> 
> > 
> > After you were trying to add kilobytes to megabytes of memory - I see little
> > reason to save 12 RO bits that can be shared by any number of VFs.
> > 
> Completely wrong reason and very late review and also does not align with every other command we did.

which other command? hash and rss are like this: capability in config
space config through cvq. For the same reason.

> > However, I do think we should create an option to access config space over
> > DMA (preferably admin commands). Let's add this quickly and then we don't
> > need to worry about legacy guests accessing flow filter through MMIO.
> 
> CVQ is already there as single interface forget and set for rss, rss context, vq moderation, statistics, flow filter caps and more.
> No reason to bifurcate.

The reason is so that we have a single consistent view where e.g. you want
to provision a device with a specific capability you just
specify how its config space looks.

If you start shuffling capabilities off to VQs that will be much
much harder.

> I won't be able to absorb this comment of DMA interface.
> If I discuss further, I will repeat the whole document [1] and I will avoid that now.
> 
> [1] https://docs.google.com/document/d/1Iyn-l3Nm0yls3pZaul4lZiVj8x1s73Ed6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr


I really worry about how provisioning will work. And I do not at all
cherish replicating all of these query capability commands for provisioning.
Instead, I propose commands for config space access to solve everything
in one go, for all device types.

-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] RE: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, November 22, 2023 8:21 PM
> 
> On Wed, Nov 22, 2023 at 02:10:29PM +0000, Parav Pandit wrote:
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Wednesday, November 22, 2023 7:32 PM
> > >
> > > On Fri, Nov 10, 2023 at 02:38:50PM +0200, Parav Pandit wrote:
> > > > The device responds flow filter capabilities using two commands.
> > > > One command indicates generic flow filter device limits such as
> > > > number of flow filters, number of flow filter groups, support or
> > > > multiple transports etc.
> > > >
> > > > The second command indicates supported match types, and fields of
> > > > the packet.
> > > >
> > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
> > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > > Signed-off-by: Parav Pandit <parav@nvidia.com>
> > > >
> > > > ---
> > > > changelog:
> > > > v2->v3:
> > > > - rebased on virtio-1.4 branch
> > > > - removed reference for flow filter virtqueue
> > > > v1->v2:
> > > > - addressed comments from Satananda
> > > > - added vlan type match field
> > > > - kept space for types between l2, l3, l4 header match types
> > > > - renamed mask to mask_supported with shorter width
> > > > - made more fields reserved for furture
> > > > - addressed comments from Heng
> > > > - grammar correction
> > > > - added field to indicate supported number of actions per flow
> > > >   filter match entry
> > > > - added missing documentation for max_flow_priorities_per_group
> > > > v0->v1:
> > > > - added mask field in the type to indicate supported mask by device
> > > >   and also in later patch to use it to indicate mask on adding
> > > >   flow filter. As a result removed the mask_supported capability
> > > >   field
> > > > ---
> > > >  device-types/net/description.tex | 208
> > > > ++++++++++++++++++++++++++++++-
> > > >  1 file changed, 206 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/device-types/net/description.tex
> > > > b/device-types/net/description.tex
> > > > index 30220b5..eccd8d6 100644
> > > > --- a/device-types/net/description.tex
> > > > +++ b/device-types/net/description.tex
> > > > @@ -1173,7 +1173,11 @@ \subsubsection{Flow
> > > > Filter}\label{sec:Device Types / Network Device / Device Ope
> > > >
> > > >  The device indicates the flow filter capabilities to the driver.
> > > > These  capabilities include various maximum device limits and
> > > > -supported packet match fields.
> > > > +supported packet match fields. These control virtqueue commands are:
> > > > +\ref{sec:Device Types / Network Device / Device Operation /
> > > > +Control Virtqueue / Flow Filter / Flow Filter Capabilities Get}
> > > > +and \ref{sec:Device Types / Network Device / Device Operation /
> > > > +Control
> > > Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}.
> > > >
> > > >  The flow filters are grouped using a flow filter group. Each flow
> > > > filter  group has a priority. The device first applies the flow
> > > > filters of the highest @@ -1224,7 +1228,136 @@ \subsubsection{Flow
> > > Filter}\label{sec:Device Types / Network Device / Device Ope
> > > >        the flow filters in group_C, the flow filters of next level
> > > > group_B are
> > > applied.
> > > >  \end{itemize}
> > > >
> > > > -\label{sec:Device Types / Network Device / Device Operation /
> > > > Control Virtqueue / Setting Promiscuous Mode}%old label for
> > > > latexdiff
> > > > +\paragraph{Match Types and Fields}\label{sec:Device Types /
> > > > +Network Device / Device Operation / Flow Filter / Match Types and
> > > > +Fields}
> > > > +
> > > > +\begin{lstlisting}
> > > > +struct virtio_net_ff_match_type_cap {
> > > > +        le16 type;
> > > > +        u8 mask_supported;
> > > > +        u8 reserved[5];
> > > > +        le64 fields_bmap;
> > > > +};
> > > > +\end{lstlisting}
> > > > +
> > > > +The \field{type} corresponds to following table:
> > > > +
> > > > +\begin{tabular}{|l|l|l|}
> > > > +\hline
> > > > +Type & Name & Description \\
> > > > +\hline \hline
> > > > +0   & VIRTIO_NET_FF_ETH_HDR & Ethernet header of the packet \\
> > > > +\hline
> > > > +0x1   & VIRTIO_NET_FF_VLAN_TAG_HDR & VLAN tag of the packet \\
> > > > +\hline
> > > > +0x200   & VIRTIO_NET_FF_IPV4_HDR & IPv4 header of the packet \\
> > > > +\hline
> > > > +0x300   & VIRTIO_NET_FF_IPV6_HDR & IPv6 header of the packet \\
> > > > +\hline
> > > > +0x400   & VIRTIO_NET_FF_TCP_HDR & TCP header of the packet \\
> > > > +\hline
> > > > +0x500   & VIRTIO_NET_FF_UDP_HDR & UDP header of the packet \\
> > > > +\hline
> > > > +other   & -    & reserved \\
> > > > +\hline
> > > > +\end{tabular}
> > > > +
> > > > +When the \field{mask_supported} is set, for the specific
> > > > +\field{type}, the device can perform masking packet fields with
> > > > +the mask supplied in the flow filter match entry.
> > > > +
> > > > +For each \field{type} the \field{fields_bmap} indicates supported
> > > > +fields of the packet header which can be matched.
> > > > +
> > > > +For the \field{type} of VIRTIO_NET_FF_ETH_HDR, header fields are
> > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > +
> > > > +\begin{tabular}{|l|l|l|}
> > > > +\hline
> > > > +Bit & Name & Description \\
> > > > +\hline \hline
> > > > +0   & VIRTIO_NET_FF_DST_MAC & Destination MAC address in the
> packet \\
> > > > +\hline
> > > > +1   & VIRTIO_NET_FF_SRC_MAC & Source MAC address in the packet \\
> > > > +\hline
> > > > +2   & VIRTIO_NET_FF_ETHER_TYPE & Ether type in the packet \\
> > > > +\hline
> > > > +other   & -    & reserved \\
> > > > +\hline
> > > > +\end{tabular}
> > > > +
> > > > +For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN tag
> > > > +fields are represented by a bitmap in \field{fields_bmap} are following:
> > > > +
> > > > +\begin{tabular}{|l|l|l|}
> > > > +\hline
> > > > +Bit & Name & Description \\
> > > > +\hline \hline
> > > > +0   & VIRTIO_NET_FF_VLAN_TAG_TCI & Vlan tag TCI 16-bit field \\
> > > > +\hline
> > > > +other   & -    & reserved \\
> > > > +\hline
> > > > +\end{tabular}
> > > > +
> > > > +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, header fields are
> > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > +
> > > > +\begin{tabular}{|l|l|l|}
> > > > +\hline
> > > > +Bit & Name & Description \\
> > > > +\hline \hline
> > > > +0   & VIRTIO_NET_FF_SRC_IPV4 & Source IPV4 address in the packet \\
> > > > +\hline
> > > > +1   & VIRTIO_NET_FF_DST_IPV4 & Destination IPV4 address in the
> packet \\
> > > > +\hline
> > > > +other   & -    & reserved \\
> > > > +\hline
> > > > +\end{tabular}
> > > > +
> > > > +For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header fields are
> > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > +
> > > > +\begin{tabular}{|l|l|l|}
> > > > +\hline
> > > > +Bit & Name & Description \\
> > > > +\hline \hline
> > > > +0   & VIRTIO_NET_FF_SRC_IPV6 & Source IPV6 address in the packet \\
> > > > +\hline
> > > > +1   & VIRTIO_NET_FF_DST_IPV6 & Destination IPV6 address in the
> packet \\
> > > > +\hline
> > > > +other   & -    & reserved \\
> > > > +\hline
> > > > +\end{tabular}
> > > > +
> > > > +For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header fields are
> > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > +
> > > > +\begin{tabular}{|l|l|l|}
> > > > +\hline
> > > > +Bit & Name & Description \\
> > > > +\hline \hline
> > > > +0   & VIRTIO_NET_FF_SRC_TCP_PORT & Source TCP port in the packet \\
> > > > +\hline
> > > > +1   & VIRTIO_NET_FF_DST_TCP_PORT & Destination TCP port in the
> packet
> > > \\
> > > > +\hline
> > > > +other   & -    & reserved \\
> > > > +\hline
> > > > +\end{tabular}
> > > > +
> > > > +For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header fields are
> > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > +
> > > > +\begin{tabular}{|l|l|l|}
> > > > +\hline
> > > > +Bit & Name & Description \\
> > > > +\hline \hline
> > > > +0   & VIRTIO_NET_FF_SRC_UDP_PORT & Source UDP port in the packet
> \\
> > > > +\hline
> > > > +1   & VIRTIO_NET_FF_DST_UDP_PORT & Destination UDP port in the
> packet
> > > \\
> > > > +\hline
> > > > +other   & -    & reserved  \\
> > > > +\hline
> > > > +\end{tabular}
> > > > +
> > >
> > > This is such an elaborate structure to report just 12 read only bits.
> > > Please let's just follow the example of  le32 supported_tunnel_types
> > > and add
> > > l32 supported_flow_control.
> > >
> > supported_tunnel_types was let go because cvq is efficient.
> > None of these fields are needed for init time configuration of the driver
> before DRIVER_OK.
> 
> I really basically disagree. Whether control flow is supported can easily
> influence how many VQs are needed.

I don't understand this line.
Can you share example?

> 
> 
> >
> > It is a very narrow view of 12 bits. It is going to grow and many.
> > This is far more structured for each type done here.
> >
> > >
> > > After you were trying to add kilobytes to megabytes of memory - I
> > > see little reason to save 12 RO bits that can be shared by any number of
> VFs.
> > >
> > Completely wrong reason and very late review and also does not align with
> every other command we did.
> 
> which other command? hash and rss are like this: capability in config space
> config through cvq. For the same reason.
> 
> > > However, I do think we should create an option to access config
> > > space over DMA (preferably admin commands). Let's add this quickly
> > > and then we don't need to worry about legacy guests accessing flow filter
> through MMIO.
> >
> > CVQ is already there as single interface forget and set for rss, rss context, vq
> moderation, statistics, flow filter caps and more.
> > No reason to bifurcate.
> 
> The reason is so that we have a single consistent view where e.g. you want to
> provision a device with a specific capability you just specify how its config
> space looks.
Provisioning will have single consistent way to provision, some will sit in config space due to history, some will not be.

> 
> If you start shuffling capabilities off to VQs that will be much much harder.
Not sure harder for which system component.

> 
> > I won't be able to absorb this comment of DMA interface.
> > If I discuss further, I will repeat the whole document [1] and I will avoid that
> now.
> >
> > [1]
> > https://docs.google.com/document/d/1Iyn-
> l3Nm0yls3pZaul4lZiVj8x1s73Ed6r
> > Osmn6LfXc/edit#heading=h.qexbtyc2jpwr
> 
> 
> I really worry about how provisioning will work. And I do not at all cherish
> replicating all of these query capability commands for provisioning.
> Instead, I propose commands for config space access to solve everything in
> one go, for all device types.
Querying the new capabilities via the dma interface is same as using CVQ with disadvantage of creating yet more registers.

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] RE: [PATCH v6 0/5] virtio-net: Support flow filter for receive packets

[Parav Pandit]

Hi Cornelia,

> From: Cornelia Huck <cohuck@redhat.com>
> Sent: Wednesday, November 22, 2023 4:57 PM

[..]

> etc.) Would you prefer a respin, or a vote now and an editorial patch on top?
> I can either comment now, or start voting.

I fixed all your comments in v7 at [1].
They are bit better now.
Updated the change log of v7 and github link too.

[1] https://lists.oasis-open.org/archives/virtio-comment/202311/msg00500.html

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Jason Wang]

On Wed, Nov 22, 2023 at 10:51 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Wed, Nov 22, 2023 at 02:10:29PM +0000, Parav Pandit wrote:
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Wednesday, November 22, 2023 7:32 PM
> > >
> > > On Fri, Nov 10, 2023 at 02:38:50PM +0200, Parav Pandit wrote:
> > > > The device responds flow filter capabilities using two commands.
> > > > One command indicates generic flow filter device limits such as number
> > > > of flow filters, number of flow filter groups, support or multiple
> > > > transports etc.
> > > >
> > > > The second command indicates supported match types, and fields of the
> > > > packet.
> > > >
> > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
> > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > > Signed-off-by: Parav Pandit <parav@nvidia.com>
> > > >
> > > > ---
> > > > changelog:
> > > > v2->v3:
> > > > - rebased on virtio-1.4 branch
> > > > - removed reference for flow filter virtqueue
> > > > v1->v2:
> > > > - addressed comments from Satananda
> > > > - added vlan type match field
> > > > - kept space for types between l2, l3, l4 header match types
> > > > - renamed mask to mask_supported with shorter width
> > > > - made more fields reserved for furture
> > > > - addressed comments from Heng
> > > > - grammar correction
> > > > - added field to indicate supported number of actions per flow
> > > >   filter match entry
> > > > - added missing documentation for max_flow_priorities_per_group
> > > > v0->v1:
> > > > - added mask field in the type to indicate supported mask by device
> > > >   and also in later patch to use it to indicate mask on adding
> > > >   flow filter. As a result removed the mask_supported capability
> > > >   field
> > > > ---
> > > >  device-types/net/description.tex | 208
> > > > ++++++++++++++++++++++++++++++-
> > > >  1 file changed, 206 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/device-types/net/description.tex
> > > > b/device-types/net/description.tex
> > > > index 30220b5..eccd8d6 100644
> > > > --- a/device-types/net/description.tex
> > > > +++ b/device-types/net/description.tex
> > > > @@ -1173,7 +1173,11 @@ \subsubsection{Flow Filter}\label{sec:Device
> > > > Types / Network Device / Device Ope
> > > >
> > > >  The device indicates the flow filter capabilities to the driver.
> > > > These  capabilities include various maximum device limits and
> > > > -supported packet match fields.
> > > > +supported packet match fields. These control virtqueue commands are:
> > > > +\ref{sec:Device Types / Network Device / Device Operation / Control
> > > > +Virtqueue / Flow Filter / Flow Filter Capabilities Get} and
> > > > +\ref{sec:Device Types / Network Device / Device Operation / Control
> > > Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}.
> > > >
> > > >  The flow filters are grouped using a flow filter group. Each flow
> > > > filter  group has a priority. The device first applies the flow
> > > > filters of the highest @@ -1224,7 +1228,136 @@ \subsubsection{Flow
> > > Filter}\label{sec:Device Types / Network Device / Device Ope
> > > >        the flow filters in group_C, the flow filters of next level group_B are
> > > applied.
> > > >  \end{itemize}
> > > >
> > > > -\label{sec:Device Types / Network Device / Device Operation / Control
> > > > Virtqueue / Setting Promiscuous Mode}%old label for latexdiff
> > > > +\paragraph{Match Types and Fields}\label{sec:Device Types / Network
> > > > +Device / Device Operation / Flow Filter / Match Types and Fields}
> > > > +
> > > > +\begin{lstlisting}
> > > > +struct virtio_net_ff_match_type_cap {
> > > > +        le16 type;
> > > > +        u8 mask_supported;
> > > > +        u8 reserved[5];
> > > > +        le64 fields_bmap;
> > > > +};
> > > > +\end{lstlisting}
> > > > +
> > > > +The \field{type} corresponds to following table:
> > > > +
> > > > +\begin{tabular}{|l|l|l|}
> > > > +\hline
> > > > +Type & Name & Description \\
> > > > +\hline \hline
> > > > +0   & VIRTIO_NET_FF_ETH_HDR & Ethernet header of the packet \\
> > > > +\hline
> > > > +0x1   & VIRTIO_NET_FF_VLAN_TAG_HDR & VLAN tag of the packet \\
> > > > +\hline
> > > > +0x200   & VIRTIO_NET_FF_IPV4_HDR & IPv4 header of the packet \\
> > > > +\hline
> > > > +0x300   & VIRTIO_NET_FF_IPV6_HDR & IPv6 header of the packet \\
> > > > +\hline
> > > > +0x400   & VIRTIO_NET_FF_TCP_HDR & TCP header of the packet \\
> > > > +\hline
> > > > +0x500   & VIRTIO_NET_FF_UDP_HDR & UDP header of the packet \\
> > > > +\hline
> > > > +other   & -    & reserved \\
> > > > +\hline
> > > > +\end{tabular}
> > > > +
> > > > +When the \field{mask_supported} is set, for the specific
> > > > +\field{type}, the device can perform masking packet fields with the
> > > > +mask supplied in the flow filter match entry.
> > > > +
> > > > +For each \field{type} the \field{fields_bmap} indicates supported
> > > > +fields of the packet header which can be matched.
> > > > +
> > > > +For the \field{type} of VIRTIO_NET_FF_ETH_HDR, header fields are
> > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > +
> > > > +\begin{tabular}{|l|l|l|}
> > > > +\hline
> > > > +Bit & Name & Description \\
> > > > +\hline \hline
> > > > +0   & VIRTIO_NET_FF_DST_MAC & Destination MAC address in the packet \\
> > > > +\hline
> > > > +1   & VIRTIO_NET_FF_SRC_MAC & Source MAC address in the packet \\
> > > > +\hline
> > > > +2   & VIRTIO_NET_FF_ETHER_TYPE & Ether type in the packet \\
> > > > +\hline
> > > > +other   & -    & reserved \\
> > > > +\hline
> > > > +\end{tabular}
> > > > +
> > > > +For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN tag fields
> > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > +
> > > > +\begin{tabular}{|l|l|l|}
> > > > +\hline
> > > > +Bit & Name & Description \\
> > > > +\hline \hline
> > > > +0   & VIRTIO_NET_FF_VLAN_TAG_TCI & Vlan tag TCI 16-bit field \\
> > > > +\hline
> > > > +other   & -    & reserved \\
> > > > +\hline
> > > > +\end{tabular}
> > > > +
> > > > +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, header fields are
> > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > +
> > > > +\begin{tabular}{|l|l|l|}
> > > > +\hline
> > > > +Bit & Name & Description \\
> > > > +\hline \hline
> > > > +0   & VIRTIO_NET_FF_SRC_IPV4 & Source IPV4 address in the packet \\
> > > > +\hline
> > > > +1   & VIRTIO_NET_FF_DST_IPV4 & Destination IPV4 address in the packet \\
> > > > +\hline
> > > > +other   & -    & reserved \\
> > > > +\hline
> > > > +\end{tabular}
> > > > +
> > > > +For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header fields are
> > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > +
> > > > +\begin{tabular}{|l|l|l|}
> > > > +\hline
> > > > +Bit & Name & Description \\
> > > > +\hline \hline
> > > > +0   & VIRTIO_NET_FF_SRC_IPV6 & Source IPV6 address in the packet \\
> > > > +\hline
> > > > +1   & VIRTIO_NET_FF_DST_IPV6 & Destination IPV6 address in the packet \\
> > > > +\hline
> > > > +other   & -    & reserved \\
> > > > +\hline
> > > > +\end{tabular}
> > > > +
> > > > +For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header fields are
> > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > +
> > > > +\begin{tabular}{|l|l|l|}
> > > > +\hline
> > > > +Bit & Name & Description \\
> > > > +\hline \hline
> > > > +0   & VIRTIO_NET_FF_SRC_TCP_PORT & Source TCP port in the packet \\
> > > > +\hline
> > > > +1   & VIRTIO_NET_FF_DST_TCP_PORT & Destination TCP port in the packet
> > > \\
> > > > +\hline
> > > > +other   & -    & reserved \\
> > > > +\hline
> > > > +\end{tabular}
> > > > +
> > > > +For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header fields are
> > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > +
> > > > +\begin{tabular}{|l|l|l|}
> > > > +\hline
> > > > +Bit & Name & Description \\
> > > > +\hline \hline
> > > > +0   & VIRTIO_NET_FF_SRC_UDP_PORT & Source UDP port in the packet \\
> > > > +\hline
> > > > +1   & VIRTIO_NET_FF_DST_UDP_PORT & Destination UDP port in the packet
> > > \\
> > > > +\hline
> > > > +other   & -    & reserved  \\
> > > > +\hline
> > > > +\end{tabular}
> > > > +
> > >
> > > This is such an elaborate structure to report just 12 read only bits.
> > > Please let's just follow the example of  le32 supported_tunnel_types and add
> > > l32 supported_flow_control.
> > >
> > supported_tunnel_types was let go because cvq is efficient.
> > None of these fields are needed for init time configuration of the driver before DRIVER_OK.
>
> I really basically disagree. Whether control flow is supported can
> easily influence how many VQs are needed.
>
>
> >
> > It is a very narrow view of 12 bits. It is going to grow and many.
> > This is far more structured for each type done here.
> >
> > >
> > > After you were trying to add kilobytes to megabytes of memory - I see little
> > > reason to save 12 RO bits that can be shared by any number of VFs.
> > >
> > Completely wrong reason and very late review and also does not align with every other command we did.
>
> which other command? hash and rss are like this: capability in config
> space config through cvq. For the same reason.
>
> > > However, I do think we should create an option to access config space over
> > > DMA (preferably admin commands). Let's add this quickly and then we don't
> > > need to worry about legacy guests accessing flow filter through MMIO.
> >
> > CVQ is already there as single interface forget and set for rss, rss context, vq moderation, statistics, flow filter caps and more.
> > No reason to bifurcate.
>
> The reason is so that we have a single consistent view where e.g. you want
> to provision a device with a specific capability you just
> specify how its config space looks.
>
> If you start shuffling capabilities off to VQs that will be much
> much harder.
>
> > I won't be able to absorb this comment of DMA interface.
> > If I discuss further, I will repeat the whole document [1] and I will avoid that now.
> >
> > [1] https://docs.google.com/document/d/1Iyn-l3Nm0yls3pZaul4lZiVj8x1s73Ed6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
>
>
> I really worry about how provisioning will work. And I do not at all
> cherish replicating all of these query capability commands for provisioning.

+1

There's nothing that prevents the config space from being implemented
in a way other than registers.

> Instead, I propose commands for config space access to solve everything
> in one go, for all device types.
>

Something similar like:

+\begin{lstlisting}
+#define VIRTIO_TRANSPTQ_CTRL_CONFIG    6
+  #define VIRTIO_TRANSPTQ_CTRL_CONFIG_GET    0
+  #define VIRTIO_TRANSPTQ_CTRL_CONFIG_SET    1
+
+struct transportq_ctrl_mdev_config_get {
+       u32 offset;
+       u32 size;
+};
+
+struct transportq_ctrl_mdev_config_set {
+       u32 offset;
+       u32 size;
+       u8  data[];
+};
+\end{lstlisting}

in the proposal of transport virtqueue.

https://lists.oasis-open.org/archives/virtio-comment/202208/msg00005.html

Thanks

> --
> MST
>
>
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
>


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Jason Wang <jasowang@redhat.com>
> Sent: Friday, November 24, 2023 9:32 AM
> 
> On Wed, Nov 22, 2023 at 10:51 PM Michael S. Tsirkin <mst@redhat.com>
> wrote:
> >
> > On Wed, Nov 22, 2023 at 02:10:29PM +0000, Parav Pandit wrote:
> > >
> > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > Sent: Wednesday, November 22, 2023 7:32 PM
> > > >
> > > > On Fri, Nov 10, 2023 at 02:38:50PM +0200, Parav Pandit wrote:
> > > > > The device responds flow filter capabilities using two commands.
> > > > > One command indicates generic flow filter device limits such as
> > > > > number of flow filters, number of flow filter groups, support or
> > > > > multiple transports etc.
> > > > >
> > > > > The second command indicates supported match types, and fields
> > > > > of the packet.
> > > > >
> > > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
> > > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > > > Signed-off-by: Parav Pandit <parav@nvidia.com>
> > > > >
> > > > > ---
> > > > > changelog:
> > > > > v2->v3:
> > > > > - rebased on virtio-1.4 branch
> > > > > - removed reference for flow filter virtqueue
> > > > > v1->v2:
> > > > > - addressed comments from Satananda
> > > > > - added vlan type match field
> > > > > - kept space for types between l2, l3, l4 header match types
> > > > > - renamed mask to mask_supported with shorter width
> > > > > - made more fields reserved for furture
> > > > > - addressed comments from Heng
> > > > > - grammar correction
> > > > > - added field to indicate supported number of actions per flow
> > > > >   filter match entry
> > > > > - added missing documentation for max_flow_priorities_per_group
> > > > > v0->v1:
> > > > > - added mask field in the type to indicate supported mask by device
> > > > >   and also in later patch to use it to indicate mask on adding
> > > > >   flow filter. As a result removed the mask_supported capability
> > > > >   field
> > > > > ---
> > > > >  device-types/net/description.tex | 208
> > > > > ++++++++++++++++++++++++++++++-
> > > > >  1 file changed, 206 insertions(+), 2 deletions(-)
> > > > >
> > > > > diff --git a/device-types/net/description.tex
> > > > > b/device-types/net/description.tex
> > > > > index 30220b5..eccd8d6 100644
> > > > > --- a/device-types/net/description.tex
> > > > > +++ b/device-types/net/description.tex
> > > > > @@ -1173,7 +1173,11 @@ \subsubsection{Flow
> > > > > Filter}\label{sec:Device Types / Network Device / Device Ope
> > > > >
> > > > >  The device indicates the flow filter capabilities to the driver.
> > > > > These  capabilities include various maximum device limits and
> > > > > -supported packet match fields.
> > > > > +supported packet match fields. These control virtqueue commands
> are:
> > > > > +\ref{sec:Device Types / Network Device / Device Operation /
> > > > > +Control Virtqueue / Flow Filter / Flow Filter Capabilities Get}
> > > > > +and \ref{sec:Device Types / Network Device / Device Operation /
> > > > > +Control
> > > > Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}.
> > > > >
> > > > >  The flow filters are grouped using a flow filter group. Each
> > > > > flow filter  group has a priority. The device first applies the
> > > > > flow filters of the highest @@ -1224,7 +1228,136 @@
> > > > > \subsubsection{Flow
> > > > Filter}\label{sec:Device Types / Network Device / Device Ope
> > > > >        the flow filters in group_C, the flow filters of next
> > > > > level group_B are
> > > > applied.
> > > > >  \end{itemize}
> > > > >
> > > > > -\label{sec:Device Types / Network Device / Device Operation /
> > > > > Control Virtqueue / Setting Promiscuous Mode}%old label for
> > > > > latexdiff
> > > > > +\paragraph{Match Types and Fields}\label{sec:Device Types /
> > > > > +Network Device / Device Operation / Flow Filter / Match Types
> > > > > +and Fields}
> > > > > +
> > > > > +\begin{lstlisting}
> > > > > +struct virtio_net_ff_match_type_cap {
> > > > > +        le16 type;
> > > > > +        u8 mask_supported;
> > > > > +        u8 reserved[5];
> > > > > +        le64 fields_bmap;
> > > > > +};
> > > > > +\end{lstlisting}
> > > > > +
> > > > > +The \field{type} corresponds to following table:
> > > > > +
> > > > > +\begin{tabular}{|l|l|l|}
> > > > > +\hline
> > > > > +Type & Name & Description \\
> > > > > +\hline \hline
> > > > > +0   & VIRTIO_NET_FF_ETH_HDR & Ethernet header of the packet \\
> > > > > +\hline
> > > > > +0x1   & VIRTIO_NET_FF_VLAN_TAG_HDR & VLAN tag of the packet \\
> > > > > +\hline
> > > > > +0x200   & VIRTIO_NET_FF_IPV4_HDR & IPv4 header of the packet \\
> > > > > +\hline
> > > > > +0x300   & VIRTIO_NET_FF_IPV6_HDR & IPv6 header of the packet \\
> > > > > +\hline
> > > > > +0x400   & VIRTIO_NET_FF_TCP_HDR & TCP header of the packet \\
> > > > > +\hline
> > > > > +0x500   & VIRTIO_NET_FF_UDP_HDR & UDP header of the packet \\
> > > > > +\hline
> > > > > +other   & -    & reserved \\
> > > > > +\hline
> > > > > +\end{tabular}
> > > > > +
> > > > > +When the \field{mask_supported} is set, for the specific
> > > > > +\field{type}, the device can perform masking packet fields with
> > > > > +the mask supplied in the flow filter match entry.
> > > > > +
> > > > > +For each \field{type} the \field{fields_bmap} indicates
> > > > > +supported fields of the packet header which can be matched.
> > > > > +
> > > > > +For the \field{type} of VIRTIO_NET_FF_ETH_HDR, header fields
> > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > +
> > > > > +\begin{tabular}{|l|l|l|}
> > > > > +\hline
> > > > > +Bit & Name & Description \\
> > > > > +\hline \hline
> > > > > +0   & VIRTIO_NET_FF_DST_MAC & Destination MAC address in the
> packet \\
> > > > > +\hline
> > > > > +1   & VIRTIO_NET_FF_SRC_MAC & Source MAC address in the packet
> \\
> > > > > +\hline
> > > > > +2   & VIRTIO_NET_FF_ETHER_TYPE & Ether type in the packet \\
> > > > > +\hline
> > > > > +other   & -    & reserved \\
> > > > > +\hline
> > > > > +\end{tabular}
> > > > > +
> > > > > +For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN tag
> > > > > +fields are represented by a bitmap in \field{fields_bmap} are
> following:
> > > > > +
> > > > > +\begin{tabular}{|l|l|l|}
> > > > > +\hline
> > > > > +Bit & Name & Description \\
> > > > > +\hline \hline
> > > > > +0   & VIRTIO_NET_FF_VLAN_TAG_TCI & Vlan tag TCI 16-bit field \\
> > > > > +\hline
> > > > > +other   & -    & reserved \\
> > > > > +\hline
> > > > > +\end{tabular}
> > > > > +
> > > > > +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, header fields
> > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > +
> > > > > +\begin{tabular}{|l|l|l|}
> > > > > +\hline
> > > > > +Bit & Name & Description \\
> > > > > +\hline \hline
> > > > > +0   & VIRTIO_NET_FF_SRC_IPV4 & Source IPV4 address in the packet
> \\
> > > > > +\hline
> > > > > +1   & VIRTIO_NET_FF_DST_IPV4 & Destination IPV4 address in the
> packet \\
> > > > > +\hline
> > > > > +other   & -    & reserved \\
> > > > > +\hline
> > > > > +\end{tabular}
> > > > > +
> > > > > +For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header fields
> > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > +
> > > > > +\begin{tabular}{|l|l|l|}
> > > > > +\hline
> > > > > +Bit & Name & Description \\
> > > > > +\hline \hline
> > > > > +0   & VIRTIO_NET_FF_SRC_IPV6 & Source IPV6 address in the packet
> \\
> > > > > +\hline
> > > > > +1   & VIRTIO_NET_FF_DST_IPV6 & Destination IPV6 address in the
> packet \\
> > > > > +\hline
> > > > > +other   & -    & reserved \\
> > > > > +\hline
> > > > > +\end{tabular}
> > > > > +
> > > > > +For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header fields
> > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > +
> > > > > +\begin{tabular}{|l|l|l|}
> > > > > +\hline
> > > > > +Bit & Name & Description \\
> > > > > +\hline \hline
> > > > > +0   & VIRTIO_NET_FF_SRC_TCP_PORT & Source TCP port in the
> packet \\
> > > > > +\hline
> > > > > +1   & VIRTIO_NET_FF_DST_TCP_PORT & Destination TCP port in the
> packet
> > > > \\
> > > > > +\hline
> > > > > +other   & -    & reserved \\
> > > > > +\hline
> > > > > +\end{tabular}
> > > > > +
> > > > > +For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header fields
> > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > +
> > > > > +\begin{tabular}{|l|l|l|}
> > > > > +\hline
> > > > > +Bit & Name & Description \\
> > > > > +\hline \hline
> > > > > +0   & VIRTIO_NET_FF_SRC_UDP_PORT & Source UDP port in the
> packet \\
> > > > > +\hline
> > > > > +1   & VIRTIO_NET_FF_DST_UDP_PORT & Destination UDP port in the
> packet
> > > > \\
> > > > > +\hline
> > > > > +other   & -    & reserved  \\
> > > > > +\hline
> > > > > +\end{tabular}
> > > > > +
> > > >
> > > > This is such an elaborate structure to report just 12 read only bits.
> > > > Please let's just follow the example of  le32
> > > > supported_tunnel_types and add
> > > > l32 supported_flow_control.
> > > >
> > > supported_tunnel_types was let go because cvq is efficient.
> > > None of these fields are needed for init time configuration of the driver
> before DRIVER_OK.
> >
> > I really basically disagree. Whether control flow is supported can
> > easily influence how many VQs are needed.
> >
> >
> > >
> > > It is a very narrow view of 12 bits. It is going to grow and many.
> > > This is far more structured for each type done here.
> > >
> > > >
> > > > After you were trying to add kilobytes to megabytes of memory - I
> > > > see little reason to save 12 RO bits that can be shared by any number of
> VFs.
> > > >
> > > Completely wrong reason and very late review and also does not align
> with every other command we did.
> >
> > which other command? hash and rss are like this: capability in config
> > space config through cvq. For the same reason.
> >
> > > > However, I do think we should create an option to access config
> > > > space over DMA (preferably admin commands). Let's add this quickly
> > > > and then we don't need to worry about legacy guests accessing flow
> filter through MMIO.
> > >
> > > CVQ is already there as single interface forget and set for rss, rss context,
> vq moderation, statistics, flow filter caps and more.
> > > No reason to bifurcate.
> >
> > The reason is so that we have a single consistent view where e.g. you
> > want to provision a device with a specific capability you just specify
> > how its config space looks.
> >
> > If you start shuffling capabilities off to VQs that will be much much
> > harder.
> >
> > > I won't be able to absorb this comment of DMA interface.
> > > If I discuss further, I will repeat the whole document [1] and I will avoid
> that now.
> > >
> > > [1]
> > > https://docs.google.com/document/d/1Iyn-
> l3Nm0yls3pZaul4lZiVj8x1s73Ed
> > > 6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
> >
> >
> > I really worry about how provisioning will work. And I do not at all
> > cherish replicating all of these query capability commands for provisioning.
> 
> +1
>
 - 1

All points are discussed and not going to repeat 7th or 8th time.
 
> There's nothing that prevents the config space from being implemented in a
> way other than registers.
>
It does.
It is discussed 7th or 8th time.

All the points are discussed already.
[1] https://docs.google.com/document/d/1Iyn-l3Nm0yls3pZaul4lZiVj8x1s73Ed6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr

many things are done over cvq and statistics capabilities over cvq is the latest approved.
I wont be able to repeat the content of document [1] anymore.

> > Instead, I propose commands for config space access to solve
> > everything in one go, for all device types.
> >
> 
> Something similar like:
> 
> +\begin{lstlisting}
> +#define VIRTIO_TRANSPTQ_CTRL_CONFIG    6
> +  #define VIRTIO_TRANSPTQ_CTRL_CONFIG_GET    0
> +  #define VIRTIO_TRANSPTQ_CTRL_CONFIG_SET    1
> +
> +struct transportq_ctrl_mdev_config_get {
> +       u32 offset;
> +       u32 size;
> +};
> +
> +struct transportq_ctrl_mdev_config_set {
> +       u32 offset;
> +       u32 size;
> +       u8  data[];
> +};
> +\end{lstlisting}
> 
> in the proposal of transport virtqueue.
> 
> https://lists.oasis-open.org/archives/virtio-comment/202208/msg00005.html
> 
-1 as cvq already exists those services the role for caps and control far beyond tiny register-based interface.

As more features grow, tunneling commands through a I2C or SPI or GPIO kind of register interface is turning the clock back.
A command interface is way more extendible than register interface.

We wont be able to support this change that does not align with all the improvements done in 1.3 for notification coalescing, hash, stats and more.

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Fri, Nov 24, 2023 at 04:13:08AM +0000, Parav Pandit wrote:
> 
> 
> > From: Jason Wang <jasowang@redhat.com>
> > Sent: Friday, November 24, 2023 9:32 AM
> > 
> > On Wed, Nov 22, 2023 at 10:51 PM Michael S. Tsirkin <mst@redhat.com>
> > wrote:
> > >
> > > On Wed, Nov 22, 2023 at 02:10:29PM +0000, Parav Pandit wrote:
> > > >
> > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > Sent: Wednesday, November 22, 2023 7:32 PM
> > > > >
> > > > > On Fri, Nov 10, 2023 at 02:38:50PM +0200, Parav Pandit wrote:
> > > > > > The device responds flow filter capabilities using two commands.
> > > > > > One command indicates generic flow filter device limits such as
> > > > > > number of flow filters, number of flow filter groups, support or
> > > > > > multiple transports etc.
> > > > > >
> > > > > > The second command indicates supported match types, and fields
> > > > > > of the packet.
> > > > > >
> > > > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
> > > > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > > > > Signed-off-by: Parav Pandit <parav@nvidia.com>
> > > > > >
> > > > > > ---
> > > > > > changelog:
> > > > > > v2->v3:
> > > > > > - rebased on virtio-1.4 branch
> > > > > > - removed reference for flow filter virtqueue
> > > > > > v1->v2:
> > > > > > - addressed comments from Satananda
> > > > > > - added vlan type match field
> > > > > > - kept space for types between l2, l3, l4 header match types
> > > > > > - renamed mask to mask_supported with shorter width
> > > > > > - made more fields reserved for furture
> > > > > > - addressed comments from Heng
> > > > > > - grammar correction
> > > > > > - added field to indicate supported number of actions per flow
> > > > > >   filter match entry
> > > > > > - added missing documentation for max_flow_priorities_per_group
> > > > > > v0->v1:
> > > > > > - added mask field in the type to indicate supported mask by device
> > > > > >   and also in later patch to use it to indicate mask on adding
> > > > > >   flow filter. As a result removed the mask_supported capability
> > > > > >   field
> > > > > > ---
> > > > > >  device-types/net/description.tex | 208
> > > > > > ++++++++++++++++++++++++++++++-
> > > > > >  1 file changed, 206 insertions(+), 2 deletions(-)
> > > > > >
> > > > > > diff --git a/device-types/net/description.tex
> > > > > > b/device-types/net/description.tex
> > > > > > index 30220b5..eccd8d6 100644
> > > > > > --- a/device-types/net/description.tex
> > > > > > +++ b/device-types/net/description.tex
> > > > > > @@ -1173,7 +1173,11 @@ \subsubsection{Flow
> > > > > > Filter}\label{sec:Device Types / Network Device / Device Ope
> > > > > >
> > > > > >  The device indicates the flow filter capabilities to the driver.
> > > > > > These  capabilities include various maximum device limits and
> > > > > > -supported packet match fields.
> > > > > > +supported packet match fields. These control virtqueue commands
> > are:
> > > > > > +\ref{sec:Device Types / Network Device / Device Operation /
> > > > > > +Control Virtqueue / Flow Filter / Flow Filter Capabilities Get}
> > > > > > +and \ref{sec:Device Types / Network Device / Device Operation /
> > > > > > +Control
> > > > > Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}.
> > > > > >
> > > > > >  The flow filters are grouped using a flow filter group. Each
> > > > > > flow filter  group has a priority. The device first applies the
> > > > > > flow filters of the highest @@ -1224,7 +1228,136 @@
> > > > > > \subsubsection{Flow
> > > > > Filter}\label{sec:Device Types / Network Device / Device Ope
> > > > > >        the flow filters in group_C, the flow filters of next
> > > > > > level group_B are
> > > > > applied.
> > > > > >  \end{itemize}
> > > > > >
> > > > > > -\label{sec:Device Types / Network Device / Device Operation /
> > > > > > Control Virtqueue / Setting Promiscuous Mode}%old label for
> > > > > > latexdiff
> > > > > > +\paragraph{Match Types and Fields}\label{sec:Device Types /
> > > > > > +Network Device / Device Operation / Flow Filter / Match Types
> > > > > > +and Fields}
> > > > > > +
> > > > > > +\begin{lstlisting}
> > > > > > +struct virtio_net_ff_match_type_cap {
> > > > > > +        le16 type;
> > > > > > +        u8 mask_supported;
> > > > > > +        u8 reserved[5];
> > > > > > +        le64 fields_bmap;
> > > > > > +};
> > > > > > +\end{lstlisting}
> > > > > > +
> > > > > > +The \field{type} corresponds to following table:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Type & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_ETH_HDR & Ethernet header of the packet \\
> > > > > > +\hline
> > > > > > +0x1   & VIRTIO_NET_FF_VLAN_TAG_HDR & VLAN tag of the packet \\
> > > > > > +\hline
> > > > > > +0x200   & VIRTIO_NET_FF_IPV4_HDR & IPv4 header of the packet \\
> > > > > > +\hline
> > > > > > +0x300   & VIRTIO_NET_FF_IPV6_HDR & IPv6 header of the packet \\
> > > > > > +\hline
> > > > > > +0x400   & VIRTIO_NET_FF_TCP_HDR & TCP header of the packet \\
> > > > > > +\hline
> > > > > > +0x500   & VIRTIO_NET_FF_UDP_HDR & UDP header of the packet \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +When the \field{mask_supported} is set, for the specific
> > > > > > +\field{type}, the device can perform masking packet fields with
> > > > > > +the mask supplied in the flow filter match entry.
> > > > > > +
> > > > > > +For each \field{type} the \field{fields_bmap} indicates
> > > > > > +supported fields of the packet header which can be matched.
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_ETH_HDR, header fields
> > > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_DST_MAC & Destination MAC address in the
> > packet \\
> > > > > > +\hline
> > > > > > +1   & VIRTIO_NET_FF_SRC_MAC & Source MAC address in the packet
> > \\
> > > > > > +\hline
> > > > > > +2   & VIRTIO_NET_FF_ETHER_TYPE & Ether type in the packet \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN tag
> > > > > > +fields are represented by a bitmap in \field{fields_bmap} are
> > following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_VLAN_TAG_TCI & Vlan tag TCI 16-bit field \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, header fields
> > > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_SRC_IPV4 & Source IPV4 address in the packet
> > \\
> > > > > > +\hline
> > > > > > +1   & VIRTIO_NET_FF_DST_IPV4 & Destination IPV4 address in the
> > packet \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header fields
> > > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_SRC_IPV6 & Source IPV6 address in the packet
> > \\
> > > > > > +\hline
> > > > > > +1   & VIRTIO_NET_FF_DST_IPV6 & Destination IPV6 address in the
> > packet \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header fields
> > > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_SRC_TCP_PORT & Source TCP port in the
> > packet \\
> > > > > > +\hline
> > > > > > +1   & VIRTIO_NET_FF_DST_TCP_PORT & Destination TCP port in the
> > packet
> > > > > \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header fields
> > > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_SRC_UDP_PORT & Source UDP port in the
> > packet \\
> > > > > > +\hline
> > > > > > +1   & VIRTIO_NET_FF_DST_UDP_PORT & Destination UDP port in the
> > packet
> > > > > \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved  \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > >
> > > > > This is such an elaborate structure to report just 12 read only bits.
> > > > > Please let's just follow the example of  le32
> > > > > supported_tunnel_types and add
> > > > > l32 supported_flow_control.
> > > > >
> > > > supported_tunnel_types was let go because cvq is efficient.
> > > > None of these fields are needed for init time configuration of the driver
> > before DRIVER_OK.
> > >
> > > I really basically disagree. Whether control flow is supported can
> > > easily influence how many VQs are needed.
> > >
> > >
> > > >
> > > > It is a very narrow view of 12 bits. It is going to grow and many.
> > > > This is far more structured for each type done here.
> > > >
> > > > >
> > > > > After you were trying to add kilobytes to megabytes of memory - I
> > > > > see little reason to save 12 RO bits that can be shared by any number of
> > VFs.
> > > > >
> > > > Completely wrong reason and very late review and also does not align
> > with every other command we did.
> > >
> > > which other command? hash and rss are like this: capability in config
> > > space config through cvq. For the same reason.
> > >
> > > > > However, I do think we should create an option to access config
> > > > > space over DMA (preferably admin commands). Let's add this quickly
> > > > > and then we don't need to worry about legacy guests accessing flow
> > filter through MMIO.
> > > >
> > > > CVQ is already there as single interface forget and set for rss, rss context,
> > vq moderation, statistics, flow filter caps and more.
> > > > No reason to bifurcate.
> > >
> > > The reason is so that we have a single consistent view where e.g. you
> > > want to provision a device with a specific capability you just specify
> > > how its config space looks.
> > >
> > > If you start shuffling capabilities off to VQs that will be much much
> > > harder.
> > >
> > > > I won't be able to absorb this comment of DMA interface.
> > > > If I discuss further, I will repeat the whole document [1] and I will avoid
> > that now.
> > > >
> > > > [1]
> > > > https://docs.google.com/document/d/1Iyn-
> > l3Nm0yls3pZaul4lZiVj8x1s73Ed
> > > > 6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
> > >
> > >
> > > I really worry about how provisioning will work. And I do not at all
> > > cherish replicating all of these query capability commands for provisioning.
> > 
> > +1
> >
>  - 1
> 
> All points are discussed and not going to repeat 7th or 8th time.
>  
> > There's nothing that prevents the config space from being implemented in a
> > way other than registers.
> >
> It does.
> It is discussed 7th or 8th time.
> 
> All the points are discussed already.
> [1] https://docs.google.com/document/d/1Iyn-l3Nm0yls3pZaul4lZiVj8x1s73Ed6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
> 
> many things are done over cvq and statistics capabilities over cvq is the latest approved.
> I wont be able to repeat the content of document [1] anymore.

The famous Cons: None document.  Yea you pushed this through with
statistics because hey it's just statistics. Worst case devices can lie,
it is not clear worrying about statistics when provisioning is important
enough. And you are now under the impression everyone got convinced when
in reality everyone just got tired of arguing. This is not a good
pattern to try and repeat.  Let me repeat the relevant part here for
you:

	So why shouldn’t we just add some transport VQ on the owner device to transport the SIOV device’s configuration?

		Ans:
		Such addition means that hardware vendors need to build runtime configurations in 4 different ways.
		One way using CVQ for PCI PF and VFs
		2nd way as backward compatible SIOV using owner’s admin VQ
		3rd way using SIOV’s own CVQ channel for TDISP
		4th way using mix of backward compatible and secure and efficient way using #2 and #3.


This is just a straw-man argument. No - we make a capability optional, we
strongly suggest that *drivers* support both old and new mechanism, 
and then *devices* will only implement what's required.

-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Fri, Nov 24, 2023 at 12:02:23PM +0800, Jason Wang wrote:
> > > I won't be able to absorb this comment of DMA interface.
> > > If I discuss further, I will repeat the whole document [1] and I will avoid that now.
> > >
> > > [1] https://docs.google.com/document/d/1Iyn-l3Nm0yls3pZaul4lZiVj8x1s73Ed6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
> >
> >
> > I really worry about how provisioning will work. And I do not at all
> > cherish replicating all of these query capability commands for provisioning.
> 
> +1
> 
> There's nothing that prevents the config space from being implemented
> in a way other than registers.

Care doing it finally? Let's see if what Parav is worrying about is then
addressed.

-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Friday, November 24, 2023 11:01 AM
> 
> On Fri, Nov 24, 2023 at 04:13:08AM +0000, Parav Pandit wrote:
> >
> >
> > > From: Jason Wang <jasowang@redhat.com>
> > > Sent: Friday, November 24, 2023 9:32 AM
> > >
> > > On Wed, Nov 22, 2023 at 10:51 PM Michael S. Tsirkin <mst@redhat.com>
> > > wrote:
> > > >
> > > > On Wed, Nov 22, 2023 at 02:10:29PM +0000, Parav Pandit wrote:
> > > > >
> > > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > > Sent: Wednesday, November 22, 2023 7:32 PM
> > > > > >
> > > > > > On Fri, Nov 10, 2023 at 02:38:50PM +0200, Parav Pandit wrote:
> > > > > > > The device responds flow filter capabilities using two commands.
> > > > > > > One command indicates generic flow filter device limits such
> > > > > > > as number of flow filters, number of flow filter groups,
> > > > > > > support or multiple transports etc.
> > > > > > >
> > > > > > > The second command indicates supported match types, and
> > > > > > > fields of the packet.
> > > > > > >
> > > > > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
> > > > > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > > > > > Signed-off-by: Parav Pandit <parav@nvidia.com>
> > > > > > >
> > > > > > > ---
> > > > > > > changelog:
> > > > > > > v2->v3:
> > > > > > > - rebased on virtio-1.4 branch
> > > > > > > - removed reference for flow filter virtqueue
> > > > > > > v1->v2:
> > > > > > > - addressed comments from Satananda
> > > > > > > - added vlan type match field
> > > > > > > - kept space for types between l2, l3, l4 header match types
> > > > > > > - renamed mask to mask_supported with shorter width
> > > > > > > - made more fields reserved for furture
> > > > > > > - addressed comments from Heng
> > > > > > > - grammar correction
> > > > > > > - added field to indicate supported number of actions per flow
> > > > > > >   filter match entry
> > > > > > > - added missing documentation for
> > > > > > > max_flow_priorities_per_group
> > > > > > > v0->v1:
> > > > > > > - added mask field in the type to indicate supported mask by device
> > > > > > >   and also in later patch to use it to indicate mask on adding
> > > > > > >   flow filter. As a result removed the mask_supported capability
> > > > > > >   field
> > > > > > > ---
> > > > > > >  device-types/net/description.tex | 208
> > > > > > > ++++++++++++++++++++++++++++++-
> > > > > > >  1 file changed, 206 insertions(+), 2 deletions(-)
> > > > > > >
> > > > > > > diff --git a/device-types/net/description.tex
> > > > > > > b/device-types/net/description.tex
> > > > > > > index 30220b5..eccd8d6 100644
> > > > > > > --- a/device-types/net/description.tex
> > > > > > > +++ b/device-types/net/description.tex
> > > > > > > @@ -1173,7 +1173,11 @@ \subsubsection{Flow
> > > > > > > Filter}\label{sec:Device Types / Network Device / Device Ope
> > > > > > >
> > > > > > >  The device indicates the flow filter capabilities to the driver.
> > > > > > > These  capabilities include various maximum device limits
> > > > > > > and -supported packet match fields.
> > > > > > > +supported packet match fields. These control virtqueue
> > > > > > > +commands
> > > are:
> > > > > > > +\ref{sec:Device Types / Network Device / Device Operation /
> > > > > > > +Control Virtqueue / Flow Filter / Flow Filter Capabilities
> > > > > > > +Get} and \ref{sec:Device Types / Network Device / Device
> > > > > > > +Operation / Control
> > > > > > Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}.
> > > > > > >
> > > > > > >  The flow filters are grouped using a flow filter group.
> > > > > > > Each flow filter  group has a priority. The device first
> > > > > > > applies the flow filters of the highest @@ -1224,7 +1228,136
> > > > > > > @@ \subsubsection{Flow
> > > > > > Filter}\label{sec:Device Types / Network Device / Device Ope
> > > > > > >        the flow filters in group_C, the flow filters of next
> > > > > > > level group_B are
> > > > > > applied.
> > > > > > >  \end{itemize}
> > > > > > >
> > > > > > > -\label{sec:Device Types / Network Device / Device Operation
> > > > > > > / Control Virtqueue / Setting Promiscuous Mode}%old label
> > > > > > > for latexdiff
> > > > > > > +\paragraph{Match Types and Fields}\label{sec:Device Types /
> > > > > > > +Network Device / Device Operation / Flow Filter / Match
> > > > > > > +Types and Fields}
> > > > > > > +
> > > > > > > +\begin{lstlisting}
> > > > > > > +struct virtio_net_ff_match_type_cap {
> > > > > > > +        le16 type;
> > > > > > > +        u8 mask_supported;
> > > > > > > +        u8 reserved[5];
> > > > > > > +        le64 fields_bmap;
> > > > > > > +};
> > > > > > > +\end{lstlisting}
> > > > > > > +
> > > > > > > +The \field{type} corresponds to following table:
> > > > > > > +
> > > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > > +\hline
> > > > > > > +Type & Name & Description \\ \hline \hline
> > > > > > > +0   & VIRTIO_NET_FF_ETH_HDR & Ethernet header of the packet \\
> > > > > > > +\hline
> > > > > > > +0x1   & VIRTIO_NET_FF_VLAN_TAG_HDR & VLAN tag of the packet
> \\
> > > > > > > +\hline
> > > > > > > +0x200   & VIRTIO_NET_FF_IPV4_HDR & IPv4 header of the packet
> \\
> > > > > > > +\hline
> > > > > > > +0x300   & VIRTIO_NET_FF_IPV6_HDR & IPv6 header of the packet
> \\
> > > > > > > +\hline
> > > > > > > +0x400   & VIRTIO_NET_FF_TCP_HDR & TCP header of the packet \\
> > > > > > > +\hline
> > > > > > > +0x500   & VIRTIO_NET_FF_UDP_HDR & UDP header of the packet
> \\
> > > > > > > +\hline
> > > > > > > +other   & -    & reserved \\
> > > > > > > +\hline
> > > > > > > +\end{tabular}
> > > > > > > +
> > > > > > > +When the \field{mask_supported} is set, for the specific
> > > > > > > +\field{type}, the device can perform masking packet fields
> > > > > > > +with the mask supplied in the flow filter match entry.
> > > > > > > +
> > > > > > > +For each \field{type} the \field{fields_bmap} indicates
> > > > > > > +supported fields of the packet header which can be matched.
> > > > > > > +
> > > > > > > +For the \field{type} of VIRTIO_NET_FF_ETH_HDR, header
> > > > > > > +fields are represented by a bitmap in \field{fields_bmap} are
> following:
> > > > > > > +
> > > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > > +\hline
> > > > > > > +Bit & Name & Description \\ \hline \hline
> > > > > > > +0   & VIRTIO_NET_FF_DST_MAC & Destination MAC address in the
> > > packet \\
> > > > > > > +\hline
> > > > > > > +1   & VIRTIO_NET_FF_SRC_MAC & Source MAC address in the
> packet
> > > \\
> > > > > > > +\hline
> > > > > > > +2   & VIRTIO_NET_FF_ETHER_TYPE & Ether type in the packet \\
> > > > > > > +\hline
> > > > > > > +other   & -    & reserved \\
> > > > > > > +\hline
> > > > > > > +\end{tabular}
> > > > > > > +
> > > > > > > +For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN
> > > > > > > +tag fields are represented by a bitmap in
> > > > > > > +\field{fields_bmap} are
> > > following:
> > > > > > > +
> > > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > > +\hline
> > > > > > > +Bit & Name & Description \\ \hline \hline
> > > > > > > +0   & VIRTIO_NET_FF_VLAN_TAG_TCI & Vlan tag TCI 16-bit field \\
> > > > > > > +\hline
> > > > > > > +other   & -    & reserved \\
> > > > > > > +\hline
> > > > > > > +\end{tabular}
> > > > > > > +
> > > > > > > +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, header
> > > > > > > +fields are represented by a bitmap in \field{fields_bmap} are
> following:
> > > > > > > +
> > > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > > +\hline
> > > > > > > +Bit & Name & Description \\ \hline \hline
> > > > > > > +0   & VIRTIO_NET_FF_SRC_IPV4 & Source IPV4 address in the
> packet
> > > \\
> > > > > > > +\hline
> > > > > > > +1   & VIRTIO_NET_FF_DST_IPV4 & Destination IPV4 address in the
> > > packet \\
> > > > > > > +\hline
> > > > > > > +other   & -    & reserved \\
> > > > > > > +\hline
> > > > > > > +\end{tabular}
> > > > > > > +
> > > > > > > +For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header
> > > > > > > +fields are represented by a bitmap in \field{fields_bmap} are
> following:
> > > > > > > +
> > > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > > +\hline
> > > > > > > +Bit & Name & Description \\ \hline \hline
> > > > > > > +0   & VIRTIO_NET_FF_SRC_IPV6 & Source IPV6 address in the
> packet
> > > \\
> > > > > > > +\hline
> > > > > > > +1   & VIRTIO_NET_FF_DST_IPV6 & Destination IPV6 address in the
> > > packet \\
> > > > > > > +\hline
> > > > > > > +other   & -    & reserved \\
> > > > > > > +\hline
> > > > > > > +\end{tabular}
> > > > > > > +
> > > > > > > +For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header
> > > > > > > +fields are represented by a bitmap in \field{fields_bmap} are
> following:
> > > > > > > +
> > > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > > +\hline
> > > > > > > +Bit & Name & Description \\ \hline \hline
> > > > > > > +0   & VIRTIO_NET_FF_SRC_TCP_PORT & Source TCP port in the
> > > packet \\
> > > > > > > +\hline
> > > > > > > +1   & VIRTIO_NET_FF_DST_TCP_PORT & Destination TCP port in the
> > > packet
> > > > > > \\
> > > > > > > +\hline
> > > > > > > +other   & -    & reserved \\
> > > > > > > +\hline
> > > > > > > +\end{tabular}
> > > > > > > +
> > > > > > > +For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header
> > > > > > > +fields are represented by a bitmap in \field{fields_bmap} are
> following:
> > > > > > > +
> > > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > > +\hline
> > > > > > > +Bit & Name & Description \\ \hline \hline
> > > > > > > +0   & VIRTIO_NET_FF_SRC_UDP_PORT & Source UDP port in the
> > > packet \\
> > > > > > > +\hline
> > > > > > > +1   & VIRTIO_NET_FF_DST_UDP_PORT & Destination UDP port in
> the
> > > packet
> > > > > > \\
> > > > > > > +\hline
> > > > > > > +other   & -    & reserved  \\
> > > > > > > +\hline
> > > > > > > +\end{tabular}
> > > > > > > +
> > > > > >
> > > > > > This is such an elaborate structure to report just 12 read only bits.
> > > > > > Please let's just follow the example of  le32
> > > > > > supported_tunnel_types and add
> > > > > > l32 supported_flow_control.
> > > > > >
> > > > > supported_tunnel_types was let go because cvq is efficient.
> > > > > None of these fields are needed for init time configuration of
> > > > > the driver
> > > before DRIVER_OK.
> > > >
> > > > I really basically disagree. Whether control flow is supported can
> > > > easily influence how many VQs are needed.
> > > >
> > > >
> > > > >
> > > > > It is a very narrow view of 12 bits. It is going to grow and many.
> > > > > This is far more structured for each type done here.
> > > > >
> > > > > >
> > > > > > After you were trying to add kilobytes to megabytes of memory
> > > > > > - I see little reason to save 12 RO bits that can be shared by
> > > > > > any number of
> > > VFs.
> > > > > >
> > > > > Completely wrong reason and very late review and also does not
> > > > > align
> > > with every other command we did.
> > > >
> > > > which other command? hash and rss are like this: capability in
> > > > config space config through cvq. For the same reason.
> > > >
> > > > > > However, I do think we should create an option to access
> > > > > > config space over DMA (preferably admin commands). Let's add
> > > > > > this quickly and then we don't need to worry about legacy
> > > > > > guests accessing flow
> > > filter through MMIO.
> > > > >
> > > > > CVQ is already there as single interface forget and set for rss,
> > > > > rss context,
> > > vq moderation, statistics, flow filter caps and more.
> > > > > No reason to bifurcate.
> > > >
> > > > The reason is so that we have a single consistent view where e.g.
> > > > you want to provision a device with a specific capability you just
> > > > specify how its config space looks.
> > > >
> > > > If you start shuffling capabilities off to VQs that will be much
> > > > much harder.
> > > >
> > > > > I won't be able to absorb this comment of DMA interface.
> > > > > If I discuss further, I will repeat the whole document [1] and I
> > > > > will avoid
> > > that now.
> > > > >
> > > > > [1]
> > > > > https://docs.google.com/document/d/1Iyn-
> > > l3Nm0yls3pZaul4lZiVj8x1s73Ed
> > > > > 6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
> > > >
> > > >
> > > > I really worry about how provisioning will work. And I do not at
> > > > all cherish replicating all of these query capability commands for
> provisioning.
> > >
> > > +1
> > >
> >  - 1
> >
> > All points are discussed and not going to repeat 7th or 8th time.
> >
> > > There's nothing that prevents the config space from being
> > > implemented in a way other than registers.
> > >
> > It does.
> > It is discussed 7th or 8th time.
> >
> > All the points are discussed already.
> > [1]
> > https://docs.google.com/document/d/1Iyn-
> l3Nm0yls3pZaul4lZiVj8x1s73Ed6r
> > Osmn6LfXc/edit#heading=h.qexbtyc2jpwr
> >
> > many things are done over cvq and statistics capabilities over cvq is the latest
> approved.
> > I wont be able to repeat the content of document [1] anymore.
> 
> The famous Cons: None document.  Yea you pushed this through with
> statistics because hey it's just statistics.

Everyone agreed.

What more, see Jason's reply when explicitly asked in [2].

[2] https://lists.oasis-open.org/archives/virtio-comment/202308/msg00044.html
Snippet here:
"Adding cvq is much easier than inventing(duplicating) the work of a transport."

> Worst case devices can lie, it is not
> clear worrying about statistics when provisioning is important enough. 
Device statistics cannot be broken when migrating.

> And
> you are now under the impression everyone got convinced when in reality
> everyone just got tired of arguing. This is not a good pattern to try and repeat.
> Let me repeat the relevant part here for
> you:
> 
> 	So why shouldn’t we just add some transport VQ on the owner device
> to transport the SIOV device’s configuration?
> 
> 		Ans:
> 		Such addition means that hardware vendors need to build
> runtime configurations in 4 different ways.
> 		One way using CVQ for PCI PF and VFs
> 		2nd way as backward compatible SIOV using owner’s admin
> VQ
> 		3rd way using SIOV’s own CVQ channel for TDISP
> 		4th way using mix of backward compatible and secure and
> efficient way using #2 and #3.
> 
> 
> This is just a straw-man argument. No - we make a capability optional, 
Capabilities are not optional.
The design has undergone carefully review to ensure that driver honors the capability, and it is migratable both.

You dint explain why capability should be optional.
The first patch has defined how the capabilities are used by the driver.

> we
> strongly suggest that *drivers* support both old and new mechanism, and
> then *devices* will only implement what's required.
There are other examples in the same document that makes things worst with old and new.

Also there is literally no way to enforce that driver supports both and new. It is just sounds like an excuse to force infinite config space.

The method proposed here is elegant and clearly promote one way to do things for driver and device with predictability.

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Friday, November 24, 2023 11:03 AM
> 
> On Fri, Nov 24, 2023 at 12:02:23PM +0800, Jason Wang wrote:
> > > > I won't be able to absorb this comment of DMA interface.
> > > > If I discuss further, I will repeat the whole document [1] and I will avoid
> that now.
> > > >
> > > > [1]
> > > > https://docs.google.com/document/d/1Iyn-
> l3Nm0yls3pZaul4lZiVj8x1s73
> > > > Ed6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
> > >
> > >
> > > I really worry about how provisioning will work. And I do not at all
> > > cherish replicating all of these query capability commands for provisioning.
> >
> > +1
> >
> > There's nothing that prevents the config space from being implemented
> > in a way other than registers.
> 
> Care doing it finally? Let's see if what Parav is worrying about is then
> addressed.

The whole concept that everything must be in one giant config space is just simply bad.
It does not exist either in virtio spec today.

once can see that what is presented in the commands cannot be placed in config space at dynamic location.
Same was the case with statistics too.
Same was the case with VQ coaleasing knobs.
Same with hash knobs.
With flow filters,
With rss contexts
With rtc
With new queues creation apis.

The endless list continues...

And reserving bits for future (for other than pad bytes) for future addition in config space is equally not elegant design.
Bits will get spread out at random location making things even harder to maintain.

The device is no longer a simple mac_addr + N queues device with some static rss config anymore.

With all modern work, every capability query and run time configuration is done over cvq interface today.
Single get/set channel from driver to device all using existing resources.

The real hw device also does not need to refer to two places of config and cvq when serving cvq commands.
Oh, the list of advantages just continues with what 1.3 spec has done.


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Jason Wang]

On Fri, Nov 24, 2023 at 1:40 PM Parav Pandit <parav@nvidia.com> wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Friday, November 24, 2023 11:01 AM
> >
> > On Fri, Nov 24, 2023 at 04:13:08AM +0000, Parav Pandit wrote:
> > >
> > >
> > > > From: Jason Wang <jasowang@redhat.com>
> > > > Sent: Friday, November 24, 2023 9:32 AM
> > > >
> > > > On Wed, Nov 22, 2023 at 10:51 PM Michael S. Tsirkin <mst@redhat.com>
> > > > wrote:
> > > > >
> > > > > On Wed, Nov 22, 2023 at 02:10:29PM +0000, Parav Pandit wrote:
> > > > > >
> > > > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > > > Sent: Wednesday, November 22, 2023 7:32 PM
> > > > > > >
> > > > > > > On Fri, Nov 10, 2023 at 02:38:50PM +0200, Parav Pandit wrote:
> > > > > > > > The device responds flow filter capabilities using two commands.
> > > > > > > > One command indicates generic flow filter device limits such
> > > > > > > > as number of flow filters, number of flow filter groups,
> > > > > > > > support or multiple transports etc.
> > > > > > > >
> > > > > > > > The second command indicates supported match types, and
> > > > > > > > fields of the packet.
> > > > > > > >
> > > > > > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
> > > > > > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > > > > > > Signed-off-by: Parav Pandit <parav@nvidia.com>
> > > > > > > >
> > > > > > > > ---
> > > > > > > > changelog:
> > > > > > > > v2->v3:
> > > > > > > > - rebased on virtio-1.4 branch
> > > > > > > > - removed reference for flow filter virtqueue
> > > > > > > > v1->v2:
> > > > > > > > - addressed comments from Satananda
> > > > > > > > - added vlan type match field
> > > > > > > > - kept space for types between l2, l3, l4 header match types
> > > > > > > > - renamed mask to mask_supported with shorter width
> > > > > > > > - made more fields reserved for furture
> > > > > > > > - addressed comments from Heng
> > > > > > > > - grammar correction
> > > > > > > > - added field to indicate supported number of actions per flow
> > > > > > > >   filter match entry
> > > > > > > > - added missing documentation for
> > > > > > > > max_flow_priorities_per_group
> > > > > > > > v0->v1:
> > > > > > > > - added mask field in the type to indicate supported mask by device
> > > > > > > >   and also in later patch to use it to indicate mask on adding
> > > > > > > >   flow filter. As a result removed the mask_supported capability
> > > > > > > >   field
> > > > > > > > ---
> > > > > > > >  device-types/net/description.tex | 208
> > > > > > > > ++++++++++++++++++++++++++++++-
> > > > > > > >  1 file changed, 206 insertions(+), 2 deletions(-)
> > > > > > > >
> > > > > > > > diff --git a/device-types/net/description.tex
> > > > > > > > b/device-types/net/description.tex
> > > > > > > > index 30220b5..eccd8d6 100644
> > > > > > > > --- a/device-types/net/description.tex
> > > > > > > > +++ b/device-types/net/description.tex
> > > > > > > > @@ -1173,7 +1173,11 @@ \subsubsection{Flow
> > > > > > > > Filter}\label{sec:Device Types / Network Device / Device Ope
> > > > > > > >
> > > > > > > >  The device indicates the flow filter capabilities to the driver.
> > > > > > > > These  capabilities include various maximum device limits
> > > > > > > > and -supported packet match fields.
> > > > > > > > +supported packet match fields. These control virtqueue
> > > > > > > > +commands
> > > > are:
> > > > > > > > +\ref{sec:Device Types / Network Device / Device Operation /
> > > > > > > > +Control Virtqueue / Flow Filter / Flow Filter Capabilities
> > > > > > > > +Get} and \ref{sec:Device Types / Network Device / Device
> > > > > > > > +Operation / Control
> > > > > > > Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}.
> > > > > > > >
> > > > > > > >  The flow filters are grouped using a flow filter group.
> > > > > > > > Each flow filter  group has a priority. The device first
> > > > > > > > applies the flow filters of the highest @@ -1224,7 +1228,136
> > > > > > > > @@ \subsubsection{Flow
> > > > > > > Filter}\label{sec:Device Types / Network Device / Device Ope
> > > > > > > >        the flow filters in group_C, the flow filters of next
> > > > > > > > level group_B are
> > > > > > > applied.
> > > > > > > >  \end{itemize}
> > > > > > > >
> > > > > > > > -\label{sec:Device Types / Network Device / Device Operation
> > > > > > > > / Control Virtqueue / Setting Promiscuous Mode}%old label
> > > > > > > > for latexdiff
> > > > > > > > +\paragraph{Match Types and Fields}\label{sec:Device Types /
> > > > > > > > +Network Device / Device Operation / Flow Filter / Match
> > > > > > > > +Types and Fields}
> > > > > > > > +
> > > > > > > > +\begin{lstlisting}
> > > > > > > > +struct virtio_net_ff_match_type_cap {
> > > > > > > > +        le16 type;
> > > > > > > > +        u8 mask_supported;
> > > > > > > > +        u8 reserved[5];
> > > > > > > > +        le64 fields_bmap;
> > > > > > > > +};
> > > > > > > > +\end{lstlisting}
> > > > > > > > +
> > > > > > > > +The \field{type} corresponds to following table:
> > > > > > > > +
> > > > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > > > +\hline
> > > > > > > > +Type & Name & Description \\ \hline \hline
> > > > > > > > +0   & VIRTIO_NET_FF_ETH_HDR & Ethernet header of the packet \\
> > > > > > > > +\hline
> > > > > > > > +0x1   & VIRTIO_NET_FF_VLAN_TAG_HDR & VLAN tag of the packet
> > \\
> > > > > > > > +\hline
> > > > > > > > +0x200   & VIRTIO_NET_FF_IPV4_HDR & IPv4 header of the packet
> > \\
> > > > > > > > +\hline
> > > > > > > > +0x300   & VIRTIO_NET_FF_IPV6_HDR & IPv6 header of the packet
> > \\
> > > > > > > > +\hline
> > > > > > > > +0x400   & VIRTIO_NET_FF_TCP_HDR & TCP header of the packet \\
> > > > > > > > +\hline
> > > > > > > > +0x500   & VIRTIO_NET_FF_UDP_HDR & UDP header of the packet
> > \\
> > > > > > > > +\hline
> > > > > > > > +other   & -    & reserved \\
> > > > > > > > +\hline
> > > > > > > > +\end{tabular}
> > > > > > > > +
> > > > > > > > +When the \field{mask_supported} is set, for the specific
> > > > > > > > +\field{type}, the device can perform masking packet fields
> > > > > > > > +with the mask supplied in the flow filter match entry.
> > > > > > > > +
> > > > > > > > +For each \field{type} the \field{fields_bmap} indicates
> > > > > > > > +supported fields of the packet header which can be matched.
> > > > > > > > +
> > > > > > > > +For the \field{type} of VIRTIO_NET_FF_ETH_HDR, header
> > > > > > > > +fields are represented by a bitmap in \field{fields_bmap} are
> > following:
> > > > > > > > +
> > > > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > > > +\hline
> > > > > > > > +Bit & Name & Description \\ \hline \hline
> > > > > > > > +0   & VIRTIO_NET_FF_DST_MAC & Destination MAC address in the
> > > > packet \\
> > > > > > > > +\hline
> > > > > > > > +1   & VIRTIO_NET_FF_SRC_MAC & Source MAC address in the
> > packet
> > > > \\
> > > > > > > > +\hline
> > > > > > > > +2   & VIRTIO_NET_FF_ETHER_TYPE & Ether type in the packet \\
> > > > > > > > +\hline
> > > > > > > > +other   & -    & reserved \\
> > > > > > > > +\hline
> > > > > > > > +\end{tabular}
> > > > > > > > +
> > > > > > > > +For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN
> > > > > > > > +tag fields are represented by a bitmap in
> > > > > > > > +\field{fields_bmap} are
> > > > following:
> > > > > > > > +
> > > > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > > > +\hline
> > > > > > > > +Bit & Name & Description \\ \hline \hline
> > > > > > > > +0   & VIRTIO_NET_FF_VLAN_TAG_TCI & Vlan tag TCI 16-bit field \\
> > > > > > > > +\hline
> > > > > > > > +other   & -    & reserved \\
> > > > > > > > +\hline
> > > > > > > > +\end{tabular}
> > > > > > > > +
> > > > > > > > +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, header
> > > > > > > > +fields are represented by a bitmap in \field{fields_bmap} are
> > following:
> > > > > > > > +
> > > > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > > > +\hline
> > > > > > > > +Bit & Name & Description \\ \hline \hline
> > > > > > > > +0   & VIRTIO_NET_FF_SRC_IPV4 & Source IPV4 address in the
> > packet
> > > > \\
> > > > > > > > +\hline
> > > > > > > > +1   & VIRTIO_NET_FF_DST_IPV4 & Destination IPV4 address in the
> > > > packet \\
> > > > > > > > +\hline
> > > > > > > > +other   & -    & reserved \\
> > > > > > > > +\hline
> > > > > > > > +\end{tabular}
> > > > > > > > +
> > > > > > > > +For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header
> > > > > > > > +fields are represented by a bitmap in \field{fields_bmap} are
> > following:
> > > > > > > > +
> > > > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > > > +\hline
> > > > > > > > +Bit & Name & Description \\ \hline \hline
> > > > > > > > +0   & VIRTIO_NET_FF_SRC_IPV6 & Source IPV6 address in the
> > packet
> > > > \\
> > > > > > > > +\hline
> > > > > > > > +1   & VIRTIO_NET_FF_DST_IPV6 & Destination IPV6 address in the
> > > > packet \\
> > > > > > > > +\hline
> > > > > > > > +other   & -    & reserved \\
> > > > > > > > +\hline
> > > > > > > > +\end{tabular}
> > > > > > > > +
> > > > > > > > +For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header
> > > > > > > > +fields are represented by a bitmap in \field{fields_bmap} are
> > following:
> > > > > > > > +
> > > > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > > > +\hline
> > > > > > > > +Bit & Name & Description \\ \hline \hline
> > > > > > > > +0   & VIRTIO_NET_FF_SRC_TCP_PORT & Source TCP port in the
> > > > packet \\
> > > > > > > > +\hline
> > > > > > > > +1   & VIRTIO_NET_FF_DST_TCP_PORT & Destination TCP port in the
> > > > packet
> > > > > > > \\
> > > > > > > > +\hline
> > > > > > > > +other   & -    & reserved \\
> > > > > > > > +\hline
> > > > > > > > +\end{tabular}
> > > > > > > > +
> > > > > > > > +For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header
> > > > > > > > +fields are represented by a bitmap in \field{fields_bmap} are
> > following:
> > > > > > > > +
> > > > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > > > +\hline
> > > > > > > > +Bit & Name & Description \\ \hline \hline
> > > > > > > > +0   & VIRTIO_NET_FF_SRC_UDP_PORT & Source UDP port in the
> > > > packet \\
> > > > > > > > +\hline
> > > > > > > > +1   & VIRTIO_NET_FF_DST_UDP_PORT & Destination UDP port in
> > the
> > > > packet
> > > > > > > \\
> > > > > > > > +\hline
> > > > > > > > +other   & -    & reserved  \\
> > > > > > > > +\hline
> > > > > > > > +\end{tabular}
> > > > > > > > +
> > > > > > >
> > > > > > > This is such an elaborate structure to report just 12 read only bits.
> > > > > > > Please let's just follow the example of  le32
> > > > > > > supported_tunnel_types and add
> > > > > > > l32 supported_flow_control.
> > > > > > >
> > > > > > supported_tunnel_types was let go because cvq is efficient.
> > > > > > None of these fields are needed for init time configuration of
> > > > > > the driver
> > > > before DRIVER_OK.
> > > > >
> > > > > I really basically disagree. Whether control flow is supported can
> > > > > easily influence how many VQs are needed.
> > > > >
> > > > >
> > > > > >
> > > > > > It is a very narrow view of 12 bits. It is going to grow and many.
> > > > > > This is far more structured for each type done here.
> > > > > >
> > > > > > >
> > > > > > > After you were trying to add kilobytes to megabytes of memory
> > > > > > > - I see little reason to save 12 RO bits that can be shared by
> > > > > > > any number of
> > > > VFs.
> > > > > > >
> > > > > > Completely wrong reason and very late review and also does not
> > > > > > align
> > > > with every other command we did.
> > > > >
> > > > > which other command? hash and rss are like this: capability in
> > > > > config space config through cvq. For the same reason.
> > > > >
> > > > > > > However, I do think we should create an option to access
> > > > > > > config space over DMA (preferably admin commands). Let's add
> > > > > > > this quickly and then we don't need to worry about legacy
> > > > > > > guests accessing flow
> > > > filter through MMIO.
> > > > > >
> > > > > > CVQ is already there as single interface forget and set for rss,
> > > > > > rss context,
> > > > vq moderation, statistics, flow filter caps and more.
> > > > > > No reason to bifurcate.
> > > > >
> > > > > The reason is so that we have a single consistent view where e.g.
> > > > > you want to provision a device with a specific capability you just
> > > > > specify how its config space looks.
> > > > >
> > > > > If you start shuffling capabilities off to VQs that will be much
> > > > > much harder.
> > > > >
> > > > > > I won't be able to absorb this comment of DMA interface.
> > > > > > If I discuss further, I will repeat the whole document [1] and I
> > > > > > will avoid
> > > > that now.
> > > > > >
> > > > > > [1]
> > > > > > https://docs.google.com/document/d/1Iyn-
> > > > l3Nm0yls3pZaul4lZiVj8x1s73Ed
> > > > > > 6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
> > > > >
> > > > >
> > > > > I really worry about how provisioning will work. And I do not at
> > > > > all cherish replicating all of these query capability commands for
> > provisioning.
> > > >
> > > > +1
> > > >
> > >  - 1
> > >
> > > All points are discussed and not going to repeat 7th or 8th time.
> > >
> > > > There's nothing that prevents the config space from being
> > > > implemented in a way other than registers.
> > > >
> > > It does.
> > > It is discussed 7th or 8th time.
> > >
> > > All the points are discussed already.
> > > [1]
> > > https://docs.google.com/document/d/1Iyn-
> > l3Nm0yls3pZaul4lZiVj8x1s73Ed6r
> > > Osmn6LfXc/edit#heading=h.qexbtyc2jpwr
> > >
> > > many things are done over cvq and statistics capabilities over cvq is the latest
> > approved.
> > > I wont be able to repeat the content of document [1] anymore.
> >
> > The famous Cons: None document.  Yea you pushed this through with
> > statistics because hey it's just statistics.
>
> Everyone agreed.
>
> What more, see Jason's reply when explicitly asked in [2].
>
> [2] https://lists.oasis-open.org/archives/virtio-comment/202308/msg00044.html
> Snippet here:
> "Adding cvq is much easier than inventing(duplicating) the work of a transport."

I've also said

"
As mentioned in the past, when developing spec, we should look for
what it can be.

My point is to stick to the device configuration space but invent DMA
interface to access them, then we are all fine.
"

Just to clarify my point, what's I'm saying is

1) For CVQ, it is used to send the commands to the device. It's not
good to use transport to send those commands
2) The configuration space is the place to advertise the capabilities,
e.g what kind of commands could the device accept.

2) doesn't conflict with 1)

And we are discussing the provisioning which is more about 2) but not 1) here.

That's it.

Thanks


>
> > Worst case devices can lie, it is not
> > clear worrying about statistics when provisioning is important enough.
> Device statistics cannot be broken when migrating.
>
> > And
> > you are now under the impression everyone got convinced when in reality
> > everyone just got tired of arguing. This is not a good pattern to try and repeat.
> > Let me repeat the relevant part here for
> > you:
> >
> >       So why shouldn’t we just add some transport VQ on the owner device
> > to transport the SIOV device’s configuration?
> >
> >               Ans:
> >               Such addition means that hardware vendors need to build
> > runtime configurations in 4 different ways.
> >               One way using CVQ for PCI PF and VFs
> >               2nd way as backward compatible SIOV using owner’s admin
> > VQ
> >               3rd way using SIOV’s own CVQ channel for TDISP
> >               4th way using mix of backward compatible and secure and
> > efficient way using #2 and #3.
> >
> >
> > This is just a straw-man argument. No - we make a capability optional,
> Capabilities are not optional.
> The design has undergone carefully review to ensure that driver honors the capability, and it is migratable both.
>
> You dint explain why capability should be optional.
> The first patch has defined how the capabilities are used by the driver.
>
> > we
> > strongly suggest that *drivers* support both old and new mechanism, and
> > then *devices* will only implement what's required.
> There are other examples in the same document that makes things worst with old and new.
>
> Also there is literally no way to enforce that driver supports both and new. It is just sounds like an excuse to force infinite config space.
>
> The method proposed here is elegant and clearly promote one way to do things for driver and device with predictability.
>
>


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Jason Wang]

On Fri, Nov 24, 2023 at 12:13 PM Parav Pandit <parav@nvidia.com> wrote:
>
>
>
> > From: Jason Wang <jasowang@redhat.com>
> > Sent: Friday, November 24, 2023 9:32 AM
> >
> > On Wed, Nov 22, 2023 at 10:51 PM Michael S. Tsirkin <mst@redhat.com>
> > wrote:
> > >
> > > On Wed, Nov 22, 2023 at 02:10:29PM +0000, Parav Pandit wrote:
> > > >
> > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > Sent: Wednesday, November 22, 2023 7:32 PM
> > > > >
> > > > > On Fri, Nov 10, 2023 at 02:38:50PM +0200, Parav Pandit wrote:
> > > > > > The device responds flow filter capabilities using two commands.
> > > > > > One command indicates generic flow filter device limits such as
> > > > > > number of flow filters, number of flow filter groups, support or
> > > > > > multiple transports etc.
> > > > > >
> > > > > > The second command indicates supported match types, and fields
> > > > > > of the packet.
> > > > > >
> > > > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
> > > > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > > > > Signed-off-by: Parav Pandit <parav@nvidia.com>
> > > > > >
> > > > > > ---
> > > > > > changelog:
> > > > > > v2->v3:
> > > > > > - rebased on virtio-1.4 branch
> > > > > > - removed reference for flow filter virtqueue
> > > > > > v1->v2:
> > > > > > - addressed comments from Satananda
> > > > > > - added vlan type match field
> > > > > > - kept space for types between l2, l3, l4 header match types
> > > > > > - renamed mask to mask_supported with shorter width
> > > > > > - made more fields reserved for furture
> > > > > > - addressed comments from Heng
> > > > > > - grammar correction
> > > > > > - added field to indicate supported number of actions per flow
> > > > > >   filter match entry
> > > > > > - added missing documentation for max_flow_priorities_per_group
> > > > > > v0->v1:
> > > > > > - added mask field in the type to indicate supported mask by device
> > > > > >   and also in later patch to use it to indicate mask on adding
> > > > > >   flow filter. As a result removed the mask_supported capability
> > > > > >   field
> > > > > > ---
> > > > > >  device-types/net/description.tex | 208
> > > > > > ++++++++++++++++++++++++++++++-
> > > > > >  1 file changed, 206 insertions(+), 2 deletions(-)
> > > > > >
> > > > > > diff --git a/device-types/net/description.tex
> > > > > > b/device-types/net/description.tex
> > > > > > index 30220b5..eccd8d6 100644
> > > > > > --- a/device-types/net/description.tex
> > > > > > +++ b/device-types/net/description.tex
> > > > > > @@ -1173,7 +1173,11 @@ \subsubsection{Flow
> > > > > > Filter}\label{sec:Device Types / Network Device / Device Ope
> > > > > >
> > > > > >  The device indicates the flow filter capabilities to the driver.
> > > > > > These  capabilities include various maximum device limits and
> > > > > > -supported packet match fields.
> > > > > > +supported packet match fields. These control virtqueue commands
> > are:
> > > > > > +\ref{sec:Device Types / Network Device / Device Operation /
> > > > > > +Control Virtqueue / Flow Filter / Flow Filter Capabilities Get}
> > > > > > +and \ref{sec:Device Types / Network Device / Device Operation /
> > > > > > +Control
> > > > > Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}.
> > > > > >
> > > > > >  The flow filters are grouped using a flow filter group. Each
> > > > > > flow filter  group has a priority. The device first applies the
> > > > > > flow filters of the highest @@ -1224,7 +1228,136 @@
> > > > > > \subsubsection{Flow
> > > > > Filter}\label{sec:Device Types / Network Device / Device Ope
> > > > > >        the flow filters in group_C, the flow filters of next
> > > > > > level group_B are
> > > > > applied.
> > > > > >  \end{itemize}
> > > > > >
> > > > > > -\label{sec:Device Types / Network Device / Device Operation /
> > > > > > Control Virtqueue / Setting Promiscuous Mode}%old label for
> > > > > > latexdiff
> > > > > > +\paragraph{Match Types and Fields}\label{sec:Device Types /
> > > > > > +Network Device / Device Operation / Flow Filter / Match Types
> > > > > > +and Fields}
> > > > > > +
> > > > > > +\begin{lstlisting}
> > > > > > +struct virtio_net_ff_match_type_cap {
> > > > > > +        le16 type;
> > > > > > +        u8 mask_supported;
> > > > > > +        u8 reserved[5];
> > > > > > +        le64 fields_bmap;
> > > > > > +};
> > > > > > +\end{lstlisting}
> > > > > > +
> > > > > > +The \field{type} corresponds to following table:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Type & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_ETH_HDR & Ethernet header of the packet \\
> > > > > > +\hline
> > > > > > +0x1   & VIRTIO_NET_FF_VLAN_TAG_HDR & VLAN tag of the packet \\
> > > > > > +\hline
> > > > > > +0x200   & VIRTIO_NET_FF_IPV4_HDR & IPv4 header of the packet \\
> > > > > > +\hline
> > > > > > +0x300   & VIRTIO_NET_FF_IPV6_HDR & IPv6 header of the packet \\
> > > > > > +\hline
> > > > > > +0x400   & VIRTIO_NET_FF_TCP_HDR & TCP header of the packet \\
> > > > > > +\hline
> > > > > > +0x500   & VIRTIO_NET_FF_UDP_HDR & UDP header of the packet \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +When the \field{mask_supported} is set, for the specific
> > > > > > +\field{type}, the device can perform masking packet fields with
> > > > > > +the mask supplied in the flow filter match entry.
> > > > > > +
> > > > > > +For each \field{type} the \field{fields_bmap} indicates
> > > > > > +supported fields of the packet header which can be matched.
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_ETH_HDR, header fields
> > > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_DST_MAC & Destination MAC address in the
> > packet \\
> > > > > > +\hline
> > > > > > +1   & VIRTIO_NET_FF_SRC_MAC & Source MAC address in the packet
> > \\
> > > > > > +\hline
> > > > > > +2   & VIRTIO_NET_FF_ETHER_TYPE & Ether type in the packet \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN tag
> > > > > > +fields are represented by a bitmap in \field{fields_bmap} are
> > following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_VLAN_TAG_TCI & Vlan tag TCI 16-bit field \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, header fields
> > > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_SRC_IPV4 & Source IPV4 address in the packet
> > \\
> > > > > > +\hline
> > > > > > +1   & VIRTIO_NET_FF_DST_IPV4 & Destination IPV4 address in the
> > packet \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header fields
> > > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_SRC_IPV6 & Source IPV6 address in the packet
> > \\
> > > > > > +\hline
> > > > > > +1   & VIRTIO_NET_FF_DST_IPV6 & Destination IPV6 address in the
> > packet \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header fields
> > > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_SRC_TCP_PORT & Source TCP port in the
> > packet \\
> > > > > > +\hline
> > > > > > +1   & VIRTIO_NET_FF_DST_TCP_PORT & Destination TCP port in the
> > packet
> > > > > \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header fields
> > > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_SRC_UDP_PORT & Source UDP port in the
> > packet \\
> > > > > > +\hline
> > > > > > +1   & VIRTIO_NET_FF_DST_UDP_PORT & Destination UDP port in the
> > packet
> > > > > \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved  \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > >
> > > > > This is such an elaborate structure to report just 12 read only bits.
> > > > > Please let's just follow the example of  le32
> > > > > supported_tunnel_types and add
> > > > > l32 supported_flow_control.
> > > > >
> > > > supported_tunnel_types was let go because cvq is efficient.
> > > > None of these fields are needed for init time configuration of the driver
> > before DRIVER_OK.
> > >
> > > I really basically disagree. Whether control flow is supported can
> > > easily influence how many VQs are needed.
> > >
> > >
> > > >
> > > > It is a very narrow view of 12 bits. It is going to grow and many.
> > > > This is far more structured for each type done here.
> > > >
> > > > >
> > > > > After you were trying to add kilobytes to megabytes of memory - I
> > > > > see little reason to save 12 RO bits that can be shared by any number of
> > VFs.
> > > > >
> > > > Completely wrong reason and very late review and also does not align
> > with every other command we did.
> > >
> > > which other command? hash and rss are like this: capability in config
> > > space config through cvq. For the same reason.
> > >
> > > > > However, I do think we should create an option to access config
> > > > > space over DMA (preferably admin commands). Let's add this quickly
> > > > > and then we don't need to worry about legacy guests accessing flow
> > filter through MMIO.
> > > >
> > > > CVQ is already there as single interface forget and set for rss, rss context,
> > vq moderation, statistics, flow filter caps and more.
> > > > No reason to bifurcate.
> > >
> > > The reason is so that we have a single consistent view where e.g. you
> > > want to provision a device with a specific capability you just specify
> > > how its config space looks.
> > >
> > > If you start shuffling capabilities off to VQs that will be much much
> > > harder.
> > >
> > > > I won't be able to absorb this comment of DMA interface.
> > > > If I discuss further, I will repeat the whole document [1] and I will avoid
> > that now.
> > > >
> > > > [1]
> > > > https://docs.google.com/document/d/1Iyn-
> > l3Nm0yls3pZaul4lZiVj8x1s73Ed
> > > > 6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
> > >
> > >
> > > I really worry about how provisioning will work. And I do not at all
> > > cherish replicating all of these query capability commands for provisioning.
> >
> > +1
> >
>  - 1
>
> All points are discussed and not going to repeat 7th or 8th time.

Discussed by not converging.


>
> > There's nothing that prevents the config space from being implemented in a
> > way other than registers.
> >
> It does.
> It is discussed 7th or 8th time.
>
> All the points are discussed already.
> [1] https://docs.google.com/document/d/1Iyn-l3Nm0yls3pZaul4lZiVj8x1s73Ed6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
>
> many things are done over cvq and statistics capabilities over cvq is the latest approved.
> I wont be able to repeat the content of document [1] anymore.

We are discussing the provisioning. Hypervisor needs to know the
capabilities of a device in order to migrate it.

Exposing device specific capabilities in device configuration space is
the way we're using now. You don't need a mini-driver to get the
capability of a device.

It doesn't conflict with CVQ.

>
> > > Instead, I propose commands for config space access to solve
> > > everything in one go, for all device types.
> > >
> >
> > Something similar like:
> >
> > +\begin{lstlisting}
> > +#define VIRTIO_TRANSPTQ_CTRL_CONFIG    6
> > +  #define VIRTIO_TRANSPTQ_CTRL_CONFIG_GET    0
> > +  #define VIRTIO_TRANSPTQ_CTRL_CONFIG_SET    1
> > +
> > +struct transportq_ctrl_mdev_config_get {
> > +       u32 offset;
> > +       u32 size;
> > +};
> > +
> > +struct transportq_ctrl_mdev_config_set {
> > +       u32 offset;
> > +       u32 size;
> > +       u8  data[];
> > +};
> > +\end{lstlisting}
> >
> > in the proposal of transport virtqueue.
> >
> > https://lists.oasis-open.org/archives/virtio-comment/202208/msg00005.html
> >
> -1 as cvq already exists those services the role for caps and control far beyond tiny register-based interface.

Again, device configuration space is not necessarily implemented in
registers. This seems to be a wrong assumption in your doc.

>
> As more features grow, tunneling commands through a I2C or SPI or GPIO kind of register interface is turning the clock back.
> A command interface is way more extendible than register interface.

I don't get here, device configuration space itself can be extended as
described in the spec:

"
Where configuration fields are optional, their existence is indicated
by feature bits: Future versions of this specification will likely
extend the device configuration space by adding extra fields at the
tail.
"

Thanks


>
> We wont be able to support this change that does not align with all the improvements done in 1.3 for notification coalescing, hash, stats and more.


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Fri, Nov 24, 2023 at 05:53:02AM +0000, Parav Pandit wrote:
> 
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Friday, November 24, 2023 11:03 AM
> > 
> > On Fri, Nov 24, 2023 at 12:02:23PM +0800, Jason Wang wrote:
> > > > > I won't be able to absorb this comment of DMA interface.
> > > > > If I discuss further, I will repeat the whole document [1] and I will avoid
> > that now.
> > > > >
> > > > > [1]
> > > > > https://docs.google.com/document/d/1Iyn-
> > l3Nm0yls3pZaul4lZiVj8x1s73
> > > > > Ed6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
> > > >
> > > >
> > > > I really worry about how provisioning will work. And I do not at all
> > > > cherish replicating all of these query capability commands for provisioning.
> > >
> > > +1
> > >
> > > There's nothing that prevents the config space from being implemented
> > > in a way other than registers.
> > 
> > Care doing it finally? Let's see if what Parav is worrying about is then
> > addressed.
> 
> The whole concept that everything must be in one giant config space is just simply bad.
> It does not exist either in virtio spec today.

But it does, this is what transports are doing.

> once can see that what is presented in the commands cannot be placed in config space at dynamic location.
> Same was the case with statistics too.
> Same was the case with VQ coaleasing knobs.
> Same with hash knobs.
> With flow filters,
> With rss contexts
> With rtc
> With new queues creation apis.
> 
> The endless list continues...
> 
> And reserving bits for future (for other than pad bytes) for future addition in config space is equally not elegant design.
> Bits will get spread out at random location making things even harder to maintain.
> 
> The device is no longer a simple mac_addr + N queues device with some static rss config anymore.
> 
> With all modern work, every capability query and run time configuration is done over cvq interface today.
> Single get/set channel from driver to device all using existing resources.
> 
> The real hw device also does not need to refer to two places of config and cvq when serving cvq commands.
> Oh, the list of advantages just continues with what 1.3 spec has done.

I don't see the problem sorry. We've been doing this for many years with
many ways to access config space.  It scaled well.


Then hardware offload guys come and say that in PCI spec current
transport is forcing use of on-device memory, and they want to build
cheap offload PCI based devices. Fine, let's build a transport variant
that does not force this. And we want optional compatibility, so let's
also find a way to do that.  This makes much more sense than forcing
transport specific issues on everyone.

To add to that, what did not historicall scale well is
transport-specific registers.  That was a bad design, with all
transports doing exactly the same thing is slightly different ways.  And
what you are advocating for with CVQ is exactly replicating the bad
design not the good one.

-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Fri, Nov 24, 2023 at 05:40:26AM +0000, Parav Pandit wrote:
> > we
> > strongly suggest that *drivers* support both old and new mechanism, and
> > then *devices* will only implement what's required.
> There are other examples in the same document that makes things worst with old and new.
> 
> Also there is literally no way to enforce that driver supports both
> and new. It is just sounds like an excuse to force infinite config
> space.

There is a very simple method though.  We allow devices to expose a
subset of features when DMA is not used. So drivers that want maximum
features will always opt for DMA. We can also strongly
recommend that all drivers support DMA if available.



> The method proposed here is elegant and clearly promote one way to do things for driver and device with predictability.
> 

I don't see it as elegant at all. What is elegant is *a single tag* that
describes each property of the device. And this single tag should be
good for everything: driver, provisioning, migration. And config space
offset serves as such.


-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Friday, November 24, 2023 11:37 AM
> 
> On Fri, Nov 24, 2023 at 05:53:02AM +0000, Parav Pandit wrote:
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Friday, November 24, 2023 11:03 AM
> > >
> > > On Fri, Nov 24, 2023 at 12:02:23PM +0800, Jason Wang wrote:
> > > > > > I won't be able to absorb this comment of DMA interface.
> > > > > > If I discuss further, I will repeat the whole document [1] and
> > > > > > I will avoid
> > > that now.
> > > > > >
> > > > > > [1]
> > > > > > https://docs.google.com/document/d/1Iyn-
> > > l3Nm0yls3pZaul4lZiVj8x1s73
> > > > > > Ed6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
> > > > >
> > > > >
> > > > > I really worry about how provisioning will work. And I do not at
> > > > > all cherish replicating all of these query capability commands for
> provisioning.
> > > >
> > > > +1
> > > >
> > > > There's nothing that prevents the config space from being
> > > > implemented in a way other than registers.
> > >
> > > Care doing it finally? Let's see if what Parav is worrying about is
> > > then addressed.
> >
> > The whole concept that everything must be in one giant config space is just
> simply bad.
> > It does not exist either in virtio spec today.
> 
> But it does, this is what transports are doing.
I don't understand what is "it".
Transport like pci transport bits.

CVQ is functional object that helps to arrange the bits in logical, functional manner instead of trying to place them in bit array.

> 
> > once can see that what is presented in the commands cannot be placed in
> config space at dynamic location.
> > Same was the case with statistics too.
> > Same was the case with VQ coaleasing knobs.
> > Same with hash knobs.
> > With flow filters,
> > With rss contexts
> > With rtc
> > With new queues creation apis.
> >
> > The endless list continues...
> >
> > And reserving bits for future (for other than pad bytes) for future addition in
> config space is equally not elegant design.
> > Bits will get spread out at random location making things even harder to
> maintain.
> >
> > The device is no longer a simple mac_addr + N queues device with some
> static rss config anymore.
> >
> > With all modern work, every capability query and run time configuration is
> done over cvq interface today.
> > Single get/set channel from driver to device all using existing resources.
> >
> > The real hw device also does not need to refer to two places of config and
> cvq when serving cvq commands.
> > Oh, the list of advantages just continues with what 1.3 spec has done.
> 
> I don't see the problem sorry. We've been doing this for many years with many
> ways to access config space.  It scaled well.
> 
I don't see it. sorry.
The configuration of the device is done using cvq.
> 
> Then hardware offload guys come and say that in PCI spec current transport is
> forcing use of on-device memory, and they want to build cheap offload PCI
> based devices. Fine, let's build a transport variant that does not force this.

All new capabilities and control is over the cvq. What is baked until 1.2 is sort of legacy.

>  And
> we want optional compatibility, so let's also find a way to do that.  This makes
> much more sense than forcing transport specific issues on everyone.
> 
Trying to attribute as some transport specific issue is just not aligned to the spec written today.

> To add to that, what did not historicall scale well is transport-specific registers.
Then you should have put the VQ notification coalescing functionality in a horrible virtio_net_config register like how a queue reset in the common config space.
Thank God that mistake was not done and many similar mistakes were avoided.

> That was a bad design, with all transports doing exactly the same thing is
> slightly different ways.  And what you are advocating for with CVQ is exactly
> replicating the bad design not the good one.

CVQ design is what extends the current spec in good way. Followed by many other non nvidia nics listed in the doc for reference.

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Friday, November 24, 2023 11:48 AM
> 
> On Fri, Nov 24, 2023 at 05:40:26AM +0000, Parav Pandit wrote:
> > > we
> > > strongly suggest that *drivers* support both old and new mechanism,
> > > and then *devices* will only implement what's required.
> > There are other examples in the same document that makes things worst
> with old and new.
> >
> > Also there is literally no way to enforce that driver supports both
> > and new. It is just sounds like an excuse to force infinite config
> > space.
> 
> There is a very simple method though.  We allow devices to expose a subset of
> features when DMA is not used. So drivers that want maximum features will
> always opt for DMA. We can also strongly recommend that all drivers support
> DMA if available.
Yeah, don't see how this is elegant at all with all mixed bits.
Nor do I see any enforcement, single method via cvq still holds strong.

> 
> 
> 
> > The method proposed here is elegant and clearly promote one way to do
> things for driver and device with predictability.
> >
> 
> I don't see it as elegant at all. What is elegant is *a single tag* that describes
> each property of the device. And this single tag should be good for everything:
> driver, provisioning, migration. And config space offset serves as such.
The single tag is the set of structures.
Provisioning access them via owner device.
Member driver access them via CVQ or 1.2 legacy config space.


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Jason Wang <jasowang@redhat.com>
> Sent: Friday, November 24, 2023 11:32 AM


> Just to clarify my point, what's I'm saying is
> 
> 1) For CVQ, it is used to send the commands to the device. It's not good to use
> transport to send those commands
This is not what is proposed.

CVQ is proposed to exchange capabilities and configuration between driver and device.
(not to transport to send commands)

> 2) The configuration space is the place to advertise the capabilities, e.g what
> kind of commands could the device accept.
> 
and it is not efficient and not used anymore with 1.3 and higher.
They are exchanged using single get/set interface where device does not need to do special cross synchronization between what is exposed via interface_1 (config space) and control via interface_2 (cvq).

> 2) doesn't conflict with 1)
> 
CVQ does not conflict with anything as it stands today in 1.3.

> And we are discussing the provisioning which is more about 2) but not 1)
> here.
Provisioning will set the device and device will run based on what is being set.
If MSIX is provisioned, it will show up in MSIX attribute.
If the BAR region size is provisioned it will show in PCI MMIO size.
If 1.3 legacy mac address configured, shows up in virtio_net_config.
if 1.3 rss configuration, shows up in virtio net config

if 1.3 stats configuration, shows up in stats.
If 1.4 flow filters provisioned shows in flow filter caps.

The primary tenet is: config space contains only the necessary driver bootstap fields.
The bright line in based on this usage: bootstap or not.
If its runtime, sure have it over CVQ.
Following the nice tenet of B.2 of the spec snippet: "Device configuration space should only be used for initialization-time parameters"
That’s it.
And everyone is already aligned to it.

> 
> That's it.
> 
> Thanks
> 
> 
> >
> > > Worst case devices can lie, it is not clear worrying about
> > > statistics when provisioning is important enough.
> > Device statistics cannot be broken when migrating.
> >
> > > And
> > > you are now under the impression everyone got convinced when in
> > > reality everyone just got tired of arguing. This is not a good pattern to try
> and repeat.
> > > Let me repeat the relevant part here for
> > > you:
> > >
> > >       So why shouldn’t we just add some transport VQ on the owner
> > > device to transport the SIOV device’s configuration?
> > >
> > >               Ans:
> > >               Such addition means that hardware vendors need to
> > > build runtime configurations in 4 different ways.
> > >               One way using CVQ for PCI PF and VFs
> > >               2nd way as backward compatible SIOV using owner’s
> > > admin VQ
> > >               3rd way using SIOV’s own CVQ channel for TDISP
> > >               4th way using mix of backward compatible and secure
> > > and efficient way using #2 and #3.
> > >
> > >
> > > This is just a straw-man argument. No - we make a capability
> > > optional,
> > Capabilities are not optional.
> > The design has undergone carefully review to ensure that driver honors the
> capability, and it is migratable both.
> >
> > You dint explain why capability should be optional.
> > The first patch has defined how the capabilities are used by the driver.
> >
> > > we
> > > strongly suggest that *drivers* support both old and new mechanism,
> > > and then *devices* will only implement what's required.
> > There are other examples in the same document that makes things worst
> with old and new.
> >
> > Also there is literally no way to enforce that driver supports both and new. It
> is just sounds like an excuse to force infinite config space.
> >
> > The method proposed here is elegant and clearly promote one way to do
> things for driver and device with predictability.
> >
> >

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Fri, Nov 24, 2023 at 06:41:11AM +0000, Parav Pandit wrote:
> 
> > From: Jason Wang <jasowang@redhat.com>
> > Sent: Friday, November 24, 2023 11:32 AM
> 
> 
> > Just to clarify my point, what's I'm saying is
> > 
> > 1) For CVQ, it is used to send the commands to the device. It's not good to use
> > transport to send those commands
> This is not what is proposed.
> 
> CVQ is proposed to exchange capabilities and configuration between driver and device.
> (not to transport to send commands)
> 
> > 2) The configuration space is the place to advertise the capabilities, e.g what
> > kind of commands could the device accept.
> > 
> and it is not efficient and not used anymore with 1.3 and higher.
> They are exchanged using single get/set interface where device does not need to do special cross synchronization between what is exposed via interface_1 (config space) and control via interface_2 (cvq).

Why because you wrote that google doc? There never was any concensus on this.

> > 2) doesn't conflict with 1)
> > 
> CVQ does not conflict with anything as it stands today in 1.3.
> 
> > And we are discussing the provisioning which is more about 2) but not 1)
> > here.
> Provisioning will set the device and device will run based on what is being set.
> If MSIX is provisioned, it will show up in MSIX attribute.
> If the BAR region size is provisioned it will show in PCI MMIO size.
> If 1.3 legacy mac address configured, shows up in virtio_net_config.
> if 1.3 rss configuration, shows up in virtio net config
> 
> if 1.3 stats configuration, shows up in stats.
> If 1.4 flow filters provisioned shows in flow filter caps.
> 
> The primary tenet is: config space contains only the necessary driver bootstap fields.

I don't want to argue what bootstrap is. Please use existing spec
terminology. initialization time is a better definition we already use
in spec - it is clearly whatever info driver needs during probe and
whatever subsystem specific initialization callback is.  For example,
for network - ndo_open.


> The bright line in based on this usage: bootstap or not.
> If its runtime, sure have it over CVQ.
> Following the nice tenet of B.2 of the spec snippet: "Device configuration space should only be used for initialization-time parameters"
> That’s it.
> And everyone is already aligned to it.

Absolutely. Specifically, when do you expect driver to probe these
caps? As you yourself explained, it has to do it before ethtool
calls - that clearly means it will do it in probe.
To me this simply screams "initialization time".


> > 
> > That's it.
> > 
> > Thanks
> > 
> > 
> > >
> > > > Worst case devices can lie, it is not clear worrying about
> > > > statistics when provisioning is important enough.
> > > Device statistics cannot be broken when migrating.
> > >
> > > > And
> > > > you are now under the impression everyone got convinced when in
> > > > reality everyone just got tired of arguing. This is not a good pattern to try
> > and repeat.
> > > > Let me repeat the relevant part here for
> > > > you:
> > > >
> > > >       So why shouldn’t we just add some transport VQ on the owner
> > > > device to transport the SIOV device’s configuration?
> > > >
> > > >               Ans:
> > > >               Such addition means that hardware vendors need to
> > > > build runtime configurations in 4 different ways.
> > > >               One way using CVQ for PCI PF and VFs
> > > >               2nd way as backward compatible SIOV using owner’s
> > > > admin VQ
> > > >               3rd way using SIOV’s own CVQ channel for TDISP
> > > >               4th way using mix of backward compatible and secure
> > > > and efficient way using #2 and #3.
> > > >
> > > >
> > > > This is just a straw-man argument. No - we make a capability
> > > > optional,
> > > Capabilities are not optional.
> > > The design has undergone carefully review to ensure that driver honors the
> > capability, and it is migratable both.
> > >
> > > You dint explain why capability should be optional.
> > > The first patch has defined how the capabilities are used by the driver.
> > >
> > > > we
> > > > strongly suggest that *drivers* support both old and new mechanism,
> > > > and then *devices* will only implement what's required.
> > > There are other examples in the same document that makes things worst
> > with old and new.
> > >
> > > Also there is literally no way to enforce that driver supports both and new. It
> > is just sounds like an excuse to force infinite config space.
> > >
> > > The method proposed here is elegant and clearly promote one way to do
> > things for driver and device with predictability.
> > >
> > >
> 


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Fri, Nov 24, 2023 at 06:27:46AM +0000, Parav Pandit wrote:
> 
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Friday, November 24, 2023 11:37 AM
> > 
> > On Fri, Nov 24, 2023 at 05:53:02AM +0000, Parav Pandit wrote:
> > >
> > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > Sent: Friday, November 24, 2023 11:03 AM
> > > >
> > > > On Fri, Nov 24, 2023 at 12:02:23PM +0800, Jason Wang wrote:
> > > > > > > I won't be able to absorb this comment of DMA interface.
> > > > > > > If I discuss further, I will repeat the whole document [1] and
> > > > > > > I will avoid
> > > > that now.
> > > > > > >
> > > > > > > [1]
> > > > > > > https://docs.google.com/document/d/1Iyn-
> > > > l3Nm0yls3pZaul4lZiVj8x1s73
> > > > > > > Ed6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
> > > > > >
> > > > > >
> > > > > > I really worry about how provisioning will work. And I do not at
> > > > > > all cherish replicating all of these query capability commands for
> > provisioning.
> > > > >
> > > > > +1
> > > > >
> > > > > There's nothing that prevents the config space from being
> > > > > implemented in a way other than registers.
> > > >
> > > > Care doing it finally? Let's see if what Parav is worrying about is
> > > > then addressed.
> > >
> > > The whole concept that everything must be in one giant config space is just
> > simply bad.
> > > It does not exist either in virtio spec today.
> > 
> > But it does, this is what transports are doing.
> I don't understand what is "it".

"it" here is passing device init time configuration to drivers.


> Transport like pci transport bits.

in a simple, logical and functional manner.

> CVQ is functional object that helps to arrange the bits in logical, functional manner instead of trying to place them in bit array.

In a device specific way.

> > 
> > > once can see that what is presented in the commands cannot be placed in
> > config space at dynamic location.
> > > Same was the case with statistics too.
> > > Same was the case with VQ coaleasing knobs.
> > > Same with hash knobs.
> > > With flow filters,
> > > With rss contexts
> > > With rtc
> > > With new queues creation apis.
> > >
> > > The endless list continues...
> > >
> > > And reserving bits for future (for other than pad bytes) for future addition in
> > config space is equally not elegant design.
> > > Bits will get spread out at random location making things even harder to
> > maintain.
> > >
> > > The device is no longer a simple mac_addr + N queues device with some
> > static rss config anymore.
> > >
> > > With all modern work, every capability query and run time configuration is
> > done over cvq interface today.
> > > Single get/set channel from driver to device all using existing resources.
> > >
> > > The real hw device also does not need to refer to two places of config and
> > cvq when serving cvq commands.
> > > Oh, the list of advantages just continues with what 1.3 spec has done.
> > 
> > I don't see the problem sorry. We've been doing this for many years with many
> > ways to access config space.  It scaled well.
> > 
> I don't see it. sorry.
> The configuration of the device is done using cvq.

runtime configuration, absolutely. We found out writeable config space
field are painful for a variety of ways, the main one being device can't
report errors.  So we generally avoid writeable config space.  But read
only - no good reason to avoid them for init time things.




> > 
> > Then hardware offload guys come and say that in PCI spec current transport is
> > forcing use of on-device memory, and they want to build cheap offload PCI
> > based devices. Fine, let's build a transport variant that does not force this.
> 
> All new capabilities and control is over the cvq. What is baked until 1.2 is sort of legacy.

Just repeating this will not make everyone agree.

> >  And
> > we want optional compatibility, so let's also find a way to do that.  This makes
> > much more sense than forcing transport specific issues on everyone.
> > 
> Trying to attribute as some transport specific issue is just not aligned to the spec written today.
> 
> > To add to that, what did not historicall scale well is transport-specific registers.
> Then you should have put the VQ notification coalescing functionality in a horrible virtio_net_config register like how a queue reset in the common config space.

No because of the 4 commands: VIRTIO_NET_CTRL_NOTF_COAL_TX_SET
VIRTIO_NET_CTRL_NOTF_COAL_RX_SET VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET
VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET none are initialization time.
They are not capabilities: they control and inspec device
runtime state.

If we had VIRTIO_NET_CTRL_NOTF_COAL_CAP_GET that would belong in
config space.

> Thank God that mistake was not done and many similar mistakes were avoided.

Let's not get emotional here please.

> > That was a bad design, with all transports doing exactly the same thing is
> > slightly different ways.  And what you are advocating for with CVQ is exactly
> > replicating the bad design not the good one.
> 
> CVQ design is what extends the current spec in good way. Followed by many other non nvidia nics listed in the doc for reference.

I don't know what you are referring to here.  Register maps are all over
the place. It's a simple, standard, well understood practice.

We have some niche uses due to need for extreme VF# counts, this
forces DMA for them, not a good reason to force it for everyone.
The sooner you just stop forcing this down everyone's throat
the faster we can make progress on things that matter.


-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Fri, Nov 24, 2023 at 06:31:03AM +0000, Parav Pandit wrote:
> 
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Friday, November 24, 2023 11:48 AM
> > 
> > On Fri, Nov 24, 2023 at 05:40:26AM +0000, Parav Pandit wrote:
> > > > we
> > > > strongly suggest that *drivers* support both old and new mechanism,
> > > > and then *devices* will only implement what's required.
> > > There are other examples in the same document that makes things worst
> > with old and new.
> > >
> > > Also there is literally no way to enforce that driver supports both
> > > and new. It is just sounds like an excuse to force infinite config
> > > space.
> > 
> > There is a very simple method though.  We allow devices to expose a subset of
> > features when DMA is not used. So drivers that want maximum features will
> > always opt for DMA. We can also strongly recommend that all drivers support
> > DMA if available.
> Yeah, don't see how this is elegant at all with all mixed bits.

It's elegant because simple low end devices can cheaply implement MMIO
and not worry about DMA.

> Nor do I see any enforcement, single method via cvq still holds strong.

You don't need to enforce things, if people want to put a lot of
RAM on device and put it in a register let them.

> 
> > 
> > 
> > 
> > > The method proposed here is elegant and clearly promote one way to do
> > things for driver and device with predictability.
> > >
> > 
> > I don't see it as elegant at all. What is elegant is *a single tag* that describes
> > each property of the device. And this single tag should be good for everything:
> > driver, provisioning, migration. And config space offset serves as such.
> The single tag is the set of structures.

I have no idea how this will work. If migration format i started
reviewing is anything to go by then there will be a huge elaborate
structure nothing single or simple. By comparison there's already
a proposal how provisioning can work by supplying config space.
it is just a clean model to grasp.

> Provisioning access them via owner device.
> Member driver access them via CVQ or 1.2 legacy config space.


-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Friday, November 24, 2023 2:43 PM
> 
> On Fri, Nov 24, 2023 at 06:41:11AM +0000, Parav Pandit wrote:
> >
> > > From: Jason Wang <jasowang@redhat.com>
> > > Sent: Friday, November 24, 2023 11:32 AM
> >
> >
> > > Just to clarify my point, what's I'm saying is
> > >
> > > 1) For CVQ, it is used to send the commands to the device. It's not
> > > good to use transport to send those commands
> > This is not what is proposed.
> >
> > CVQ is proposed to exchange capabilities and configuration between driver
> and device.
> > (not to transport to send commands)
> >
> > > 2) The configuration space is the place to advertise the
> > > capabilities, e.g what kind of commands could the device accept.
> > >
> > and it is not efficient and not used anymore with 1.3 and higher.
> > They are exchanged using single get/set interface where device does not
> need to do special cross synchronization between what is exposed via
> interface_1 (config space) and control via interface_2 (cvq).
> 

> Why because you wrote that google doc? There never was any concensus on
> this.
No. The cross synchronization needs within the device to be done even without writing the doc.
The doc captures multiple discussions that captured the pros and cons of each in FAQ form and our view in this area.

> 
> > > 2) doesn't conflict with 1)
> > >
> > CVQ does not conflict with anything as it stands today in 1.3.
> >
> > > And we are discussing the provisioning which is more about 2) but
> > > not 1) here.
> > Provisioning will set the device and device will run based on what is being
> set.
> > If MSIX is provisioned, it will show up in MSIX attribute.
> > If the BAR region size is provisioned it will show in PCI MMIO size.
> > If 1.3 legacy mac address configured, shows up in virtio_net_config.
> > if 1.3 rss configuration, shows up in virtio net config
> >
> > if 1.3 stats configuration, shows up in stats.
> > If 1.4 flow filters provisioned shows in flow filter caps.
> >
> > The primary tenet is: config space contains only the necessary driver
> bootstap fields.
> 
> I don't want to argue what bootstrap is. Please use existing spec terminology.
> initialization time is a better definition we already use in spec - it is clearly
> whatever info driver needs during probe and whatever subsystem specific
> initialization callback is.  For example, for network - ndo_open.
> 
Initialization time as defined in spec as listed in " Device Initialization".
This means only upto DRIVER_OK phase.

Does the driver need to know about flow filter capabilities to initialize the device? No.

Can it be detected post initialization? i.e. after DRIVER_OK without burning some MMIO registers? Sure yes.

> 
> > The bright line in based on this usage: bootstap or not.
> > If its runtime, sure have it over CVQ.
> > Following the nice tenet of B.2 of the spec snippet: "Device configuration
> space should only be used for initialization-time parameters"
> > That’s it.
> > And everyone is already aligned to it.
> 
> Absolutely. Specifically, when do you expect driver to probe these caps? As
> you yourself explained, it has to do it before ethtool calls - that clearly means it
> will do it in probe.
> To me this simply screams "initialization time".
> 
It can be done in driver probe routine after DRIVER_OK, this is fine.

> 
> > >
> > > That's it.
> > >
> > > Thanks
> > >
> > >
> > > >
> > > > > Worst case devices can lie, it is not clear worrying about
> > > > > statistics when provisioning is important enough.
> > > > Device statistics cannot be broken when migrating.
> > > >
> > > > > And
> > > > > you are now under the impression everyone got convinced when in
> > > > > reality everyone just got tired of arguing. This is not a good
> > > > > pattern to try
> > > and repeat.
> > > > > Let me repeat the relevant part here for
> > > > > you:
> > > > >
> > > > >       So why shouldn’t we just add some transport VQ on the
> > > > > owner device to transport the SIOV device’s configuration?
> > > > >
> > > > >               Ans:
> > > > >               Such addition means that hardware vendors need to
> > > > > build runtime configurations in 4 different ways.
> > > > >               One way using CVQ for PCI PF and VFs
> > > > >               2nd way as backward compatible SIOV using owner’s
> > > > > admin VQ
> > > > >               3rd way using SIOV’s own CVQ channel for TDISP
> > > > >               4th way using mix of backward compatible and
> > > > > secure and efficient way using #2 and #3.
> > > > >
> > > > >
> > > > > This is just a straw-man argument. No - we make a capability
> > > > > optional,
> > > > Capabilities are not optional.
> > > > The design has undergone carefully review to ensure that driver
> > > > honors the
> > > capability, and it is migratable both.
> > > >
> > > > You dint explain why capability should be optional.
> > > > The first patch has defined how the capabilities are used by the driver.
> > > >
> > > > > we
> > > > > strongly suggest that *drivers* support both old and new
> > > > > mechanism, and then *devices* will only implement what's required.
> > > > There are other examples in the same document that makes things
> > > > worst
> > > with old and new.
> > > >
> > > > Also there is literally no way to enforce that driver supports
> > > > both and new. It
> > > is just sounds like an excuse to force infinite config space.
> > > >
> > > > The method proposed here is elegant and clearly promote one way to
> > > > do
> > > things for driver and device with predictability.
> > > >
> > > >
> >

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Friday, November 24, 2023 5:04 PM
> 
> On Fri, Nov 24, 2023 at 06:31:03AM +0000, Parav Pandit wrote:
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Friday, November 24, 2023 11:48 AM
> > >
> > > On Fri, Nov 24, 2023 at 05:40:26AM +0000, Parav Pandit wrote:
> > > > > we
> > > > > strongly suggest that *drivers* support both old and new
> > > > > mechanism, and then *devices* will only implement what's required.
> > > > There are other examples in the same document that makes things
> > > > worst
> > > with old and new.
> > > >
> > > > Also there is literally no way to enforce that driver supports
> > > > both and new. It is just sounds like an excuse to force infinite
> > > > config space.
> > >
> > > There is a very simple method though.  We allow devices to expose a
> > > subset of features when DMA is not used. So drivers that want
> > > maximum features will always opt for DMA. We can also strongly
> > > recommend that all drivers support DMA if available.
> > Yeah, don't see how this is elegant at all with all mixed bits.
> 
> It's elegant because simple low end devices can cheaply implement MMIO and
> not worry about DMA.
> 
It is not of much help in this case because any low end cheap device which want to support flow filter commands need to have CVQ anyway.
And hence reusing the same CVQ is more elegant that already does the DMA.

So CVQ is fulfilling all the below needs.
1. Single interface for the get/set config flow filters
2. DMA the data
3. Not have any partial issues
4. provides consistent structures that provisioning side will be able to use

> > Nor do I see any enforcement, single method via cvq still holds strong.
> 
> You don't need to enforce things, if people want to put a lot of RAM on device
> and put it in a register let them.
> 
Not enforced. It uses the CVQ for flow group and flow filter life cycles and for the sharing this config as well.
Also aligns with stats that rest also agreed on.

> >
> > >
> > >
> > >
> > > > The method proposed here is elegant and clearly promote one way to
> > > > do
> > > things for driver and device with predictability.
> > > >
> > >
> > > I don't see it as elegant at all. What is elegant is *a single tag*
> > > that describes each property of the device. And this single tag should be
> good for everything:
> > > driver, provisioning, migration. And config space offset serves as such.
> > The single tag is the set of structures.
> 
> I have no idea how this will work. If migration format i started reviewing is
> anything to go by then there will be a huge elaborate structure nothing single
> or simple. By comparison there's already a proposal how provisioning can
> work by supplying config space.
> it is just a clean model to grasp.
> 
The provisioning model is simple is to supply all the configuration.
To draw parallels to some sw side,

There is per functionality socket option to set things, instead of one giant structure.
There is per functionality ethtool option/cmd instead of Set ALL/get ALL enforcement.

> > Provisioning access them via owner device.
> > Member driver access them via CVQ or 1.2 legacy config space.
> 


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Friday, November 24, 2023 3:58 PM
> 
> On Fri, Nov 24, 2023 at 06:27:46AM +0000, Parav Pandit wrote:
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Friday, November 24, 2023 11:37 AM
> > >
> > > On Fri, Nov 24, 2023 at 05:53:02AM +0000, Parav Pandit wrote:
> > > >
> > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > Sent: Friday, November 24, 2023 11:03 AM
> > > > >
> > > > > On Fri, Nov 24, 2023 at 12:02:23PM +0800, Jason Wang wrote:
> > > > > > > > I won't be able to absorb this comment of DMA interface.
> > > > > > > > If I discuss further, I will repeat the whole document [1]
> > > > > > > > and I will avoid
> > > > > that now.
> > > > > > > >
> > > > > > > > [1]
> > > > > > > > https://docs.google.com/document/d/1Iyn-
> > > > > l3Nm0yls3pZaul4lZiVj8x1s73
> > > > > > > > Ed6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
> > > > > > >
> > > > > > >
> > > > > > > I really worry about how provisioning will work. And I do
> > > > > > > not at all cherish replicating all of these query capability
> > > > > > > commands for
> > > provisioning.
> > > > > >
> > > > > > +1
> > > > > >
> > > > > > There's nothing that prevents the config space from being
> > > > > > implemented in a way other than registers.
> > > > >
> > > > > Care doing it finally? Let's see if what Parav is worrying about
> > > > > is then addressed.
> > > >
> > > > The whole concept that everything must be in one giant config
> > > > space is just
> > > simply bad.
> > > > It does not exist either in virtio spec today.
> > >
> > > But it does, this is what transports are doing.
> > I don't understand what is "it".
> 
> "it" here is passing device init time configuration to drivers.
> 
> 
> > Transport like pci transport bits.
> 
> in a simple, logical and functional manner.
> 
> > CVQ is functional object that helps to arrange the bits in logical, functional
> manner instead of trying to place them in bit array.
> 
> In a device specific way.
> 
Yes, and in multiple of these examples, they are device specific structures based on functionality to extend.

> > >
> > > > once can see that what is presented in the commands cannot be
> > > > placed in
> > > config space at dynamic location.
> > > > Same was the case with statistics too.
> > > > Same was the case with VQ coaleasing knobs.
> > > > Same with hash knobs.
> > > > With flow filters,
> > > > With rss contexts
> > > > With rtc
> > > > With new queues creation apis.
> > > >
> > > > The endless list continues...
> > > >
> > > > And reserving bits for future (for other than pad bytes) for
> > > > future addition in
> > > config space is equally not elegant design.
> > > > Bits will get spread out at random location making things even
> > > > harder to
> > > maintain.
> > > >
> > > > The device is no longer a simple mac_addr + N queues device with
> > > > some
> > > static rss config anymore.
> > > >
> > > > With all modern work, every capability query and run time
> > > > configuration is
> > > done over cvq interface today.
> > > > Single get/set channel from driver to device all using existing resources.
> > > >
> > > > The real hw device also does not need to refer to two places of
> > > > config and
> > > cvq when serving cvq commands.
> > > > Oh, the list of advantages just continues with what 1.3 spec has done.
> > >
> > > I don't see the problem sorry. We've been doing this for many years
> > > with many ways to access config space.  It scaled well.
> > >
> > I don't see it. sorry.
> > The configuration of the device is done using cvq.
> 
> runtime configuration, absolutely. We found out writeable config space field
> are painful for a variety of ways, the main one being device can't report errors.
> So we generally avoid writeable config space.  But read only - no good reason
> to avoid them for init time things.
> 
The good reason as explained in the doc is to not place them based on the policy of read_only vs read-write, 
Instead based on the access pattern whether it is needed in early driver initialization time or can it be read at later point over.

And the based on the access pattern has given more flexibility to implementations across sw and hw based devices without any lose.

From the device side it has consistent view of get/set via single channel = cvq when dealing with the guest driver.

The two non nvidia examples of these devices are Microsoft MANA nic and Amazon ENA nic.

> 
> 
> 
> > >
> > > Then hardware offload guys come and say that in PCI spec current
> > > transport is forcing use of on-device memory, and they want to build
> > > cheap offload PCI based devices. Fine, let's build a transport variant that
> does not force this.
> >
> > All new capabilities and control is over the cvq. What is baked until 1.2 is sort
> of legacy.
> 
> Just repeating this will not make everyone agree.
> 

I see that Satananda also is fine for Marvell to use CVQ.

> > >  And
> > > we want optional compatibility, so let's also find a way to do that.
> > > This makes much more sense than forcing transport specific issues on
> everyone.
> > >
> > Trying to attribute as some transport specific issue is just not aligned to the
> spec written today.
> >
> > > To add to that, what did not historicall scale well is transport-specific
> registers.
> > Then you should have put the VQ notification coalescing functionality in a
> horrible virtio_net_config register like how a queue reset in the common config
> space.
> 
> No because of the 4 commands: VIRTIO_NET_CTRL_NOTF_COAL_TX_SET
> VIRTIO_NET_CTRL_NOTF_COAL_RX_SET
> VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET
> VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET none are initialization time.
> They are not capabilities: they control and inspec device runtime state.
> 
> If we had VIRTIO_NET_CTRL_NOTF_COAL_CAP_GET that would belong in
> config space.
> 
I understood you.
As we propose above, those caps reading is not must during initialization time phase.
Hence, cvq delivers better scale like stats and consistent get/set.

> > Thank God that mistake was not done and many similar mistakes were
> avoided.
> 
> Let's not get emotional here please.
> 
> > > That was a bad design, with all transports doing exactly the same
> > > thing is slightly different ways.  And what you are advocating for
> > > with CVQ is exactly replicating the bad design not the good one.
> >
> > CVQ design is what extends the current spec in good way. Followed by many
> other non nvidia nics listed in the doc for reference.
> 
> I don't know what you are referring to here.  Register maps are all over the
> place. It's a simple, standard, well understood practice.
> 
I provided various examples of virtio and non nvidia devices which prefers to avoid registers.
Satananda from Marvell also expressed their view.

> We have some niche uses due to need for extreme VF# counts, this forces
> DMA for them, not a good reason to force it for everyone.
PFs are equally benefiting from them. Some of our hw devices has large PF count.

> The sooner you just stop forcing this down everyone's throat the faster we can
> make progress on things that matter.

CVQ is established good configuration channel. So we all want to utilize instead of being forced to grow config space which is very hard to maintain and implement.

I will fix your comments in v7 regarding the honoring max limits in driver and device.

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Mon, Nov 27, 2023 at 10:19:45AM +0000, Parav Pandit wrote:
> > 
> > > The bright line in based on this usage: bootstap or not.
> > > If its runtime, sure have it over CVQ.
> > > Following the nice tenet of B.2 of the spec snippet: "Device configuration
> > space should only be used for initialization-time parameters"
> > > That’s it.
> > > And everyone is already aligned to it.
> > 
> > Absolutely. Specifically, when do you expect driver to probe these caps? As
> > you yourself explained, it has to do it before ethtool calls - that clearly means it
> > will do it in probe.
> > To me this simply screams "initialization time".
> > 
> It can be done in driver probe routine after DRIVER_OK, this is fine.

No - it is preferable to do it before DRIVER_OK because
VQs are probed and initialized before DRIVER_OK.
For example, it is reasonable to decide to only enable MQ
if specific flow control can later be enabled.

-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, November 27, 2023 4:40 PM
> 
> On Mon, Nov 27, 2023 at 10:19:45AM +0000, Parav Pandit wrote:
> > >
> > > > The bright line in based on this usage: bootstap or not.
> > > > If its runtime, sure have it over CVQ.
> > > > Following the nice tenet of B.2 of the spec snippet: "Device
> > > > configuration
> > > space should only be used for initialization-time parameters"
> > > > That’s it.
> > > > And everyone is already aligned to it.
> > >
> > > Absolutely. Specifically, when do you expect driver to probe these
> > > caps? As you yourself explained, it has to do it before ethtool
> > > calls - that clearly means it will do it in probe.
> > > To me this simply screams "initialization time".
> > >
> > It can be done in driver probe routine after DRIVER_OK, this is fine.
> 
> No - it is preferable to do it before DRIVER_OK because VQs are probed and
> initialized before DRIVER_OK.
> For example, it is reasonable to decide to only enable MQ if specific flow
> control can later be enabled.

For flow filter related caps has no hard dependency to it before driver_ok, same as stats.
MQ has users even without flow filter.

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Mon, Nov 27, 2023 at 10:19:49AM +0000, Parav Pandit wrote:
> 
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Friday, November 24, 2023 5:04 PM
> > 
> > On Fri, Nov 24, 2023 at 06:31:03AM +0000, Parav Pandit wrote:
> > >
> > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > Sent: Friday, November 24, 2023 11:48 AM
> > > >
> > > > On Fri, Nov 24, 2023 at 05:40:26AM +0000, Parav Pandit wrote:
> > > > > > we
> > > > > > strongly suggest that *drivers* support both old and new
> > > > > > mechanism, and then *devices* will only implement what's required.
> > > > > There are other examples in the same document that makes things
> > > > > worst
> > > > with old and new.
> > > > >
> > > > > Also there is literally no way to enforce that driver supports
> > > > > both and new. It is just sounds like an excuse to force infinite
> > > > > config space.
> > > >
> > > > There is a very simple method though.  We allow devices to expose a
> > > > subset of features when DMA is not used. So drivers that want
> > > > maximum features will always opt for DMA. We can also strongly
> > > > recommend that all drivers support DMA if available.
> > > Yeah, don't see how this is elegant at all with all mixed bits.
> > 
> > It's elegant because simple low end devices can cheaply implement MMIO and
> > not worry about DMA.
> > 
> It is not of much help in this case because any low end cheap device which want to support flow filter commands need to have CVQ anyway.
> And hence reusing the same CVQ is more elegant that already does the DMA.
> 
> So CVQ is fulfilling all the below needs.
> 1. Single interface for the get/set config flow filters
> 2. DMA the data
> 3. Not have any partial issues

I don't know what these are.

> 4. provides consistent structures that provisioning side will be able to use

Problem for provisioning is extra definitions will be needed, in a
device specific way.

> > > Nor do I see any enforcement, single method via cvq still holds strong.
> > 
> > You don't need to enforce things, if people want to put a lot of RAM on device
> > and put it in a register let them.
> > 
> Not enforced. It uses the CVQ for flow group and flow filter life cycles and for the sharing this config as well.
> Also aligns with stats that rest also agreed on.

I am talking about your attempt to generally say "no more config fields
everything must be in CVQ". I think it's wrong definitiely for non
network devices must sometimes for network too and generally we need
a solution for config over DMA. This specific thing - whether it
fits in CVQ is a separate discussion.


> > >
> > > >
> > > >
> > > >
> > > > > The method proposed here is elegant and clearly promote one way to
> > > > > do
> > > > things for driver and device with predictability.
> > > > >
> > > >
> > > > I don't see it as elegant at all. What is elegant is *a single tag*
> > > > that describes each property of the device. And this single tag should be
> > good for everything:
> > > > driver, provisioning, migration. And config space offset serves as such.
> > > The single tag is the set of structures.
> > 
> > I have no idea how this will work. If migration format i started reviewing is
> > anything to go by then there will be a huge elaborate structure nothing single
> > or simple. By comparison there's already a proposal how provisioning can
> > work by supplying config space.
> > it is just a clean model to grasp.
> > 
> The provisioning model is simple is to supply all the configuration.
> To draw parallels to some sw side,
> 
> There is per functionality socket option to set things, instead of one giant structure.
> There is per functionality ethtool option/cmd instead of Set ALL/get ALL enforcement.

I'm not sure how much of a parallel one can draw.
Do not see a lot of similarity.
Devices commonly use register map. Everyone understands
this paradigm.


I am not altogether happy with the way you are making
migration generate duplicate definitions for lots of
things we already have definitions for.
Having a 3rd one for provisioning? Gimme a break.


> > > Provisioning access them via owner device.
> > > Member driver access them via CVQ or 1.2 legacy config space.
> > 


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Mon, Nov 27, 2023 at 10:19:51AM +0000, Parav Pandit wrote:
> > runtime configuration, absolutely. We found out writeable config space field
> > are painful for a variety of ways, the main one being device can't report errors.
> > So we generally avoid writeable config space.  But read only - no good reason
> > to avoid them for init time things.
> > 
> The good reason as explained in the doc is to not place them based on the policy of read_only vs read-write, 
> Instead based on the access pattern whether it is needed in early driver initialization time or can it be read at later point over.
> 
> And the based on the access pattern has given more flexibility to implementations across sw and hw based devices without any lose.

Right. Whatever is done in driver probe.
And why? Basically because this makes initialization robust,
simple, and no parallelizm is used during probe anyway, so
a queue makes no sense - it gives 0 advantage.


And if you are going to require that driver allow or disallow ethtool
commands based on this info then it is clearly initialization time
in my book.


> >From the device side it has consistent view of get/set via single channel = cvq when dealing with the guest driver.
> 
> The two non nvidia examples of these devices are Microsoft MANA nic and Amazon ENA nic.
> 
> > 
> > 
> > 
> > > >
> > > > Then hardware offload guys come and say that in PCI spec current
> > > > transport is forcing use of on-device memory, and they want to build
> > > > cheap offload PCI based devices. Fine, let's build a transport variant that
> > does not force this.
> > >
> > > All new capabilities and control is over the cvq. What is baked until 1.2 is sort
> > of legacy.
> > 
> > Just repeating this will not make everyone agree.
> > 
> 
> I see that Satananda also is fine for Marvell to use CVQ.
> 
> > > >  And
> > > > we want optional compatibility, so let's also find a way to do that.
> > > > This makes much more sense than forcing transport specific issues on
> > everyone.
> > > >
> > > Trying to attribute as some transport specific issue is just not aligned to the
> > spec written today.
> > >
> > > > To add to that, what did not historicall scale well is transport-specific
> > registers.
> > > Then you should have put the VQ notification coalescing functionality in a
> > horrible virtio_net_config register like how a queue reset in the common config
> > space.
> > 
> > No because of the 4 commands: VIRTIO_NET_CTRL_NOTF_COAL_TX_SET
> > VIRTIO_NET_CTRL_NOTF_COAL_RX_SET
> > VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET
> > VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET none are initialization time.
> > They are not capabilities: they control and inspec device runtime state.
> > 
> > If we had VIRTIO_NET_CTRL_NOTF_COAL_CAP_GET that would belong in
> > config space.
> > 
> I understood you.
> As we propose above, those caps reading is not must during initialization time phase.

Yes, it is a must initialization time.
In my book initialization time is what happens in probe.

> Hence, cvq delivers better scale like stats and consistent get/set.
> 
> > > Thank God that mistake was not done and many similar mistakes were
> > avoided.
> > 
> > Let's not get emotional here please.
> > 
> > > > That was a bad design, with all transports doing exactly the same
> > > > thing is slightly different ways.  And what you are advocating for
> > > > with CVQ is exactly replicating the bad design not the good one.
> > >
> > > CVQ design is what extends the current spec in good way. Followed by many
> > other non nvidia nics listed in the doc for reference.
> > 
> > I don't know what you are referring to here.  Register maps are all over the
> > place. It's a simple, standard, well understood practice.
> > 
> I provided various examples of virtio and non nvidia devices which prefers to avoid registers.
> Satananda from Marvell also expressed their view.
> 
> > We have some niche uses due to need for extreme VF# counts, this forces
> > DMA for them, not a good reason to force it for everyone.
> PFs are equally benefiting from them. Some of our hw devices has large PF count.
> 
> > The sooner you just stop forcing this down everyone's throat the faster we can
> > make progress on things that matter.
> 
> CVQ is established good configuration channel. So we all want to utilize instead of being forced to grow config space which is very hard to maintain and implement.

Specifically this thing or generally?
This thing - I think I don't understand what this capability is, at this point,
so I am asking you to clarify much more.
If it's a huge array it can't fit in config space by definition
but I don't understand why it needs to be.


Generally - forget it. device config space is one of the few things that
virtio driver writers for different OS-es actually get consistently
right. It is simple, easy to use and understand. Limitations - slow,
serialized and small so only probe time and only little amounts of data.
Stop trying to kill it as a general capability and replace with a hodge
podge of commands I'm not going to support that.


> I will fix your comments in v7 regarding the honoring max limits in driver and device.


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, November 27, 2023 4:58 PM
> 
> On Mon, Nov 27, 2023 at 10:19:49AM +0000, Parav Pandit wrote:
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Friday, November 24, 2023 5:04 PM
> > >
> > > On Fri, Nov 24, 2023 at 06:31:03AM +0000, Parav Pandit wrote:
> > > >
> > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > Sent: Friday, November 24, 2023 11:48 AM
> > > > >
> > > > > On Fri, Nov 24, 2023 at 05:40:26AM +0000, Parav Pandit wrote:
> > > > > > > we
> > > > > > > strongly suggest that *drivers* support both old and new
> > > > > > > mechanism, and then *devices* will only implement what's
> required.
> > > > > > There are other examples in the same document that makes
> > > > > > things worst
> > > > > with old and new.
> > > > > >
> > > > > > Also there is literally no way to enforce that driver supports
> > > > > > both and new. It is just sounds like an excuse to force
> > > > > > infinite config space.
> > > > >
> > > > > There is a very simple method though.  We allow devices to
> > > > > expose a subset of features when DMA is not used. So drivers
> > > > > that want maximum features will always opt for DMA. We can also
> > > > > strongly recommend that all drivers support DMA if available.
> > > > Yeah, don't see how this is elegant at all with all mixed bits.
> > >
> > > It's elegant because simple low end devices can cheaply implement
> > > MMIO and not worry about DMA.
> > >
> > It is not of much help in this case because any low end cheap device which
> want to support flow filter commands need to have CVQ anyway.
> > And hence reusing the same CVQ is more elegant that already does the DMA.
> >
> > So CVQ is fulfilling all the below needs.
> > 1. Single interface for the get/set config flow filters 2. DMA the
> > data 3. Not have any partial issues
> 
> I don't know what these are.
I mean partial writes for fields.

> 
> > 4. provides consistent structures that provisioning side will be able
> > to use
> 
> Problem for provisioning is extra definitions will be needed, in a device specific
> way.
In vdpa tool and other OS tools of iproute2 developed, setting and getting those device specific values are useful.
It is ok.

> 
> > > > Nor do I see any enforcement, single method via cvq still holds strong.
> > >
> > > You don't need to enforce things, if people want to put a lot of RAM
> > > on device and put it in a register let them.
> > >
> > Not enforced. It uses the CVQ for flow group and flow filter life cycles and for
> the sharing this config as well.
> > Also aligns with stats that rest also agreed on.
> 
> I am talking about your attempt to generally say "no more config fields
> everything must be in CVQ". 
Config fields for initialization time is fine as the spec allows it today.
Things which can differ, it is ok to use cvq interface.

> I think it's wrong definitiely for non network
> devices must sometimes for network too and generally we need a solution for
> config over DMA. This specific thing - whether it fits in CVQ is a separate
> discussion.
> 
I explained it before, that 6 out of 19 devices has cvq which are complex enough doing things over cvq.
These are non-network devices already.

If one of those remaining device becomes complex, it is likely it will need a cvq to suffice for the dma interface and it can just do with depth = 1.

> 
> > > >
> > > > >
> > > > >
> > > > >
> > > > > > The method proposed here is elegant and clearly promote one
> > > > > > way to do
> > > > > things for driver and device with predictability.
> > > > > >
> > > > >
> > > > > I don't see it as elegant at all. What is elegant is *a single
> > > > > tag* that describes each property of the device. And this single
> > > > > tag should be
> > > good for everything:
> > > > > driver, provisioning, migration. And config space offset serves as such.
> > > > The single tag is the set of structures.
> > >
> > > I have no idea how this will work. If migration format i started
> > > reviewing is anything to go by then there will be a huge elaborate
> > > structure nothing single or simple. By comparison there's already a
> > > proposal how provisioning can work by supplying config space.
> > > it is just a clean model to grasp.
> > >
> > The provisioning model is simple is to supply all the configuration.
> > To draw parallels to some sw side,
> >
> > There is per functionality socket option to set things, instead of one giant
> structure.
> > There is per functionality ethtool option/cmd instead of Set ALL/get ALL
> enforcement.
> 
> I'm not sure how much of a parallel one can draw.
> Do not see a lot of similarity.
For lot of configuration they are similar that happens at slow pace.

> Devices commonly use register map. Everyone understands this paradigm.
> 
For initialization early device setup time, yes.

> 
> I am not altogether happy with the way you are making migration generate
> duplicate definitions for lots of things we already have definitions for.
> Having a 3rd one for provisioning? Gimme a break.

For migration, we are not duplicating. Some structures are not well defined, it has some duplication.
But large part seems be able to utilized pre-defined structs.
And here for flow filter also same structs will be used.

> 
> 
> > > > Provisioning access them via owner device.
> > > > Member driver access them via CVQ or 1.2 legacy config space.
> > >


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Mon, Nov 27, 2023 at 11:40:25AM +0000, Parav Pandit wrote:
> 
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, November 27, 2023 4:58 PM
> > 
> > On Mon, Nov 27, 2023 at 10:19:49AM +0000, Parav Pandit wrote:
> > >
> > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > Sent: Friday, November 24, 2023 5:04 PM
> > > >
> > > > On Fri, Nov 24, 2023 at 06:31:03AM +0000, Parav Pandit wrote:
> > > > >
> > > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > > Sent: Friday, November 24, 2023 11:48 AM
> > > > > >
> > > > > > On Fri, Nov 24, 2023 at 05:40:26AM +0000, Parav Pandit wrote:
> > > > > > > > we
> > > > > > > > strongly suggest that *drivers* support both old and new
> > > > > > > > mechanism, and then *devices* will only implement what's
> > required.
> > > > > > > There are other examples in the same document that makes
> > > > > > > things worst
> > > > > > with old and new.
> > > > > > >
> > > > > > > Also there is literally no way to enforce that driver supports
> > > > > > > both and new. It is just sounds like an excuse to force
> > > > > > > infinite config space.
> > > > > >
> > > > > > There is a very simple method though.  We allow devices to
> > > > > > expose a subset of features when DMA is not used. So drivers
> > > > > > that want maximum features will always opt for DMA. We can also
> > > > > > strongly recommend that all drivers support DMA if available.
> > > > > Yeah, don't see how this is elegant at all with all mixed bits.
> > > >
> > > > It's elegant because simple low end devices can cheaply implement
> > > > MMIO and not worry about DMA.
> > > >
> > > It is not of much help in this case because any low end cheap device which
> > want to support flow filter commands need to have CVQ anyway.
> > > And hence reusing the same CVQ is more elegant that already does the DMA.
> > >
> > > So CVQ is fulfilling all the below needs.
> > > 1. Single interface for the get/set config flow filters 2. DMA the
> > > data 3. Not have any partial issues
> > 
> > I don't know what these are.
> I mean partial writes for fields.

We don't allow these and in any case, writeble fields are best avoided.


> > 
> > > 4. provides consistent structures that provisioning side will be able
> > > to use
> > 
> > Problem for provisioning is extra definitions will be needed, in a device specific
> > way.
> In vdpa tool and other OS tools of iproute2 developed, setting and getting those device specific values are useful.
> It is ok.

It does not become ok just by saying so. You are taking a single RO
value and instead of it having an address there are now 2 other ways
to address it. And you fail to see the problem and the pain you are
inflicting on software developers. Just stick with an address if you
can.

> > 
> > > > > Nor do I see any enforcement, single method via cvq still holds strong.
> > > >
> > > > You don't need to enforce things, if people want to put a lot of RAM
> > > > on device and put it in a register let them.
> > > >
> > > Not enforced. It uses the CVQ for flow group and flow filter life cycles and for
> > the sharing this config as well.
> > > Also aligns with stats that rest also agreed on.
> > 
> > I am talking about your attempt to generally say "no more config fields
> > everything must be in CVQ". 
> Config fields for initialization time is fine as the spec allows it today.
> Things which can differ, it is ok to use cvq interface.

I don't know what does "Things that can differ" means. Generally
device caps are perfect for config space. Accessed at init time only,
RO.

> > I think it's wrong definitiely for non network
> > devices must sometimes for network too and generally we need a solution for
> > config over DMA. This specific thing - whether it fits in CVQ is a separate
> > discussion.
> > 
> I explained it before, that 6 out of 19 devices has cvq which are complex enough doing things over cvq.
> These are non-network devices already.
> 
> If one of those remaining device becomes complex, it is likely it will need a cvq to suffice for the dma interface and it can just do with depth = 1.

Using generic caps and not net specific ones is a good idea.



> > 
> > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > > The method proposed here is elegant and clearly promote one
> > > > > > > way to do
> > > > > > things for driver and device with predictability.
> > > > > > >
> > > > > >
> > > > > > I don't see it as elegant at all. What is elegant is *a single
> > > > > > tag* that describes each property of the device. And this single
> > > > > > tag should be
> > > > good for everything:
> > > > > > driver, provisioning, migration. And config space offset serves as such.
> > > > > The single tag is the set of structures.
> > > >
> > > > I have no idea how this will work. If migration format i started
> > > > reviewing is anything to go by then there will be a huge elaborate
> > > > structure nothing single or simple. By comparison there's already a
> > > > proposal how provisioning can work by supplying config space.
> > > > it is just a clean model to grasp.
> > > >
> > > The provisioning model is simple is to supply all the configuration.
> > > To draw parallels to some sw side,
> > >
> > > There is per functionality socket option to set things, instead of one giant
> > structure.
> > > There is per functionality ethtool option/cmd instead of Set ALL/get ALL
> > enforcement.
> > 
> > I'm not sure how much of a parallel one can draw.
> > Do not see a lot of similarity.
> For lot of configuration they are similar that happens at slow pace.
> 
> > Devices commonly use register map. Everyone understands this paradigm.
> > 
> For initialization early device setup time, yes.
> 
> > 
> > I am not altogether happy with the way you are making migration generate
> > duplicate definitions for lots of things we already have definitions for.
> > Having a 3rd one for provisioning? Gimme a break.
> 
> For migration, we are not duplicating. Some structures are not well defined, it has some duplication.

And fyi it's already making people unhappy.

> But large part seems be able to utilized pre-defined structs.
> And here for flow filter also same structs will be used.

So if there's a 64 bit bitmap in config space, then provisioning
command which already gets config space can just use its offset.
Simpler, better.


> > 
> > 
> > > > > Provisioning access them via owner device.
> > > > > Member driver access them via CVQ or 1.2 legacy config space.
> > > >


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, November 27, 2023 5:08 PM
> To: Parav Pandit <parav@nvidia.com>
> Cc: Jason Wang <jasowang@redhat.com>; virtio-comment@lists.oasis-
> open.org; cohuck@redhat.com; sburla@marvell.com; Shahaf Shuler
> <shahafs@nvidia.com>; si-wei.liu@oracle.com; xuanzhuo@linux.alibaba.com;
> Heng Qi <hengqi@linux.alibaba.com>
> Subject: Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter
> capabilities read commands
> 
> On Mon, Nov 27, 2023 at 10:19:51AM +0000, Parav Pandit wrote:
> > > runtime configuration, absolutely. We found out writeable config
> > > space field are painful for a variety of ways, the main one being device can't
> report errors.
> > > So we generally avoid writeable config space.  But read only - no
> > > good reason to avoid them for init time things.
> > >
> > The good reason as explained in the doc is to not place them based on
> > the policy of read_only vs read-write, Instead based on the access pattern
> whether it is needed in early driver initialization time or can it be read at later
> point over.
> >
> > And the based on the access pattern has given more flexibility to
> implementations across sw and hw based devices without any lose.
> 
> Right. Whatever is done in driver probe.
> And why? Basically because this makes initialization robust, simple, and no
> parallelizm is used during probe anyway, so a queue makes no sense - it gives 0
> advantage.
> 
I listed all advantages.

> 
> And if you are going to require that driver allow or disallow ethtool commands
> based on this info then it is clearly initialization time in my book.

I understood your view.
It is less efficient and works well over cvq with listed advantages.

> 
> 
> > >From the device side it has consistent view of get/set via single channel =
> cvq when dealing with the guest driver.
> >
> > The two non nvidia examples of these devices are Microsoft MANA nic and
> Amazon ENA nic.
> >
> > >
> > >
> > >
> > > > >
> > > > > Then hardware offload guys come and say that in PCI spec current
> > > > > transport is forcing use of on-device memory, and they want to
> > > > > build cheap offload PCI based devices. Fine, let's build a
> > > > > transport variant that
> > > does not force this.
> > > >
> > > > All new capabilities and control is over the cvq. What is baked
> > > > until 1.2 is sort
> > > of legacy.
> > >
> > > Just repeating this will not make everyone agree.
> > >
> >
> > I see that Satananda also is fine for Marvell to use CVQ.
> >
> > > > >  And
> > > > > we want optional compatibility, so let's also find a way to do that.
> > > > > This makes much more sense than forcing transport specific
> > > > > issues on
> > > everyone.
> > > > >
> > > > Trying to attribute as some transport specific issue is just not
> > > > aligned to the
> > > spec written today.
> > > >
> > > > > To add to that, what did not historicall scale well is
> > > > > transport-specific
> > > registers.
> > > > Then you should have put the VQ notification coalescing
> > > > functionality in a
> > > horrible virtio_net_config register like how a queue reset in the
> > > common config space.
> > >
> > > No because of the 4 commands: VIRTIO_NET_CTRL_NOTF_COAL_TX_SET
> > > VIRTIO_NET_CTRL_NOTF_COAL_RX_SET
> VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET
> > > VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET none are initialization time.
> > > They are not capabilities: they control and inspec device runtime state.
> > >
> > > If we had VIRTIO_NET_CTRL_NOTF_COAL_CAP_GET that would belong in
> > > config space.
> > >
> > I understood you.
> > As we propose above, those caps reading is not must during initialization
> time phase.
> 
> Yes, it is a must initialization time.
Why it is must?

> In my book initialization time is what happens in probe.
And asking that this initialization time to somehow use the DMA and interrupt is not making the initialization crisper.
In that case the DMA interface scope must be before driver_ok to strickly fullfil the need of initialization time.
And that way it wont be usable for other things.
And bending that for other work is not helpful either.

So you should let things progress.

> 
> > Hence, cvq delivers better scale like stats and consistent get/set.
> >
> > > > Thank God that mistake was not done and many similar mistakes were
> > > avoided.
> > >
> > > Let's not get emotional here please.
> > >
> > > > > That was a bad design, with all transports doing exactly the
> > > > > same thing is slightly different ways.  And what you are
> > > > > advocating for with CVQ is exactly replicating the bad design not the
> good one.
> > > >
> > > > CVQ design is what extends the current spec in good way. Followed
> > > > by many
> > > other non nvidia nics listed in the doc for reference.
> > >
> > > I don't know what you are referring to here.  Register maps are all
> > > over the place. It's a simple, standard, well understood practice.
> > >
> > I provided various examples of virtio and non nvidia devices which prefers to
> avoid registers.
> > Satananda from Marvell also expressed their view.
> >
> > > We have some niche uses due to need for extreme VF# counts, this
> > > forces DMA for them, not a good reason to force it for everyone.
> > PFs are equally benefiting from them. Some of our hw devices has large PF
> count.
> >
> > > The sooner you just stop forcing this down everyone's throat the
> > > faster we can make progress on things that matter.
> >
> > CVQ is established good configuration channel. So we all want to utilize
> instead of being forced to grow config space which is very hard to maintain
> and implement.
> 
> Specifically this thing or generally?

> This thing - I think I don't understand what this capability is, at this point, so I
> am asking you to clarify much more.
> If it's a huge array it can't fit in config space by definition but I don't
> understand why it needs to be.
Each array entry is indicating what it tells.
> 
> 
> Generally - forget it. device config space is one of the few things that virtio
> driver writers for different OS-es actually get consistently right. It is simple,
> easy to use and understand.

> Limitations - slow, serialized and small so only
> probe time and only little amounts of data.
> Stop trying to kill it as a general capability and replace with a hodge podge of
> commands I'm not going to support that.
For small things that is common across all devices can stay like that.
Here is it per device unique data values.
In context of flow filter it is just overkill to put all of this in the config space.
It is bigger content that what we query via stats cap command.

> 
> > I will fix your comments in v7 regarding the honoring max limits in driver and
> device.


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, November 27, 2023 5:17 PM
> 
> On Mon, Nov 27, 2023 at 11:40:25AM +0000, Parav Pandit wrote:
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Monday, November 27, 2023 4:58 PM
> > >
> > > On Mon, Nov 27, 2023 at 10:19:49AM +0000, Parav Pandit wrote:
> > > >
> > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > Sent: Friday, November 24, 2023 5:04 PM
> > > > >
> > > > > On Fri, Nov 24, 2023 at 06:31:03AM +0000, Parav Pandit wrote:
> > > > > >
> > > > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > > > Sent: Friday, November 24, 2023 11:48 AM
> > > > > > >
> > > > > > > On Fri, Nov 24, 2023 at 05:40:26AM +0000, Parav Pandit wrote:
> > > > > > > > > we
> > > > > > > > > strongly suggest that *drivers* support both old and new
> > > > > > > > > mechanism, and then *devices* will only implement what's
> > > required.
> > > > > > > > There are other examples in the same document that makes
> > > > > > > > things worst
> > > > > > > with old and new.
> > > > > > > >
> > > > > > > > Also there is literally no way to enforce that driver
> > > > > > > > supports both and new. It is just sounds like an excuse to
> > > > > > > > force infinite config space.
> > > > > > >
> > > > > > > There is a very simple method though.  We allow devices to
> > > > > > > expose a subset of features when DMA is not used. So drivers
> > > > > > > that want maximum features will always opt for DMA. We can
> > > > > > > also strongly recommend that all drivers support DMA if available.
> > > > > > Yeah, don't see how this is elegant at all with all mixed bits.
> > > > >
> > > > > It's elegant because simple low end devices can cheaply
> > > > > implement MMIO and not worry about DMA.
> > > > >
> > > > It is not of much help in this case because any low end cheap
> > > > device which
> > > want to support flow filter commands need to have CVQ anyway.
> > > > And hence reusing the same CVQ is more elegant that already does the
> DMA.
> > > >
> > > > So CVQ is fulfilling all the below needs.
> > > > 1. Single interface for the get/set config flow filters 2. DMA the
> > > > data 3. Not have any partial issues
> > >
> > > I don't know what these are.
> > I mean partial writes for fields.
> 
> We don't allow these and in any case, writeble fields are best avoided.
> 
Right. This is why flow filter objects are created using cvq, and its associated caps also comes via same cvq channel.

> 
> > >
> > > > 4. provides consistent structures that provisioning side will be
> > > > able to use
> > >
> > > Problem for provisioning is extra definitions will be needed, in a
> > > device specific way.
> > In vdpa tool and other OS tools of iproute2 developed, setting and getting
> those device specific values are useful.
> > It is ok.
> 
> It does not become ok just by saying so. You are taking a single RO value and
> instead of it having an address there are now 2 other ways to address it. And
> you fail to see the problem and the pain you are inflicting on software
> developers. Just stick with an address if you can.
There is zero problem with sw.
Sw just need to issue send_command() and done with it, like rest of the commands.
A pain would be create yet another DMA interface.

> 
> > >
> > > > > > Nor do I see any enforcement, single method via cvq still holds strong.
> > > > >
> > > > > You don't need to enforce things, if people want to put a lot of
> > > > > RAM on device and put it in a register let them.
> > > > >
> > > > Not enforced. It uses the CVQ for flow group and flow filter life
> > > > cycles and for
> > > the sharing this config as well.
> > > > Also aligns with stats that rest also agreed on.
> > >
> > > I am talking about your attempt to generally say "no more config
> > > fields everything must be in CVQ".
> > Config fields for initialization time is fine as the spec allows it today.
> > Things which can differ, it is ok to use cvq interface.
> 
> I don't know what does "Things that can differ" means. Generally device caps
> are perfect for config space. Accessed at init time only, RO.
> 
You ignore the comment I answered before that proposal here is not based on RO/RW.
It is based on initialization time vs run time.

> > > I think it's wrong definitiely for non network devices must
> > > sometimes for network too and generally we need a solution for
> > > config over DMA. This specific thing - whether it fits in CVQ is a
> > > separate discussion.
> > >
> > I explained it before, that 6 out of 19 devices has cvq which are complex
> enough doing things over cvq.
> > These are non-network devices already.
> >
> > If one of those remaining device becomes complex, it is likely it will need a
> cvq to suffice for the dma interface and it can just do with depth = 1.
> 
> Using generic caps and not net specific ones is a good idea.
> 
context here is cvq and net.

> 
> 
> > >
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > The method proposed here is elegant and clearly promote
> > > > > > > > one way to do
> > > > > > > things for driver and device with predictability.
> > > > > > > >
> > > > > > >
> > > > > > > I don't see it as elegant at all. What is elegant is *a
> > > > > > > single
> > > > > > > tag* that describes each property of the device. And this
> > > > > > > single tag should be
> > > > > good for everything:
> > > > > > > driver, provisioning, migration. And config space offset serves as
> such.
> > > > > > The single tag is the set of structures.
> > > > >
> > > > > I have no idea how this will work. If migration format i started
> > > > > reviewing is anything to go by then there will be a huge
> > > > > elaborate structure nothing single or simple. By comparison
> > > > > there's already a proposal how provisioning can work by supplying
> config space.
> > > > > it is just a clean model to grasp.
> > > > >
> > > > The provisioning model is simple is to supply all the configuration.
> > > > To draw parallels to some sw side,
> > > >
> > > > There is per functionality socket option to set things, instead of
> > > > one giant
> > > structure.
> > > > There is per functionality ethtool option/cmd instead of Set
> > > > ALL/get ALL
> > > enforcement.
> > >
> > > I'm not sure how much of a parallel one can draw.
> > > Do not see a lot of similarity.
> > For lot of configuration they are similar that happens at slow pace.
> >
> > > Devices commonly use register map. Everyone understands this paradigm.
> > >
> > For initialization early device setup time, yes.
> >
> > >
> > > I am not altogether happy with the way you are making migration
> > > generate duplicate definitions for lots of things we already have definitions
> for.
> > > Having a 3rd one for provisioning? Gimme a break.
> >
> > For migration, we are not duplicating. Some structures are not well defined,
> it has some duplication.
> 
> And fyi it's already making people unhappy.
> 
Those exceptions are not the interesting one to take as example here.

> > But large part seems be able to utilized pre-defined structs.
> > And here for flow filter also same structs will be used.
> 
> So if there's a 64 bit bitmap in config space, then provisioning command which
> already gets config space can just use its offset.
> Simpler, better.
> 
It is not simple to implement per device unique config space as we discussed already.
And no need another DMA interface either as cvq service that need already.

> 
> > >
> > >
> > > > > > Provisioning access them via owner device.
> > > > > > Member driver access them via CVQ or 1.2 legacy config space.
> > > > >


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Mon, Nov 27, 2023 at 11:54:52AM +0000, Parav Pandit wrote:
> 
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, November 27, 2023 5:17 PM
> > 
> > On Mon, Nov 27, 2023 at 11:40:25AM +0000, Parav Pandit wrote:
> > >
> > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > Sent: Monday, November 27, 2023 4:58 PM
> > > >
> > > > On Mon, Nov 27, 2023 at 10:19:49AM +0000, Parav Pandit wrote:
> > > > >
> > > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > > Sent: Friday, November 24, 2023 5:04 PM
> > > > > >
> > > > > > On Fri, Nov 24, 2023 at 06:31:03AM +0000, Parav Pandit wrote:
> > > > > > >
> > > > > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > > > > Sent: Friday, November 24, 2023 11:48 AM
> > > > > > > >
> > > > > > > > On Fri, Nov 24, 2023 at 05:40:26AM +0000, Parav Pandit wrote:
> > > > > > > > > > we
> > > > > > > > > > strongly suggest that *drivers* support both old and new
> > > > > > > > > > mechanism, and then *devices* will only implement what's
> > > > required.
> > > > > > > > > There are other examples in the same document that makes
> > > > > > > > > things worst
> > > > > > > > with old and new.
> > > > > > > > >
> > > > > > > > > Also there is literally no way to enforce that driver
> > > > > > > > > supports both and new. It is just sounds like an excuse to
> > > > > > > > > force infinite config space.
> > > > > > > >
> > > > > > > > There is a very simple method though.  We allow devices to
> > > > > > > > expose a subset of features when DMA is not used. So drivers
> > > > > > > > that want maximum features will always opt for DMA. We can
> > > > > > > > also strongly recommend that all drivers support DMA if available.
> > > > > > > Yeah, don't see how this is elegant at all with all mixed bits.
> > > > > >
> > > > > > It's elegant because simple low end devices can cheaply
> > > > > > implement MMIO and not worry about DMA.
> > > > > >
> > > > > It is not of much help in this case because any low end cheap
> > > > > device which
> > > > want to support flow filter commands need to have CVQ anyway.
> > > > > And hence reusing the same CVQ is more elegant that already does the
> > DMA.
> > > > >
> > > > > So CVQ is fulfilling all the below needs.
> > > > > 1. Single interface for the get/set config flow filters 2. DMA the
> > > > > data 3. Not have any partial issues
> > > >
> > > > I don't know what these are.
> > > I mean partial writes for fields.
> > 
> > We don't allow these and in any case, writeble fields are best avoided.
> > 
> Right. This is why flow filter objects are created using cvq, and its associated caps also comes via same cvq channel.
> 
> > 
> > > >
> > > > > 4. provides consistent structures that provisioning side will be
> > > > > able to use
> > > >
> > > > Problem for provisioning is extra definitions will be needed, in a
> > > > device specific way.
> > > In vdpa tool and other OS tools of iproute2 developed, setting and getting
> > those device specific values are useful.
> > > It is ok.
> > 
> > It does not become ok just by saying so. You are taking a single RO value and
> > instead of it having an address there are now 2 other ways to address it. And
> > you fail to see the problem and the pain you are inflicting on software
> > developers. Just stick with an address if you can.
> There is zero problem with sw.
> Sw just need to issue send_command() and done with it, like rest of the commands.
> A pain would be create yet another DMA interface.

No because it will be a generic thing for all types.


> > 
> > > >
> > > > > > > Nor do I see any enforcement, single method via cvq still holds strong.
> > > > > >
> > > > > > You don't need to enforce things, if people want to put a lot of
> > > > > > RAM on device and put it in a register let them.
> > > > > >
> > > > > Not enforced. It uses the CVQ for flow group and flow filter life
> > > > > cycles and for
> > > > the sharing this config as well.
> > > > > Also aligns with stats that rest also agreed on.
> > > >
> > > > I am talking about your attempt to generally say "no more config
> > > > fields everything must be in CVQ".
> > > Config fields for initialization time is fine as the spec allows it today.
> > > Things which can differ, it is ok to use cvq interface.
> > 
> > I don't know what does "Things that can differ" means. Generally device caps
> > are perfect for config space. Accessed at init time only, RO.
> > 
> You ignore the comment I answered before that proposal here is not based on RO/RW.
> It is based on initialization time vs run time.

initialization time is really probe. That thing which only happens
once when driver is setup. Clearly there is no reason to
check capabilities many times so it will be done during setup,
once.


> > > > I think it's wrong definitiely for non network devices must
> > > > sometimes for network too and generally we need a solution for
> > > > config over DMA. This specific thing - whether it fits in CVQ is a
> > > > separate discussion.
> > > >
> > > I explained it before, that 6 out of 19 devices has cvq which are complex
> > enough doing things over cvq.
> > > These are non-network devices already.
> > >
> > > If one of those remaining device becomes complex, it is likely it will need a
> > cvq to suffice for the dma interface and it can just do with depth = 1.
> > 
> > Using generic caps and not net specific ones is a good idea.
> > 
> context here is cvq and net.

yes, you can use cvq command for this.  this will only serve net.  I am
saying a generic interface for config over DMA is better and will
serve everyone.

We are doing this commonly, and when we did not push for a generic
interface like with e.g. s/g limits for blk we later regret it.


> > 
> > 
> > > >
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > The method proposed here is elegant and clearly promote
> > > > > > > > > one way to do
> > > > > > > > things for driver and device with predictability.
> > > > > > > > >
> > > > > > > >
> > > > > > > > I don't see it as elegant at all. What is elegant is *a
> > > > > > > > single
> > > > > > > > tag* that describes each property of the device. And this
> > > > > > > > single tag should be
> > > > > > good for everything:
> > > > > > > > driver, provisioning, migration. And config space offset serves as
> > such.
> > > > > > > The single tag is the set of structures.
> > > > > >
> > > > > > I have no idea how this will work. If migration format i started
> > > > > > reviewing is anything to go by then there will be a huge
> > > > > > elaborate structure nothing single or simple. By comparison
> > > > > > there's already a proposal how provisioning can work by supplying
> > config space.
> > > > > > it is just a clean model to grasp.
> > > > > >
> > > > > The provisioning model is simple is to supply all the configuration.
> > > > > To draw parallels to some sw side,
> > > > >
> > > > > There is per functionality socket option to set things, instead of
> > > > > one giant
> > > > structure.
> > > > > There is per functionality ethtool option/cmd instead of Set
> > > > > ALL/get ALL
> > > > enforcement.
> > > >
> > > > I'm not sure how much of a parallel one can draw.
> > > > Do not see a lot of similarity.
> > > For lot of configuration they are similar that happens at slow pace.
> > >
> > > > Devices commonly use register map. Everyone understands this paradigm.
> > > >
> > > For initialization early device setup time, yes.
> > >
> > > >
> > > > I am not altogether happy with the way you are making migration
> > > > generate duplicate definitions for lots of things we already have definitions
> > for.
> > > > Having a 3rd one for provisioning? Gimme a break.
> > >
> > > For migration, we are not duplicating. Some structures are not well defined,
> > it has some duplication.
> > 
> > And fyi it's already making people unhappy.
> > 
> Those exceptions are not the interesting one to take as example here.
> 
> > > But large part seems be able to utilized pre-defined structs.
> > > And here for flow filter also same structs will be used.
> > 
> > So if there's a 64 bit bitmap in config space, then provisioning command which
> > already gets config space can just use its offset.
> > Simpler, better.
> > 
> It is not simple to implement per device unique config space as we discussed already.

we did it for many years and many device types.
99% of devices do config over a register map of some kind.
It is not as *flexible* true but flexibility here is an enemy,
too easy to make a mess.


And your array of types where a single field would suffice
is a classical case in point. config space keeps us disciplined.

> And no need another DMA interface either as cvq service that need already.

This is too focused on net and short term, for my taste.
we need it so we
1. don't need to have this discussion every time
2. can begin to save on registers for config space in X years and
   for new device types


And, while apparently focusing on short term people also somehow manage to
make a point out of every bikeshed instead of the "let's just ship it"
attitude that would be consistent. I find this baffling.


-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Mon, Nov 27, 2023 at 11:12:47AM +0000, Parav Pandit wrote:
> 
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, November 27, 2023 4:40 PM
> > 
> > On Mon, Nov 27, 2023 at 10:19:45AM +0000, Parav Pandit wrote:
> > > >
> > > > > The bright line in based on this usage: bootstap or not.
> > > > > If its runtime, sure have it over CVQ.
> > > > > Following the nice tenet of B.2 of the spec snippet: "Device
> > > > > configuration
> > > > space should only be used for initialization-time parameters"
> > > > > That’s it.
> > > > > And everyone is already aligned to it.
> > > >
> > > > Absolutely. Specifically, when do you expect driver to probe these
> > > > caps? As you yourself explained, it has to do it before ethtool
> > > > calls - that clearly means it will do it in probe.
> > > > To me this simply screams "initialization time".
> > > >
> > > It can be done in driver probe routine after DRIVER_OK, this is fine.
> > 
> > No - it is preferable to do it before DRIVER_OK because VQs are probed and
> > initialized before DRIVER_OK.
> > For example, it is reasonable to decide to only enable MQ if specific flow
> > control can later be enabled.
> 
> For flow filter related caps has no hard dependency to it before driver_ok, same as stats.
> MQ has users even without flow filter.

driver ok is not the end of driver initialization. Things like
pre-populating receive queues are also part of that.

Yes, there is a dependency.  The capability is either not useful at all
(as in the current form - driver makes no decisions here so it
is pointless). Or it will be useful and then it has to be
probed at init time so the data is available at runtime
when decisions are to be made. How *hard* is this dependency?
I don't know. I do know that 100% of drivers will either not
use this command at all or use it at init time.


-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, November 27, 2023 6:22 PM
> 
> On Mon, Nov 27, 2023 at 11:12:47AM +0000, Parav Pandit wrote:
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Monday, November 27, 2023 4:40 PM
> > >
> > > On Mon, Nov 27, 2023 at 10:19:45AM +0000, Parav Pandit wrote:
> > > > >
> > > > > > The bright line in based on this usage: bootstap or not.
> > > > > > If its runtime, sure have it over CVQ.
> > > > > > Following the nice tenet of B.2 of the spec snippet: "Device
> > > > > > configuration
> > > > > space should only be used for initialization-time parameters"
> > > > > > That’s it.
> > > > > > And everyone is already aligned to it.
> > > > >
> > > > > Absolutely. Specifically, when do you expect driver to probe
> > > > > these caps? As you yourself explained, it has to do it before
> > > > > ethtool calls - that clearly means it will do it in probe.
> > > > > To me this simply screams "initialization time".
> > > > >
> > > > It can be done in driver probe routine after DRIVER_OK, this is fine.
> > >
> > > No - it is preferable to do it before DRIVER_OK because VQs are
> > > probed and initialized before DRIVER_OK.
> > > For example, it is reasonable to decide to only enable MQ if
> > > specific flow control can later be enabled.
> >
> > For flow filter related caps has no hard dependency to it before driver_ok,
> same as stats.
> > MQ has users even without flow filter.
> 
> driver ok is not the end of driver initialization. Things like pre-populating
> receive queues are also part of that.

Rq populating is unrelated here.

> 
> Yes, there is a dependency.  The capability is either not useful at all (as in the
> current form - driver makes no decisions here so it is pointless). 
It is making decisions based on group and flow id ranges.
As I acked few times, that normative lines are missing and I am in middle of adding them for v7.

> Or it will be
> useful and then it has to be probed at init time so the data is available at
> runtime when decisions are to be made. How *hard* is this dependency?
It is wrong for me to put this cap in the config space which is not needed for the device initialization.

> I don't know. I do know that 100% of drivers will either not use this command
> at all or use it at init time.

I don’t see without this how driver can work as it does not know the valid id and group ranges to use.

Driver can use this command at init time before registering the device to the OS.

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Parav Pandit]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, November 27, 2023 6:13 PM
> 
> On Mon, Nov 27, 2023 at 11:54:52AM +0000, Parav Pandit wrote:
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Monday, November 27, 2023 5:17 PM
> > >
> > > On Mon, Nov 27, 2023 at 11:40:25AM +0000, Parav Pandit wrote:
> > > >
> > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > Sent: Monday, November 27, 2023 4:58 PM
> > > > >
> > > > > On Mon, Nov 27, 2023 at 10:19:49AM +0000, Parav Pandit wrote:
> > > > > >
> > > > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > > > Sent: Friday, November 24, 2023 5:04 PM
> > > > > > >
> > > > > > > On Fri, Nov 24, 2023 at 06:31:03AM +0000, Parav Pandit wrote:
> > > > > > > >
> > > > > > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > > > > > Sent: Friday, November 24, 2023 11:48 AM
> > > > > > > > >
> > > > > > > > > On Fri, Nov 24, 2023 at 05:40:26AM +0000, Parav Pandit wrote:
> > > > > > > > > > > we
> > > > > > > > > > > strongly suggest that *drivers* support both old and
> > > > > > > > > > > new mechanism, and then *devices* will only
> > > > > > > > > > > implement what's
> > > > > required.
> > > > > > > > > > There are other examples in the same document that
> > > > > > > > > > makes things worst
> > > > > > > > > with old and new.
> > > > > > > > > >
> > > > > > > > > > Also there is literally no way to enforce that driver
> > > > > > > > > > supports both and new. It is just sounds like an
> > > > > > > > > > excuse to force infinite config space.
> > > > > > > > >
> > > > > > > > > There is a very simple method though.  We allow devices
> > > > > > > > > to expose a subset of features when DMA is not used. So
> > > > > > > > > drivers that want maximum features will always opt for
> > > > > > > > > DMA. We can also strongly recommend that all drivers support
> DMA if available.
> > > > > > > > Yeah, don't see how this is elegant at all with all mixed bits.
> > > > > > >
> > > > > > > It's elegant because simple low end devices can cheaply
> > > > > > > implement MMIO and not worry about DMA.
> > > > > > >
> > > > > > It is not of much help in this case because any low end cheap
> > > > > > device which
> > > > > want to support flow filter commands need to have CVQ anyway.
> > > > > > And hence reusing the same CVQ is more elegant that already
> > > > > > does the
> > > DMA.
> > > > > >
> > > > > > So CVQ is fulfilling all the below needs.
> > > > > > 1. Single interface for the get/set config flow filters 2. DMA
> > > > > > the data 3. Not have any partial issues
> > > > >
> > > > > I don't know what these are.
> > > > I mean partial writes for fields.
> > >
> > > We don't allow these and in any case, writeble fields are best avoided.
> > >
> > Right. This is why flow filter objects are created using cvq, and its associated
> caps also comes via same cvq channel.
> >
> > >
> > > > >
> > > > > > 4. provides consistent structures that provisioning side will
> > > > > > be able to use
> > > > >
> > > > > Problem for provisioning is extra definitions will be needed, in
> > > > > a device specific way.
> > > > In vdpa tool and other OS tools of iproute2 developed, setting and
> > > > getting
> > > those device specific values are useful.
> > > > It is ok.
> > >
> > > It does not become ok just by saying so. You are taking a single RO
> > > value and instead of it having an address there are now 2 other ways
> > > to address it. And you fail to see the problem and the pain you are
> > > inflicting on software developers. Just stick with an address if you can.
> > There is zero problem with sw.
> > Sw just need to issue send_command() and done with it, like rest of the
> commands.
> > A pain would be create yet another DMA interface.
> 
> No because it will be a generic thing for all types.
> 
That does not make pain of one device go away.

> 
> > >
> > > > >
> > > > > > > > Nor do I see any enforcement, single method via cvq still holds
> strong.
> > > > > > >
> > > > > > > You don't need to enforce things, if people want to put a
> > > > > > > lot of RAM on device and put it in a register let them.
> > > > > > >
> > > > > > Not enforced. It uses the CVQ for flow group and flow filter
> > > > > > life cycles and for
> > > > > the sharing this config as well.
> > > > > > Also aligns with stats that rest also agreed on.
> > > > >
> > > > > I am talking about your attempt to generally say "no more config
> > > > > fields everything must be in CVQ".
> > > > Config fields for initialization time is fine as the spec allows it today.
> > > > Things which can differ, it is ok to use cvq interface.
> > >
> > > I don't know what does "Things that can differ" means. Generally
> > > device caps are perfect for config space. Accessed at init time only, RO.
> > >
> > You ignore the comment I answered before that proposal here is not based
> on RO/RW.
> > It is based on initialization time vs run time.
> 
> initialization time is really probe. That thing which only happens once when
> driver is setup. Clearly there is no reason to check capabilities many times so it
> will be done during setup, once.
> 
Capabilities will be queried once the valid id ranges are learnt by the driver during driver initialization time.

> 
> > > > > I think it's wrong definitiely for non network devices must
> > > > > sometimes for network too and generally we need a solution for
> > > > > config over DMA. This specific thing - whether it fits in CVQ is
> > > > > a separate discussion.
> > > > >
> > > > I explained it before, that 6 out of 19 devices has cvq which are
> > > > complex
> > > enough doing things over cvq.
> > > > These are non-network devices already.
> > > >
> > > > If one of those remaining device becomes complex, it is likely it
> > > > will need a
> > > cvq to suffice for the dma interface and it can just do with depth = 1.
> > >
> > > Using generic caps and not net specific ones is a good idea.
> > >
> > context here is cvq and net.
> 
> yes, you can use cvq command for this.  this will only serve net.  I am saying a
> generic interface for config over DMA is better and will serve everyone.
> 
6 out of 19 device do not need them.
And when those remaining devices needs complexity, they will likely need more than just this.

> We are doing this commonly, and when we did not push for a generic interface
> like with e.g. s/g limits for blk we later regret it.
> 

> 
> > >
> > >
> > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > The method proposed here is elegant and clearly
> > > > > > > > > > promote one way to do
> > > > > > > > > things for driver and device with predictability.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > I don't see it as elegant at all. What is elegant is *a
> > > > > > > > > single
> > > > > > > > > tag* that describes each property of the device. And
> > > > > > > > > this single tag should be
> > > > > > > good for everything:
> > > > > > > > > driver, provisioning, migration. And config space offset
> > > > > > > > > serves as
> > > such.
> > > > > > > > The single tag is the set of structures.
> > > > > > >
> > > > > > > I have no idea how this will work. If migration format i
> > > > > > > started reviewing is anything to go by then there will be a
> > > > > > > huge elaborate structure nothing single or simple. By
> > > > > > > comparison there's already a proposal how provisioning can
> > > > > > > work by supplying
> > > config space.
> > > > > > > it is just a clean model to grasp.
> > > > > > >
> > > > > > The provisioning model is simple is to supply all the configuration.
> > > > > > To draw parallels to some sw side,
> > > > > >
> > > > > > There is per functionality socket option to set things,
> > > > > > instead of one giant
> > > > > structure.
> > > > > > There is per functionality ethtool option/cmd instead of Set
> > > > > > ALL/get ALL
> > > > > enforcement.
> > > > >
> > > > > I'm not sure how much of a parallel one can draw.
> > > > > Do not see a lot of similarity.
> > > > For lot of configuration they are similar that happens at slow pace.
> > > >
> > > > > Devices commonly use register map. Everyone understands this
> paradigm.
> > > > >
> > > > For initialization early device setup time, yes.
> > > >
> > > > >
> > > > > I am not altogether happy with the way you are making migration
> > > > > generate duplicate definitions for lots of things we already
> > > > > have definitions
> > > for.
> > > > > Having a 3rd one for provisioning? Gimme a break.
> > > >
> > > > For migration, we are not duplicating. Some structures are not
> > > > well defined,
> > > it has some duplication.
> > >
> > > And fyi it's already making people unhappy.
> > >
> > Those exceptions are not the interesting one to take as example here.
> >
> > > > But large part seems be able to utilized pre-defined structs.
> > > > And here for flow filter also same structs will be used.
> > >
> > > So if there's a 64 bit bitmap in config space, then provisioning
> > > command which already gets config space can just use its offset.
> > > Simpler, better.
> > >
> > It is not simple to implement per device unique config space as we discussed
> already.
> 
> we did it for many years and many device types.
> 99% of devices do config over a register map of some kind.
> It is not as *flexible* true but flexibility here is an enemy, too easy to make a
> mess.
> 
> 
Don't see it this mess, otherwise it should apply to rest of the cvq commands.

> And your array of types where a single field would suffice is a classical case in
> point. config space keeps us disciplined.
> 
> > And no need another DMA interface either as cvq service that need already.
> 
> This is too focused on net and short term, for my taste.
> we need it so we
> 1. don't need to have this discussion every time 2. can begin to save on
> registers for config space in X years and
>    for new device types
> 
> 
> And, while apparently focusing on short term people also somehow manage
> to make a point out of every bikeshed instead of the "let's just ship it"
> attitude that would be consistent. I find this baffling.

Not really. using CVQ for the dma interface seems to fit all the needs without defining new interface across all device types.


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Mon, Nov 27, 2023 at 12:58:47PM +0000, Parav Pandit wrote:
> 
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, November 27, 2023 6:22 PM
> > 
> > On Mon, Nov 27, 2023 at 11:12:47AM +0000, Parav Pandit wrote:
> > >
> > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > Sent: Monday, November 27, 2023 4:40 PM
> > > >
> > > > On Mon, Nov 27, 2023 at 10:19:45AM +0000, Parav Pandit wrote:
> > > > > >
> > > > > > > The bright line in based on this usage: bootstap or not.
> > > > > > > If its runtime, sure have it over CVQ.
> > > > > > > Following the nice tenet of B.2 of the spec snippet: "Device
> > > > > > > configuration
> > > > > > space should only be used for initialization-time parameters"
> > > > > > > That’s it.
> > > > > > > And everyone is already aligned to it.
> > > > > >
> > > > > > Absolutely. Specifically, when do you expect driver to probe
> > > > > > these caps? As you yourself explained, it has to do it before
> > > > > > ethtool calls - that clearly means it will do it in probe.
> > > > > > To me this simply screams "initialization time".
> > > > > >
> > > > > It can be done in driver probe routine after DRIVER_OK, this is fine.
> > > >
> > > > No - it is preferable to do it before DRIVER_OK because VQs are
> > > > probed and initialized before DRIVER_OK.
> > > > For example, it is reasonable to decide to only enable MQ if
> > > > specific flow control can later be enabled.
> > >
> > > For flow filter related caps has no hard dependency to it before driver_ok,
> > same as stats.
> > > MQ has users even without flow filter.
> > 
> > driver ok is not the end of driver initialization. Things like pre-populating
> > receive queues are also part of that.
> 
> Rq populating is unrelated here.

it is related because you need to know which rqs to populate and
for this knowning how traffic is shaped, matters.

> > 
> > Yes, there is a dependency.  The capability is either not useful at all (as in the
> > current form - driver makes no decisions here so it is pointless). 
> It is making decisions based on group and flow id ranges.
> As I acked few times, that normative lines are missing and I am in middle of adding them for v7.
> 
> > Or it will be
> > useful and then it has to be probed at init time so the data is available at
> > runtime when decisions are to be made. How *hard* is this dependency?
> It is wrong for me to put this cap in the config space which is not needed for the device initialization.
> 
> > I don't know. I do know that 100% of drivers will either not use this command
> > at all or use it at init time.
> 
> I don’t see without this how driver can work as it does not know the valid id and group ranges to use.

> Driver can use this command at init time before registering the device to the OS.

Re-read what you just wrote and contrast with "is not needed for the
device initialization".

Do I need to spell how you are contradicting yourself here?

Do you take "device initialization" to mean some hardware thing?
This is not what it means. It means whatever driver does once,
before it starts using the device.
-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

RE: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Cornelia Huck]

>> > > > 4. provides consistent structures that provisioning side will be
>> > > > able to use
>> > >
>> > > Problem for provisioning is extra definitions will be needed, in a
>> > > device specific way.
>> > In vdpa tool and other OS tools of iproute2 developed, setting and getting
>> those device specific values are useful.
>> > It is ok.
>> 
>> It does not become ok just by saying so. You are taking a single RO value and
>> instead of it having an address there are now 2 other ways to address it. And
>> you fail to see the problem and the pain you are inflicting on software
>> developers. Just stick with an address if you can.
> There is zero problem with sw.
> Sw just need to issue send_command() and done with it, like rest of the commands.
> A pain would be create yet another DMA interface.
>
>> 
>> > >
>> > > > > > Nor do I see any enforcement, single method via cvq still holds strong.
>> > > > >
>> > > > > You don't need to enforce things, if people want to put a lot of
>> > > > > RAM on device and put it in a register let them.
>> > > > >
>> > > > Not enforced. It uses the CVQ for flow group and flow filter life
>> > > > cycles and for
>> > > the sharing this config as well.
>> > > > Also aligns with stats that rest also agreed on.
>> > >
>> > > I am talking about your attempt to generally say "no more config
>> > > fields everything must be in CVQ".
>> > Config fields for initialization time is fine as the spec allows it today.
>> > Things which can differ, it is ok to use cvq interface.
>> 
>> I don't know what does "Things that can differ" means. Generally device caps
>> are perfect for config space. Accessed at init time only, RO.
>> 
> You ignore the comment I answered before that proposal here is not based on RO/RW.
> It is based on initialization time vs run time.
>
>> > > I think it's wrong definitiely for non network devices must
>> > > sometimes for network too and generally we need a solution for
>> > > config over DMA. This specific thing - whether it fits in CVQ is a
>> > > separate discussion.
>> > >
>> > I explained it before, that 6 out of 19 devices has cvq which are complex
>> enough doing things over cvq.
>> > These are non-network devices already.
>> >
>> > If one of those remaining device becomes complex, it is likely it will need a
>> cvq to suffice for the dma interface and it can just do with depth = 1.
>> 
>> Using generic caps and not net specific ones is a good idea.
>> 
> context here is cvq and net.
>
>> 
>> 
>> > >
>> > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > > The method proposed here is elegant and clearly promote
>> > > > > > > > one way to do
>> > > > > > > things for driver and device with predictability.
>> > > > > > > >
>> > > > > > >
>> > > > > > > I don't see it as elegant at all. What is elegant is *a
>> > > > > > > single
>> > > > > > > tag* that describes each property of the device. And this
>> > > > > > > single tag should be
>> > > > > good for everything:
>> > > > > > > driver, provisioning, migration. And config space offset serves as
>> such.
>> > > > > > The single tag is the set of structures.
>> > > > >
>> > > > > I have no idea how this will work. If migration format i started
>> > > > > reviewing is anything to go by then there will be a huge
>> > > > > elaborate structure nothing single or simple. By comparison
>> > > > > there's already a proposal how provisioning can work by supplying
>> config space.
>> > > > > it is just a clean model to grasp.
>> > > > >
>> > > > The provisioning model is simple is to supply all the configuration.
>> > > > To draw parallels to some sw side,
>> > > >
>> > > > There is per functionality socket option to set things, instead of
>> > > > one giant
>> > > structure.
>> > > > There is per functionality ethtool option/cmd instead of Set
>> > > > ALL/get ALL
>> > > enforcement.
>> > >
>> > > I'm not sure how much of a parallel one can draw.
>> > > Do not see a lot of similarity.
>> > For lot of configuration they are similar that happens at slow pace.
>> >
>> > > Devices commonly use register map. Everyone understands this paradigm.
>> > >
>> > For initialization early device setup time, yes.
>> >
>> > >
>> > > I am not altogether happy with the way you are making migration
>> > > generate duplicate definitions for lots of things we already have definitions
>> for.
>> > > Having a 3rd one for provisioning? Gimme a break.
>> >
>> > For migration, we are not duplicating. Some structures are not well defined,
>> it has some duplication.
>> 
>> And fyi it's already making people unhappy.
>> 
> Those exceptions are not the interesting one to take as example here.
>
>> > But large part seems be able to utilized pre-defined structs.
>> > And here for flow filter also same structs will be used.
>> 
>> So if there's a 64 bit bitmap in config space, then provisioning command which
>> already gets config space can just use its offset.
>> Simpler, better.
>> 
> It is not simple to implement per device unique config space as we discussed already.
> And no need another DMA interface either as cvq service that need already.

A general injection from me.

We're seeing more and more of those monster threads on the list, where
everything is going in circles, sometimes spawning new versions, which
create their own monster threads... I really dread looking at
virtio-comment nowadays. This is drowning out other things on the list,
which have a tendency to just get lost in the noise.

We should stop and think, figure out why that happens, and how we can
get back to a productive environment.

From what I've seen in this discussion here (and it seems to mirror
other discussions I've browsed), it seems to boil down to narrow
viewpoints and problematic ways of discussing the design.

Is PCI important? Of course. Is virtio-net important? Nobody disputes
that. Are hardware implementations something we want to support well?
Sure. But that does not mean other use cases are not important as
well. Having a particular use case in mind is completely fine, treating
other use cases as second-class citizens is not.

What has also stuck out to me in many of those discussions is something
I'll call "argument by assertion", i.e. statements that something is
this-and-that, without qualifiers like "I think that..." or "we've seen
in other instances that...". From what I've seen, this tends to make
people on the other side of the argument defensive instead of
considering the arguments -- it certainly does not seem to foster a
productive discussion, or at least that's my impression. I've also seen
a tendency to reduce arguments to absolutes, which tends to bring out
defensiveness as well.

Last but not least, I don't think rapid posting is helping -- it rather
just seems to speed up ping-pong posting with each side becoming
entrenched in their point of view even more. It's certainly not
pleasurable to try to follow these threads. Can we try to calm down a
bit, please?


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Mon, Nov 27, 2023 at 01:05:15PM +0000, Parav Pandit wrote:
> 
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, November 27, 2023 6:13 PM
> > 
> > On Mon, Nov 27, 2023 at 11:54:52AM +0000, Parav Pandit wrote:
> > >
> > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > Sent: Monday, November 27, 2023 5:17 PM
> > > >
> > > > On Mon, Nov 27, 2023 at 11:40:25AM +0000, Parav Pandit wrote:
> > > > >
> > > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > > Sent: Monday, November 27, 2023 4:58 PM
> > > > > >
> > > > > > On Mon, Nov 27, 2023 at 10:19:49AM +0000, Parav Pandit wrote:
> > > > > > >
> > > > > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > > > > Sent: Friday, November 24, 2023 5:04 PM
> > > > > > > >
> > > > > > > > On Fri, Nov 24, 2023 at 06:31:03AM +0000, Parav Pandit wrote:
> > > > > > > > >
> > > > > > > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > > > > > > Sent: Friday, November 24, 2023 11:48 AM
> > > > > > > > > >
> > > > > > > > > > On Fri, Nov 24, 2023 at 05:40:26AM +0000, Parav Pandit wrote:
> > > > > > > > > > > > we
> > > > > > > > > > > > strongly suggest that *drivers* support both old and
> > > > > > > > > > > > new mechanism, and then *devices* will only
> > > > > > > > > > > > implement what's
> > > > > > required.
> > > > > > > > > > > There are other examples in the same document that
> > > > > > > > > > > makes things worst
> > > > > > > > > > with old and new.
> > > > > > > > > > >
> > > > > > > > > > > Also there is literally no way to enforce that driver
> > > > > > > > > > > supports both and new. It is just sounds like an
> > > > > > > > > > > excuse to force infinite config space.
> > > > > > > > > >
> > > > > > > > > > There is a very simple method though.  We allow devices
> > > > > > > > > > to expose a subset of features when DMA is not used. So
> > > > > > > > > > drivers that want maximum features will always opt for
> > > > > > > > > > DMA. We can also strongly recommend that all drivers support
> > DMA if available.
> > > > > > > > > Yeah, don't see how this is elegant at all with all mixed bits.
> > > > > > > >
> > > > > > > > It's elegant because simple low end devices can cheaply
> > > > > > > > implement MMIO and not worry about DMA.
> > > > > > > >
> > > > > > > It is not of much help in this case because any low end cheap
> > > > > > > device which
> > > > > > want to support flow filter commands need to have CVQ anyway.
> > > > > > > And hence reusing the same CVQ is more elegant that already
> > > > > > > does the
> > > > DMA.
> > > > > > >
> > > > > > > So CVQ is fulfilling all the below needs.
> > > > > > > 1. Single interface for the get/set config flow filters 2. DMA
> > > > > > > the data 3. Not have any partial issues
> > > > > >
> > > > > > I don't know what these are.
> > > > > I mean partial writes for fields.
> > > >
> > > > We don't allow these and in any case, writeble fields are best avoided.
> > > >
> > > Right. This is why flow filter objects are created using cvq, and its associated
> > caps also comes via same cvq channel.
> > >
> > > >
> > > > > >
> > > > > > > 4. provides consistent structures that provisioning side will
> > > > > > > be able to use
> > > > > >
> > > > > > Problem for provisioning is extra definitions will be needed, in
> > > > > > a device specific way.
> > > > > In vdpa tool and other OS tools of iproute2 developed, setting and
> > > > > getting
> > > > those device specific values are useful.
> > > > > It is ok.
> > > >
> > > > It does not become ok just by saying so. You are taking a single RO
> > > > value and instead of it having an address there are now 2 other ways
> > > > to address it. And you fail to see the problem and the pain you are
> > > > inflicting on software developers. Just stick with an address if you can.
> > > There is zero problem with sw.
> > > Sw just need to issue send_command() and done with it, like rest of the
> > commands.
> > > A pain would be create yet another DMA interface.
> > 
> > No because it will be a generic thing for all types.
> > 
> That does not make pain of one device go away.

This gateway will be needed sooner or later.

Or devices can also build a register map. All this is really just so you
can stop beating the "infinite registers!" drum.
I suspect after looking at it, in the end practically hardware designers
will go "oh it's 0.002$ per device let's just keep it simple".
But yea, let's add an option.


> > 
> > > >
> > > > > >
> > > > > > > > > Nor do I see any enforcement, single method via cvq still holds
> > strong.
> > > > > > > >
> > > > > > > > You don't need to enforce things, if people want to put a
> > > > > > > > lot of RAM on device and put it in a register let them.
> > > > > > > >
> > > > > > > Not enforced. It uses the CVQ for flow group and flow filter
> > > > > > > life cycles and for
> > > > > > the sharing this config as well.
> > > > > > > Also aligns with stats that rest also agreed on.
> > > > > >
> > > > > > I am talking about your attempt to generally say "no more config
> > > > > > fields everything must be in CVQ".
> > > > > Config fields for initialization time is fine as the spec allows it today.
> > > > > Things which can differ, it is ok to use cvq interface.
> > > >
> > > > I don't know what does "Things that can differ" means. Generally
> > > > device caps are perfect for config space. Accessed at init time only, RO.
> > > >
> > > You ignore the comment I answered before that proposal here is not based
> > on RO/RW.
> > > It is based on initialization time vs run time.
> > 
> > initialization time is really probe. That thing which only happens once when
> > driver is setup. Clearly there is no reason to check capabilities many times so it
> > will be done during setup, once.
> > 
> Capabilities will be queried once the valid id ranges are learnt by the driver during driver initialization time.
>

right. And being initialization time, it belongs in config space.
 
> > 
> > > > > > I think it's wrong definitiely for non network devices must
> > > > > > sometimes for network too and generally we need a solution for
> > > > > > config over DMA. This specific thing - whether it fits in CVQ is
> > > > > > a separate discussion.
> > > > > >
> > > > > I explained it before, that 6 out of 19 devices has cvq which are
> > > > > complex
> > > > enough doing things over cvq.
> > > > > These are non-network devices already.
> > > > >
> > > > > If one of those remaining device becomes complex, it is likely it
> > > > > will need a
> > > > cvq to suffice for the dma interface and it can just do with depth = 1.
> > > >
> > > > Using generic caps and not net specific ones is a good idea.
> > > >
> > > context here is cvq and net.
> > 
> > yes, you can use cvq command for this.  this will only serve net.  I am saying a
> > generic interface for config over DMA is better and will serve everyone.
> > 
> 6 out of 19 device do not need them.
> And when those remaining devices needs complexity, they will likely need more than just this.
> 
> > We are doing this commonly, and when we did not push for a generic interface
> > like with e.g. s/g limits for blk we later regret it.
> > 
> 
> > 
> > > >
> > > >
> > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > The method proposed here is elegant and clearly
> > > > > > > > > > > promote one way to do
> > > > > > > > > > things for driver and device with predictability.
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I don't see it as elegant at all. What is elegant is *a
> > > > > > > > > > single
> > > > > > > > > > tag* that describes each property of the device. And
> > > > > > > > > > this single tag should be
> > > > > > > > good for everything:
> > > > > > > > > > driver, provisioning, migration. And config space offset
> > > > > > > > > > serves as
> > > > such.
> > > > > > > > > The single tag is the set of structures.
> > > > > > > >
> > > > > > > > I have no idea how this will work. If migration format i
> > > > > > > > started reviewing is anything to go by then there will be a
> > > > > > > > huge elaborate structure nothing single or simple. By
> > > > > > > > comparison there's already a proposal how provisioning can
> > > > > > > > work by supplying
> > > > config space.
> > > > > > > > it is just a clean model to grasp.
> > > > > > > >
> > > > > > > The provisioning model is simple is to supply all the configuration.
> > > > > > > To draw parallels to some sw side,
> > > > > > >
> > > > > > > There is per functionality socket option to set things,
> > > > > > > instead of one giant
> > > > > > structure.
> > > > > > > There is per functionality ethtool option/cmd instead of Set
> > > > > > > ALL/get ALL
> > > > > > enforcement.
> > > > > >
> > > > > > I'm not sure how much of a parallel one can draw.
> > > > > > Do not see a lot of similarity.
> > > > > For lot of configuration they are similar that happens at slow pace.
> > > > >
> > > > > > Devices commonly use register map. Everyone understands this
> > paradigm.
> > > > > >
> > > > > For initialization early device setup time, yes.
> > > > >
> > > > > >
> > > > > > I am not altogether happy with the way you are making migration
> > > > > > generate duplicate definitions for lots of things we already
> > > > > > have definitions
> > > > for.
> > > > > > Having a 3rd one for provisioning? Gimme a break.
> > > > >
> > > > > For migration, we are not duplicating. Some structures are not
> > > > > well defined,
> > > > it has some duplication.
> > > >
> > > > And fyi it's already making people unhappy.
> > > >
> > > Those exceptions are not the interesting one to take as example here.
> > >
> > > > > But large part seems be able to utilized pre-defined structs.
> > > > > And here for flow filter also same structs will be used.
> > > >
> > > > So if there's a 64 bit bitmap in config space, then provisioning
> > > > command which already gets config space can just use its offset.
> > > > Simpler, better.
> > > >
> > > It is not simple to implement per device unique config space as we discussed
> > already.
> > 
> > we did it for many years and many device types.
> > 99% of devices do config over a register map of some kind.
> > It is not as *flexible* true but flexibility here is an enemy, too easy to make a
> > mess.
> > 
> > 
> Don't see it this mess, otherwise it should apply to rest of the cvq commands.

yes, they are kind of messy, at that.  we don't use cvq by choice. We
use it because for things that need to keep changing at runtime config
space turned out to be a bad choice. But this query, is not.


> > And your array of types where a single field would suffice is a classical case in
> > point. config space keeps us disciplined.
> > 
> > > And no need another DMA interface either as cvq service that need already.
> > 
> > This is too focused on net and short term, for my taste.
> > we need it so we
> > 1. don't need to have this discussion every time 2. can begin to save on
> > registers for config space in X years and
> >    for new device types
> > 
> > 
> > And, while apparently focusing on short term people also somehow manage
> > to make a point out of every bikeshed instead of the "let's just ship it"
> > attitude that would be consistent. I find this baffling.
> 
> Not really. using CVQ for the dma interface seems to fit all the needs without defining new interface across all device types.

Of course, because you just ignore all other needs declaring them out of
scope. Really not merging patches is the only tool we have to make
people work on generic infra, otherwise each vendor only wants to work
on device specific things.

-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Mon, Nov 27, 2023 at 02:06:27PM +0100, Cornelia Huck wrote:
> >> > > > 4. provides consistent structures that provisioning side will be
> >> > > > able to use
> >> > >
> >> > > Problem for provisioning is extra definitions will be needed, in a
> >> > > device specific way.
> >> > In vdpa tool and other OS tools of iproute2 developed, setting and getting
> >> those device specific values are useful.
> >> > It is ok.
> >> 
> >> It does not become ok just by saying so. You are taking a single RO value and
> >> instead of it having an address there are now 2 other ways to address it. And
> >> you fail to see the problem and the pain you are inflicting on software
> >> developers. Just stick with an address if you can.
> > There is zero problem with sw.
> > Sw just need to issue send_command() and done with it, like rest of the commands.
> > A pain would be create yet another DMA interface.
> >
> >> 
> >> > >
> >> > > > > > Nor do I see any enforcement, single method via cvq still holds strong.
> >> > > > >
> >> > > > > You don't need to enforce things, if people want to put a lot of
> >> > > > > RAM on device and put it in a register let them.
> >> > > > >
> >> > > > Not enforced. It uses the CVQ for flow group and flow filter life
> >> > > > cycles and for
> >> > > the sharing this config as well.
> >> > > > Also aligns with stats that rest also agreed on.
> >> > >
> >> > > I am talking about your attempt to generally say "no more config
> >> > > fields everything must be in CVQ".
> >> > Config fields for initialization time is fine as the spec allows it today.
> >> > Things which can differ, it is ok to use cvq interface.
> >> 
> >> I don't know what does "Things that can differ" means. Generally device caps
> >> are perfect for config space. Accessed at init time only, RO.
> >> 
> > You ignore the comment I answered before that proposal here is not based on RO/RW.
> > It is based on initialization time vs run time.
> >
> >> > > I think it's wrong definitiely for non network devices must
> >> > > sometimes for network too and generally we need a solution for
> >> > > config over DMA. This specific thing - whether it fits in CVQ is a
> >> > > separate discussion.
> >> > >
> >> > I explained it before, that 6 out of 19 devices has cvq which are complex
> >> enough doing things over cvq.
> >> > These are non-network devices already.
> >> >
> >> > If one of those remaining device becomes complex, it is likely it will need a
> >> cvq to suffice for the dma interface and it can just do with depth = 1.
> >> 
> >> Using generic caps and not net specific ones is a good idea.
> >> 
> > context here is cvq and net.
> >
> >> 
> >> 
> >> > >
> >> > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > > The method proposed here is elegant and clearly promote
> >> > > > > > > > one way to do
> >> > > > > > > things for driver and device with predictability.
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > > I don't see it as elegant at all. What is elegant is *a
> >> > > > > > > single
> >> > > > > > > tag* that describes each property of the device. And this
> >> > > > > > > single tag should be
> >> > > > > good for everything:
> >> > > > > > > driver, provisioning, migration. And config space offset serves as
> >> such.
> >> > > > > > The single tag is the set of structures.
> >> > > > >
> >> > > > > I have no idea how this will work. If migration format i started
> >> > > > > reviewing is anything to go by then there will be a huge
> >> > > > > elaborate structure nothing single or simple. By comparison
> >> > > > > there's already a proposal how provisioning can work by supplying
> >> config space.
> >> > > > > it is just a clean model to grasp.
> >> > > > >
> >> > > > The provisioning model is simple is to supply all the configuration.
> >> > > > To draw parallels to some sw side,
> >> > > >
> >> > > > There is per functionality socket option to set things, instead of
> >> > > > one giant
> >> > > structure.
> >> > > > There is per functionality ethtool option/cmd instead of Set
> >> > > > ALL/get ALL
> >> > > enforcement.
> >> > >
> >> > > I'm not sure how much of a parallel one can draw.
> >> > > Do not see a lot of similarity.
> >> > For lot of configuration they are similar that happens at slow pace.
> >> >
> >> > > Devices commonly use register map. Everyone understands this paradigm.
> >> > >
> >> > For initialization early device setup time, yes.
> >> >
> >> > >
> >> > > I am not altogether happy with the way you are making migration
> >> > > generate duplicate definitions for lots of things we already have definitions
> >> for.
> >> > > Having a 3rd one for provisioning? Gimme a break.
> >> >
> >> > For migration, we are not duplicating. Some structures are not well defined,
> >> it has some duplication.
> >> 
> >> And fyi it's already making people unhappy.
> >> 
> > Those exceptions are not the interesting one to take as example here.
> >
> >> > But large part seems be able to utilized pre-defined structs.
> >> > And here for flow filter also same structs will be used.
> >> 
> >> So if there's a 64 bit bitmap in config space, then provisioning command which
> >> already gets config space can just use its offset.
> >> Simpler, better.
> >> 
> > It is not simple to implement per device unique config space as we discussed already.
> > And no need another DMA interface either as cvq service that need already.
> 
> A general injection from me.
> 
> We're seeing more and more of those monster threads on the list, where
> everything is going in circles, sometimes spawning new versions, which
> create their own monster threads... I really dread looking at
> virtio-comment nowadays. This is drowning out other things on the list,
> which have a tendency to just get lost in the noise.
> 
> We should stop and think, figure out why that happens, and how we can
> get back to a productive environment.
> 
> >From what I've seen in this discussion here (and it seems to mirror
> other discussions I've browsed), it seems to boil down to narrow
> viewpoints and problematic ways of discussing the design.
> 
> Is PCI important? Of course. Is virtio-net important? Nobody disputes
> that. Are hardware implementations something we want to support well?
> Sure. But that does not mean other use cases are not important as
> well. Having a particular use case in mind is completely fine, treating
> other use cases as second-class citizens is not.
> 
> What has also stuck out to me in many of those discussions is something
> I'll call "argument by assertion", i.e. statements that something is
> this-and-that, without qualifiers like "I think that..." or "we've seen
> in other instances that...". From what I've seen, this tends to make
> people on the other side of the argument defensive instead of
> considering the arguments -- it certainly does not seem to foster a
> productive discussion, or at least that's my impression. I've also seen
> a tendency to reduce arguments to absolutes, which tends to bring out
> defensiveness as well.
> 
> Last but not least, I don't think rapid posting is helping -- it rather
> just seems to speed up ping-pong posting with each side becoming
> entrenched in their point of view even more. It's certainly not
> pleasurable to try to follow these threads. Can we try to calm down a
> bit, please?


Thanks for bringing this up Cornelia, I agree and think I've been
guilty of being too assertive and posting too fast recently, too.
I'll try to address that.

Thanks,
-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Michael S. Tsirkin]

On Fri, Nov 24, 2023 at 12:02:23PM +0800, Jason Wang wrote:
> On Wed, Nov 22, 2023 at 10:51 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Wed, Nov 22, 2023 at 02:10:29PM +0000, Parav Pandit wrote:
> > >
> > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > Sent: Wednesday, November 22, 2023 7:32 PM
> > > >
> > > > On Fri, Nov 10, 2023 at 02:38:50PM +0200, Parav Pandit wrote:
> > > > > The device responds flow filter capabilities using two commands.
> > > > > One command indicates generic flow filter device limits such as number
> > > > > of flow filters, number of flow filter groups, support or multiple
> > > > > transports etc.
> > > > >
> > > > > The second command indicates supported match types, and fields of the
> > > > > packet.
> > > > >
> > > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
> > > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > > > Signed-off-by: Parav Pandit <parav@nvidia.com>
> > > > >
> > > > > ---
> > > > > changelog:
> > > > > v2->v3:
> > > > > - rebased on virtio-1.4 branch
> > > > > - removed reference for flow filter virtqueue
> > > > > v1->v2:
> > > > > - addressed comments from Satananda
> > > > > - added vlan type match field
> > > > > - kept space for types between l2, l3, l4 header match types
> > > > > - renamed mask to mask_supported with shorter width
> > > > > - made more fields reserved for furture
> > > > > - addressed comments from Heng
> > > > > - grammar correction
> > > > > - added field to indicate supported number of actions per flow
> > > > >   filter match entry
> > > > > - added missing documentation for max_flow_priorities_per_group
> > > > > v0->v1:
> > > > > - added mask field in the type to indicate supported mask by device
> > > > >   and also in later patch to use it to indicate mask on adding
> > > > >   flow filter. As a result removed the mask_supported capability
> > > > >   field
> > > > > ---
> > > > >  device-types/net/description.tex | 208
> > > > > ++++++++++++++++++++++++++++++-
> > > > >  1 file changed, 206 insertions(+), 2 deletions(-)
> > > > >
> > > > > diff --git a/device-types/net/description.tex
> > > > > b/device-types/net/description.tex
> > > > > index 30220b5..eccd8d6 100644
> > > > > --- a/device-types/net/description.tex
> > > > > +++ b/device-types/net/description.tex
> > > > > @@ -1173,7 +1173,11 @@ \subsubsection{Flow Filter}\label{sec:Device
> > > > > Types / Network Device / Device Ope
> > > > >
> > > > >  The device indicates the flow filter capabilities to the driver.
> > > > > These  capabilities include various maximum device limits and
> > > > > -supported packet match fields.
> > > > > +supported packet match fields. These control virtqueue commands are:
> > > > > +\ref{sec:Device Types / Network Device / Device Operation / Control
> > > > > +Virtqueue / Flow Filter / Flow Filter Capabilities Get} and
> > > > > +\ref{sec:Device Types / Network Device / Device Operation / Control
> > > > Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}.
> > > > >
> > > > >  The flow filters are grouped using a flow filter group. Each flow
> > > > > filter  group has a priority. The device first applies the flow
> > > > > filters of the highest @@ -1224,7 +1228,136 @@ \subsubsection{Flow
> > > > Filter}\label{sec:Device Types / Network Device / Device Ope
> > > > >        the flow filters in group_C, the flow filters of next level group_B are
> > > > applied.
> > > > >  \end{itemize}
> > > > >
> > > > > -\label{sec:Device Types / Network Device / Device Operation / Control
> > > > > Virtqueue / Setting Promiscuous Mode}%old label for latexdiff
> > > > > +\paragraph{Match Types and Fields}\label{sec:Device Types / Network
> > > > > +Device / Device Operation / Flow Filter / Match Types and Fields}
> > > > > +
> > > > > +\begin{lstlisting}
> > > > > +struct virtio_net_ff_match_type_cap {
> > > > > +        le16 type;
> > > > > +        u8 mask_supported;
> > > > > +        u8 reserved[5];
> > > > > +        le64 fields_bmap;
> > > > > +};
> > > > > +\end{lstlisting}
> > > > > +
> > > > > +The \field{type} corresponds to following table:
> > > > > +
> > > > > +\begin{tabular}{|l|l|l|}
> > > > > +\hline
> > > > > +Type & Name & Description \\
> > > > > +\hline \hline
> > > > > +0   & VIRTIO_NET_FF_ETH_HDR & Ethernet header of the packet \\
> > > > > +\hline
> > > > > +0x1   & VIRTIO_NET_FF_VLAN_TAG_HDR & VLAN tag of the packet \\
> > > > > +\hline
> > > > > +0x200   & VIRTIO_NET_FF_IPV4_HDR & IPv4 header of the packet \\
> > > > > +\hline
> > > > > +0x300   & VIRTIO_NET_FF_IPV6_HDR & IPv6 header of the packet \\
> > > > > +\hline
> > > > > +0x400   & VIRTIO_NET_FF_TCP_HDR & TCP header of the packet \\
> > > > > +\hline
> > > > > +0x500   & VIRTIO_NET_FF_UDP_HDR & UDP header of the packet \\
> > > > > +\hline
> > > > > +other   & -    & reserved \\
> > > > > +\hline
> > > > > +\end{tabular}
> > > > > +
> > > > > +When the \field{mask_supported} is set, for the specific
> > > > > +\field{type}, the device can perform masking packet fields with the
> > > > > +mask supplied in the flow filter match entry.
> > > > > +
> > > > > +For each \field{type} the \field{fields_bmap} indicates supported
> > > > > +fields of the packet header which can be matched.
> > > > > +
> > > > > +For the \field{type} of VIRTIO_NET_FF_ETH_HDR, header fields are
> > > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > > +
> > > > > +\begin{tabular}{|l|l|l|}
> > > > > +\hline
> > > > > +Bit & Name & Description \\
> > > > > +\hline \hline
> > > > > +0   & VIRTIO_NET_FF_DST_MAC & Destination MAC address in the packet \\
> > > > > +\hline
> > > > > +1   & VIRTIO_NET_FF_SRC_MAC & Source MAC address in the packet \\
> > > > > +\hline
> > > > > +2   & VIRTIO_NET_FF_ETHER_TYPE & Ether type in the packet \\
> > > > > +\hline
> > > > > +other   & -    & reserved \\
> > > > > +\hline
> > > > > +\end{tabular}
> > > > > +
> > > > > +For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN tag fields
> > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > +
> > > > > +\begin{tabular}{|l|l|l|}
> > > > > +\hline
> > > > > +Bit & Name & Description \\
> > > > > +\hline \hline
> > > > > +0   & VIRTIO_NET_FF_VLAN_TAG_TCI & Vlan tag TCI 16-bit field \\
> > > > > +\hline
> > > > > +other   & -    & reserved \\
> > > > > +\hline
> > > > > +\end{tabular}
> > > > > +
> > > > > +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, header fields are
> > > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > > +
> > > > > +\begin{tabular}{|l|l|l|}
> > > > > +\hline
> > > > > +Bit & Name & Description \\
> > > > > +\hline \hline
> > > > > +0   & VIRTIO_NET_FF_SRC_IPV4 & Source IPV4 address in the packet \\
> > > > > +\hline
> > > > > +1   & VIRTIO_NET_FF_DST_IPV4 & Destination IPV4 address in the packet \\
> > > > > +\hline
> > > > > +other   & -    & reserved \\
> > > > > +\hline
> > > > > +\end{tabular}
> > > > > +
> > > > > +For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header fields are
> > > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > > +
> > > > > +\begin{tabular}{|l|l|l|}
> > > > > +\hline
> > > > > +Bit & Name & Description \\
> > > > > +\hline \hline
> > > > > +0   & VIRTIO_NET_FF_SRC_IPV6 & Source IPV6 address in the packet \\
> > > > > +\hline
> > > > > +1   & VIRTIO_NET_FF_DST_IPV6 & Destination IPV6 address in the packet \\
> > > > > +\hline
> > > > > +other   & -    & reserved \\
> > > > > +\hline
> > > > > +\end{tabular}
> > > > > +
> > > > > +For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header fields are
> > > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > > +
> > > > > +\begin{tabular}{|l|l|l|}
> > > > > +\hline
> > > > > +Bit & Name & Description \\
> > > > > +\hline \hline
> > > > > +0   & VIRTIO_NET_FF_SRC_TCP_PORT & Source TCP port in the packet \\
> > > > > +\hline
> > > > > +1   & VIRTIO_NET_FF_DST_TCP_PORT & Destination TCP port in the packet
> > > > \\
> > > > > +\hline
> > > > > +other   & -    & reserved \\
> > > > > +\hline
> > > > > +\end{tabular}
> > > > > +
> > > > > +For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header fields are
> > > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > > +
> > > > > +\begin{tabular}{|l|l|l|}
> > > > > +\hline
> > > > > +Bit & Name & Description \\
> > > > > +\hline \hline
> > > > > +0   & VIRTIO_NET_FF_SRC_UDP_PORT & Source UDP port in the packet \\
> > > > > +\hline
> > > > > +1   & VIRTIO_NET_FF_DST_UDP_PORT & Destination UDP port in the packet
> > > > \\
> > > > > +\hline
> > > > > +other   & -    & reserved  \\
> > > > > +\hline
> > > > > +\end{tabular}
> > > > > +
> > > >
> > > > This is such an elaborate structure to report just 12 read only bits.
> > > > Please let's just follow the example of  le32 supported_tunnel_types and add
> > > > l32 supported_flow_control.
> > > >
> > > supported_tunnel_types was let go because cvq is efficient.
> > > None of these fields are needed for init time configuration of the driver before DRIVER_OK.
> >
> > I really basically disagree. Whether control flow is supported can
> > easily influence how many VQs are needed.
> >
> >
> > >
> > > It is a very narrow view of 12 bits. It is going to grow and many.
> > > This is far more structured for each type done here.
> > >
> > > >
> > > > After you were trying to add kilobytes to megabytes of memory - I see little
> > > > reason to save 12 RO bits that can be shared by any number of VFs.
> > > >
> > > Completely wrong reason and very late review and also does not align with every other command we did.
> >
> > which other command? hash and rss are like this: capability in config
> > space config through cvq. For the same reason.
> >
> > > > However, I do think we should create an option to access config space over
> > > > DMA (preferably admin commands). Let's add this quickly and then we don't
> > > > need to worry about legacy guests accessing flow filter through MMIO.
> > >
> > > CVQ is already there as single interface forget and set for rss, rss context, vq moderation, statistics, flow filter caps and more.
> > > No reason to bifurcate.
> >
> > The reason is so that we have a single consistent view where e.g. you want
> > to provision a device with a specific capability you just
> > specify how its config space looks.
> >
> > If you start shuffling capabilities off to VQs that will be much
> > much harder.
> >
> > > I won't be able to absorb this comment of DMA interface.
> > > If I discuss further, I will repeat the whole document [1] and I will avoid that now.
> > >
> > > [1] https://docs.google.com/document/d/1Iyn-l3Nm0yls3pZaul4lZiVj8x1s73Ed6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
> >
> >
> > I really worry about how provisioning will work. And I do not at all
> > cherish replicating all of these query capability commands for provisioning.
> 
> +1
> 
> There's nothing that prevents the config space from being implemented
> in a way other than registers.

Jason I'm not sure what does your +1 mean here. Are you ok with the
patchset as is? While I feel config space would be nicer I also find it
unlikely drivers will actually bother to check the supported commands.
So from that POV, for this use-case I am not too stressed out.

-- 
MST


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

Re: [virtio-comment] Re: [PATCH v6 2/5] virtio-net: Add flow filter capabilities read commands

[Jason Wang]

On Tue, Dec 12, 2023 at 11:52 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Fri, Nov 24, 2023 at 12:02:23PM +0800, Jason Wang wrote:
> > On Wed, Nov 22, 2023 at 10:51 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > On Wed, Nov 22, 2023 at 02:10:29PM +0000, Parav Pandit wrote:
> > > >
> > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > Sent: Wednesday, November 22, 2023 7:32 PM
> > > > >
> > > > > On Fri, Nov 10, 2023 at 02:38:50PM +0200, Parav Pandit wrote:
> > > > > > The device responds flow filter capabilities using two commands.
> > > > > > One command indicates generic flow filter device limits such as number
> > > > > > of flow filters, number of flow filter groups, support or multiple
> > > > > > transports etc.
> > > > > >
> > > > > > The second command indicates supported match types, and fields of the
> > > > > > packet.
> > > > > >
> > > > > > Fixes: https://github.com/oasis-tcs/virtio-spec/issues/179
> > > > > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > > > > Signed-off-by: Parav Pandit <parav@nvidia.com>
> > > > > >
> > > > > > ---
> > > > > > changelog:
> > > > > > v2->v3:
> > > > > > - rebased on virtio-1.4 branch
> > > > > > - removed reference for flow filter virtqueue
> > > > > > v1->v2:
> > > > > > - addressed comments from Satananda
> > > > > > - added vlan type match field
> > > > > > - kept space for types between l2, l3, l4 header match types
> > > > > > - renamed mask to mask_supported with shorter width
> > > > > > - made more fields reserved for furture
> > > > > > - addressed comments from Heng
> > > > > > - grammar correction
> > > > > > - added field to indicate supported number of actions per flow
> > > > > >   filter match entry
> > > > > > - added missing documentation for max_flow_priorities_per_group
> > > > > > v0->v1:
> > > > > > - added mask field in the type to indicate supported mask by device
> > > > > >   and also in later patch to use it to indicate mask on adding
> > > > > >   flow filter. As a result removed the mask_supported capability
> > > > > >   field
> > > > > > ---
> > > > > >  device-types/net/description.tex | 208
> > > > > > ++++++++++++++++++++++++++++++-
> > > > > >  1 file changed, 206 insertions(+), 2 deletions(-)
> > > > > >
> > > > > > diff --git a/device-types/net/description.tex
> > > > > > b/device-types/net/description.tex
> > > > > > index 30220b5..eccd8d6 100644
> > > > > > --- a/device-types/net/description.tex
> > > > > > +++ b/device-types/net/description.tex
> > > > > > @@ -1173,7 +1173,11 @@ \subsubsection{Flow Filter}\label{sec:Device
> > > > > > Types / Network Device / Device Ope
> > > > > >
> > > > > >  The device indicates the flow filter capabilities to the driver.
> > > > > > These  capabilities include various maximum device limits and
> > > > > > -supported packet match fields.
> > > > > > +supported packet match fields. These control virtqueue commands are:
> > > > > > +\ref{sec:Device Types / Network Device / Device Operation / Control
> > > > > > +Virtqueue / Flow Filter / Flow Filter Capabilities Get} and
> > > > > > +\ref{sec:Device Types / Network Device / Device Operation / Control
> > > > > Virtqueue / Flow Filter / Flow Filter Match Capabilities Get}.
> > > > > >
> > > > > >  The flow filters are grouped using a flow filter group. Each flow
> > > > > > filter  group has a priority. The device first applies the flow
> > > > > > filters of the highest @@ -1224,7 +1228,136 @@ \subsubsection{Flow
> > > > > Filter}\label{sec:Device Types / Network Device / Device Ope
> > > > > >        the flow filters in group_C, the flow filters of next level group_B are
> > > > > applied.
> > > > > >  \end{itemize}
> > > > > >
> > > > > > -\label{sec:Device Types / Network Device / Device Operation / Control
> > > > > > Virtqueue / Setting Promiscuous Mode}%old label for latexdiff
> > > > > > +\paragraph{Match Types and Fields}\label{sec:Device Types / Network
> > > > > > +Device / Device Operation / Flow Filter / Match Types and Fields}
> > > > > > +
> > > > > > +\begin{lstlisting}
> > > > > > +struct virtio_net_ff_match_type_cap {
> > > > > > +        le16 type;
> > > > > > +        u8 mask_supported;
> > > > > > +        u8 reserved[5];
> > > > > > +        le64 fields_bmap;
> > > > > > +};
> > > > > > +\end{lstlisting}
> > > > > > +
> > > > > > +The \field{type} corresponds to following table:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Type & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_ETH_HDR & Ethernet header of the packet \\
> > > > > > +\hline
> > > > > > +0x1   & VIRTIO_NET_FF_VLAN_TAG_HDR & VLAN tag of the packet \\
> > > > > > +\hline
> > > > > > +0x200   & VIRTIO_NET_FF_IPV4_HDR & IPv4 header of the packet \\
> > > > > > +\hline
> > > > > > +0x300   & VIRTIO_NET_FF_IPV6_HDR & IPv6 header of the packet \\
> > > > > > +\hline
> > > > > > +0x400   & VIRTIO_NET_FF_TCP_HDR & TCP header of the packet \\
> > > > > > +\hline
> > > > > > +0x500   & VIRTIO_NET_FF_UDP_HDR & UDP header of the packet \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +When the \field{mask_supported} is set, for the specific
> > > > > > +\field{type}, the device can perform masking packet fields with the
> > > > > > +mask supplied in the flow filter match entry.
> > > > > > +
> > > > > > +For each \field{type} the \field{fields_bmap} indicates supported
> > > > > > +fields of the packet header which can be matched.
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_ETH_HDR, header fields are
> > > > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_DST_MAC & Destination MAC address in the packet \\
> > > > > > +\hline
> > > > > > +1   & VIRTIO_NET_FF_SRC_MAC & Source MAC address in the packet \\
> > > > > > +\hline
> > > > > > +2   & VIRTIO_NET_FF_ETHER_TYPE & Ether type in the packet \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_VLAN_TAG_HDR, VLAN tag fields
> > > > > > +are represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_VLAN_TAG_TCI & Vlan tag TCI 16-bit field \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_IPV4_HDR, header fields are
> > > > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_SRC_IPV4 & Source IPV4 address in the packet \\
> > > > > > +\hline
> > > > > > +1   & VIRTIO_NET_FF_DST_IPV4 & Destination IPV4 address in the packet \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_IPV6_HDR, header fields are
> > > > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_SRC_IPV6 & Source IPV6 address in the packet \\
> > > > > > +\hline
> > > > > > +1   & VIRTIO_NET_FF_DST_IPV6 & Destination IPV6 address in the packet \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_TCP_HDR, header fields are
> > > > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_SRC_TCP_PORT & Source TCP port in the packet \\
> > > > > > +\hline
> > > > > > +1   & VIRTIO_NET_FF_DST_TCP_PORT & Destination TCP port in the packet
> > > > > \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > > > +For the \field{type} of VIRTIO_NET_FF_UDP_HDR, header fields are
> > > > > > +represented by a bitmap in \field{fields_bmap} are following:
> > > > > > +
> > > > > > +\begin{tabular}{|l|l|l|}
> > > > > > +\hline
> > > > > > +Bit & Name & Description \\
> > > > > > +\hline \hline
> > > > > > +0   & VIRTIO_NET_FF_SRC_UDP_PORT & Source UDP port in the packet \\
> > > > > > +\hline
> > > > > > +1   & VIRTIO_NET_FF_DST_UDP_PORT & Destination UDP port in the packet
> > > > > \\
> > > > > > +\hline
> > > > > > +other   & -    & reserved  \\
> > > > > > +\hline
> > > > > > +\end{tabular}
> > > > > > +
> > > > >
> > > > > This is such an elaborate structure to report just 12 read only bits.
> > > > > Please let's just follow the example of  le32 supported_tunnel_types and add
> > > > > l32 supported_flow_control.
> > > > >
> > > > supported_tunnel_types was let go because cvq is efficient.
> > > > None of these fields are needed for init time configuration of the driver before DRIVER_OK.
> > >
> > > I really basically disagree. Whether control flow is supported can
> > > easily influence how many VQs are needed.
> > >
> > >
> > > >
> > > > It is a very narrow view of 12 bits. It is going to grow and many.
> > > > This is far more structured for each type done here.
> > > >
> > > > >
> > > > > After you were trying to add kilobytes to megabytes of memory - I see little
> > > > > reason to save 12 RO bits that can be shared by any number of VFs.
> > > > >
> > > > Completely wrong reason and very late review and also does not align with every other command we did.
> > >
> > > which other command? hash and rss are like this: capability in config
> > > space config through cvq. For the same reason.
> > >
> > > > > However, I do think we should create an option to access config space over
> > > > > DMA (preferably admin commands). Let's add this quickly and then we don't
> > > > > need to worry about legacy guests accessing flow filter through MMIO.
> > > >
> > > > CVQ is already there as single interface forget and set for rss, rss context, vq moderation, statistics, flow filter caps and more.
> > > > No reason to bifurcate.
> > >
> > > The reason is so that we have a single consistent view where e.g. you want
> > > to provision a device with a specific capability you just
> > > specify how its config space looks.
> > >
> > > If you start shuffling capabilities off to VQs that will be much
> > > much harder.
> > >
> > > > I won't be able to absorb this comment of DMA interface.
> > > > If I discuss further, I will repeat the whole document [1] and I will avoid that now.
> > > >
> > > > [1] https://docs.google.com/document/d/1Iyn-l3Nm0yls3pZaul4lZiVj8x1s73Ed6rOsmn6LfXc/edit#heading=h.qexbtyc2jpwr
> > >
> > >
> > > I really worry about how provisioning will work. And I do not at all
> > > cherish replicating all of these query capability commands for provisioning.
> >
> > +1
> >
> > There's nothing that prevents the config space from being implemented
> > in a way other than registers.
>
> Jason I'm not sure what does your +1 mean here. Are you ok with the
> patchset as is? While I feel config space would be nicer I also find it
> unlikely drivers will actually bother to check the supported commands.
> So from that POV, for this use-case I am not too stressed out.

I meant I prefer config space access.

1) It simplifies the provisioning and compatibility check a lot.
2) We don't need a mini driver to probe the features

We could invent a DMA interface for config space if necessary
(actually there's one in the transport vq series).

Thanks

>
> --
> MST
>


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/