[virtio-comment] [PATCH v6] virtio-net: Add support for the flexible driver notification structure

[virtio-comment] [PATCH v6] virtio-net: Add support for the flexible driver notification structure

[Vitaly Mireyno]

Currently, the driver notification (available buffer notification) has a fixed structure, which includes the virtqueue number.
If notify_off_multiplier > 0, the VQ number can be derived by the device from the Queue Notify address, so providing virtqueue number as part of the data may be redundant.

Some devices benefit from receiving an additional data with driver notifications. This data can optionally replace the vqn field in the driver notification structure.
In its simplest form, it would be sufficient for this data to be a per-device constant value.

Changes from v5:
 * Changed the feature name to VIRTIO_NET_F_NOTIF_CONFIG_DATA
 * Removed dependency on VIRTIO_F_NOTIFICATION_DATA


Signed-off-by: Vitaly Mireyno <vmireyno@marvell.com>
---
 content.tex | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/content.tex b/content.tex
index b91a132..abe9c71 100644
--- a/content.tex
+++ b/content.tex
@@ -965,6 +965,8 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
 struct virtio_pci_notify_cap {
         struct virtio_pci_cap cap;
         le32 notify_off_multiplier; /* Multiplier for queue_notify_off. */
+        le16 notify_data; /* Data to be placed in the vqn field */
+        le16 padding; /* Pad to a dword */
 };
 \end{lstlisting}
 
@@ -984,6 +986,21 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
 the same Queue Notify address for all queues.
 \end{note}
 
+\field{notify_data} is the data that the driver will set in the available
+buffer notification (instead of the virtqueue number), if
+VIRTIO_NET_F_NOTIF_CONFIG_DATA has been negotiated.
+
+\begin{note}
+If \field{notify_off_multiplier} > 0, the virtqueue number can potentially be
+derived by the device from the Queue Notify address, so \field{vqn} may be
+redundant. Some devices benefit from receiving additional data with driver
+notifications. An example could be a hardware device implementing multiple
+protocols (with virtio being one of them), so the extra notification data could
+serve as a notification type indication or a protocol indication.
+Another example could be using shared hardware memory space for driver
+notifications for multiple virtio devices.
+\end{note}
+
 \devicenormative{\paragraph}{Notification capability}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability}
 The device MUST present at least one notification capability.
 
@@ -1020,6 +1037,9 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
 cap.length >= queue_notify_off * notify_off_multiplier + 4
 \end{lstlisting}
 
+If the device offers VIRTIO_NET_F_NOTIF_CONFIG_DATA, it MUST set
+\field{notify_off_multiplier} > 0 in at least one capability.
+
 \subsubsection{ISR status capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / ISR status capability}
 
 The VIRTIO_PCI_CAP_ISR_CFG capability
@@ -1519,6 +1539,15 @@ \subsubsection{Available Buffer Notifications}\label{sec:Virtio Transport Option
 See \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability}
 for how to calculate the Queue Notify address.
 
+\drivernormative{\paragraph}{Available Buffer Notifications}{Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Available Buffer Notifications}
+The driver SHOULD accept the VIRTIO_NET_F_NOTIF_CONFIG_DATA feature if it has
+been offered.
+
+If VIRTIO_NET_F_NOTIF_CONFIG_DATA has been negotiated and
+\field{notify_off_multiplier} > 0, the driver MUST set the 'virtqueue number'
+field of the available buffer notification structure to the \field{notify_data}
+value.
+
 \subsubsection{Used Buffer Notifications}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Used Buffer Notifications}
 
 If a used buffer notification is necessary for a virtqueue, the device would typically act as follows:
@@ -2895,6 +2924,10 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
 \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
     channel.
 
+\item[VIRTIO_NET_F_NOTIF_CONFIG_DATA(57)] Driver provides extra data with
+    available buffer notifications, to aid in notification processing by the
+    device.
+
 \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
     value. Device benefits from knowing the exact header length.
 
--

This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/

[virtio-comment] Re: [PATCH v6] virtio-net: Add support for the flexible driver notification structure

[Michael S. Tsirkin]

On Thu, Mar 12, 2020 at 04:42:26PM +0000, Vitaly Mireyno wrote:
> Currently, the driver notification (available buffer notification) has a fixed structure, which includes the virtqueue number.
> If notify_off_multiplier > 0, the VQ number can be derived by the device from the Queue Notify address, so providing virtqueue number as part of the data may be redundant.
> 
> Some devices benefit from receiving an additional data with driver notifications. This data can optionally replace the vqn field in the driver notification structure.
> In its simplest form, it would be sufficient for this data to be a per-device constant value.
> 
> Changes from v5:
>  * Changed the feature name to VIRTIO_NET_F_NOTIF_CONFIG_DATA
>  * Removed dependency on VIRTIO_F_NOTIFICATION_DATA
> 
> 
> Signed-off-by: Vitaly Mireyno <vmireyno@marvell.com>

OK so I guess we should cancel the vote on the previous version?

> ---
>  content.tex | 33 +++++++++++++++++++++++++++++++++
>  1 file changed, 33 insertions(+)
> 
> diff --git a/content.tex b/content.tex
> index b91a132..abe9c71 100644
> --- a/content.tex
> +++ b/content.tex
> @@ -965,6 +965,8 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
>  struct virtio_pci_notify_cap {
>          struct virtio_pci_cap cap;
>          le32 notify_off_multiplier; /* Multiplier for queue_notify_off. */
> +        le16 notify_data; /* Data to be placed in the vqn field */
> +        le16 padding; /* Pad to a dword */
>  };
>  \end{lstlisting}
>  
> @@ -984,6 +986,21 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
>  the same Queue Notify address for all queues.
>  \end{note}
>  
> +\field{notify_data} is the data that the driver will set in the available
> +buffer notification (instead of the virtqueue number), if
> +VIRTIO_NET_F_NOTIF_CONFIG_DATA has been negotiated.
> +
> +\begin{note}
> +If \field{notify_off_multiplier} > 0, the virtqueue number can potentially be
> +derived by the device from the Queue Notify address, so \field{vqn} may be
> +redundant. Some devices benefit from receiving additional data with driver
> +notifications. An example could be a hardware device implementing multiple
> +protocols (with virtio being one of them), so the extra notification data could
> +serve as a notification type indication or a protocol indication.
> +Another example could be using shared hardware memory space for driver
> +notifications for multiple virtio devices.


I think this last example is bogus. It can not guarantee protection
between devices. Let's just drop this example.

> +\end{note}
> +
>  \devicenormative{\paragraph}{Notification capability}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability}
>  The device MUST present at least one notification capability.
>  
> @@ -1020,6 +1037,9 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
>  cap.length >= queue_notify_off * notify_off_multiplier + 4
>  \end{lstlisting}
>  
> +If the device offers VIRTIO_NET_F_NOTIF_CONFIG_DATA, it MUST set
> +\field{notify_off_multiplier} > 0 in at least one capability.
> +
>  \subsubsection{ISR status capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / ISR status capability}
>  
>  The VIRTIO_PCI_CAP_ISR_CFG capability
> @@ -1519,6 +1539,15 @@ \subsubsection{Available Buffer Notifications}\label{sec:Virtio Transport Option
>  See \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability}
>  for how to calculate the Queue Notify address.
>  
> +\drivernormative{\paragraph}{Available Buffer Notifications}{Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Available Buffer Notifications}
> +The driver SHOULD accept the VIRTIO_NET_F_NOTIF_CONFIG_DATA feature if it has
> +been offered.
> +
> +If VIRTIO_NET_F_NOTIF_CONFIG_DATA has been negotiated and
> +\field{notify_off_multiplier} > 0, the driver MUST set the 'virtqueue number'
> +field of the available buffer notification structure to the \field{notify_data}
> +value.
> +
>  \subsubsection{Used Buffer Notifications}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Used Buffer Notifications}
>  
>  If a used buffer notification is necessary for a virtqueue, the device would typically act as follows:
> @@ -2895,6 +2924,10 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
>  \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
>      channel.
>  
> +\item[VIRTIO_NET_F_NOTIF_CONFIG_DATA(57)] Driver provides extra data with
> +    available buffer notifications, to aid in notification processing by the
> +    device.
> +
>  \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
>      value. Device benefits from knowing the exact header length.
>  
> --


This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.

In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.

Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/