[PATCH v3] virtio-net: clarify coalescing parameters settings

[PATCH v3] virtio-net: clarify coalescing parameters settings

[Heng Qi]

The device can set any initial coalescing parameters (0 or non-zero)
for the receive/send queue before the setting command is executed,
not just 0, enhancing device performance even without DIM enabled.

So we need to clarify descriptions that don't fit the behavior.

Suggested-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
---
v2->v3:
  - Clarify description to be more generic. @Parav

v1->v2:
  - Update description. @Jason

 device-types/net/description.tex | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/device-types/net/description.tex b/device-types/net/description.tex
index 61cce1f..f51e0d8 100644
--- a/device-types/net/description.tex
+++ b/device-types/net/description.tex
@@ -1803,6 +1803,14 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
                                         for an enabled transmit/receive virtqueue whose index is \field{vq_index}.
 \end{enumerate}
 
+If the VIRTIO_NET_F_NOTF_COAL or VIRTIO_NET_F_VQ_NOTF_COAL feature is negotiated,
+the device may apply any coalescing parameters to each transmit/receive virtqueue
+before the driver successfully performs one of the VIRTIO_NET_CTRL_NOTF_COAL set commands.
+
+The driver can query the current coalescing parameters of any enabled transmit/receive
+virtqueue using the VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET command, before or after any
+VIRTIO_NET_CTRL_NOTF_COAL set command is done.
+
 The device may generate notifications more or less frequently than specified by set commands of the VIRTIO_NET_CTRL_NOTF_COAL class.
 
 If coalescing parameters are being set, the device applies the last coalescing parameters set for a
@@ -1884,18 +1892,16 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
 The device MUST respond to VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET and VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET commands with
 VIRTIO_NET_ERR if the designated virtqueue is not an enabled transmit or receive virtqueue.
 
-Upon disabling and re-enabling a transmit virtqueue, the device MUST set the coalescing parameters of the virtqueue
-to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command, or, if the driver did not set any TX coalescing parameters, to 0.
-
-Upon disabling and re-enabling a receive virtqueue, the device MUST set the coalescing parameters of the virtqueue
-to those configured through the VIRTIO_NET_CTRL_NOTF_COAL_RX_SET command, or, if the driver did not set any RX coalescing parameters, to 0.
-
 The behavior of the device in response to set commands of the VIRTIO_NET_CTRL_NOTF_COAL class is best-effort:
 the device MAY generate notifications more or less frequently than specified.
 
 A device SHOULD NOT send used buffer notifications to the driver if the notifications are suppressed, even if the notification conditions are met.
 
-Upon reset, a device MUST initialize all coalescing parameters to 0.
+Upon disabling and then re-enabling a transmit or receive virtqueue, the device
+MUST apply the same coalescing parameters to the virtqueue as were set before
+it was disabled.
+
+Upon reset, a device MAY set any coalescing parameters for all transmit or receive virtqueues.
 
 \paragraph{Device Statistics}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Device Statistics}
 
-- 
2.32.0.3.g01195cf9f

RE: [PATCH v3] virtio-net: clarify coalescing parameters settings

[Parav Pandit]

> From: Heng Qi <hengqi@linux.alibaba.com>
> Sent: Thursday, May 23, 2024 8:54 AM
> 
> The device can set any initial coalescing parameters (0 or non-zero) for the
> receive/send queue before the setting command is executed, not just 0,
> enhancing device performance even without DIM enabled.
> 
> So we need to clarify descriptions that don't fit the behavior.
> 
> Suggested-by: Jason Wang <jasowang@redhat.com>
> Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> ---
> v2->v3:
>   - Clarify description to be more generic. @Parav
> 
> v1->v2:
>   - Update description. @Jason
> 
>  device-types/net/description.tex | 20 +++++++++++++-------
>  1 file changed, 13 insertions(+), 7 deletions(-)
> 
> diff --git a/device-types/net/description.tex b/device-
> types/net/description.tex
> index 61cce1f..f51e0d8 100644
> --- a/device-types/net/description.tex
> +++ b/device-types/net/description.tex
> @@ -1803,6 +1803,14 @@ \subsubsection{Control
> Virtqueue}\label{sec:Device Types / Network Device / Devi
>                                          for an enabled transmit/receive virtqueue whose index is
> \field{vq_index}.
>  \end{enumerate}
> 
> +If the VIRTIO_NET_F_NOTF_COAL or VIRTIO_NET_F_VQ_NOTF_COAL feature
> is
> +negotiated, the device may apply any coalescing parameters to each
> +transmit/receive virtqueue before the driver successfully performs one of the
> VIRTIO_NET_CTRL_NOTF_COAL set commands.
> +
> +The driver can query the current coalescing parameters of any enabled
> +transmit/receive virtqueue using the
> VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET
> +command, before or after any VIRTIO_NET_CTRL_NOTF_COAL set command
> is done.
> +
>  The device may generate notifications more or less frequently than specified
> by set commands of the VIRTIO_NET_CTRL_NOTF_COAL class.
> 
>  If coalescing parameters are being set, the device applies the last coalescing
> parameters set for a @@ -1884,18 +1892,16 @@ \subsubsection{Control
> Virtqueue}\label{sec:Device Types / Network Device / Devi  The device MUST
> respond to VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET and
> VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET commands with  VIRTIO_NET_ERR if
> the designated virtqueue is not an enabled transmit or receive virtqueue.
> 
> -Upon disabling and re-enabling a transmit virtqueue, the device MUST set the
> coalescing parameters of the virtqueue -to those configured through the
> VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command, or, if the driver did not set
> any TX coalescing parameters, to 0.
> -
> -Upon disabling and re-enabling a receive virtqueue, the device MUST set the
> coalescing parameters of the virtqueue -to those configured through the
> VIRTIO_NET_CTRL_NOTF_COAL_RX_SET command, or, if the driver did not set
> any RX coalescing parameters, to 0.
> -

These two normative statements are correct, the change should be only for 0 and only for TX and RX commands.
(no need to extend that to per VQ level commands).

There is no need to burden the device to remember the old value for each queue when the q is disabled.
No point in remember something when the device does not know if it will be ever enabled or not.

So it should be,  

"or, If the driver did not set any coalescing parameters, the device MAY initialize it to any value."

>  The behavior of the device in response to set commands of the
> VIRTIO_NET_CTRL_NOTF_COAL class is best-effort:
>  the device MAY generate notifications more or less frequently than specified.
> 
>  A device SHOULD NOT send used buffer notifications to the driver if the
> notifications are suppressed, even if the notification conditions are met.
> 
> -Upon reset, a device MUST initialize all coalescing parameters to 0.

> +Upon disabling and then re-enabling a transmit or receive virtqueue,
> +the device MUST apply the same coalescing parameters to the virtqueue
> +as were set before it was disabled.
> +

I think it should be, 
When VIRTIO_NET_F_VQ_NOTF_COAL feature is negotiated, upon disabling and then re-enabling a transmit or receive virtqueue, the device MAY set any coalescing parameters for all transmit or
receive virtqueues.

This clarifies the current behavior and also does not burden the device to remember any values related to VQ resource that is disabled (or destroyed).

> +Upon reset, a device MAY set any coalescing parameters for all transmit or
> receive virtqueues.
> 
>  \paragraph{Device Statistics}\label{sec:Device Types / Network Device /
> Device Operation / Control Virtqueue / Device Statistics}
> 
> --
> 2.32.0.3.g01195cf9f

Re: RE: [PATCH v3] virtio-net: clarify coalescing parameters settings

[Heng Qi]

On Thu, 23 May 2024 05:07:16 +0000, Parav Pandit <parav@nvidia.com> wrote:
> 
> > From: Heng Qi <hengqi@linux.alibaba.com>
> > Sent: Thursday, May 23, 2024 8:54 AM
> > 
> > The device can set any initial coalescing parameters (0 or non-zero) for the
> > receive/send queue before the setting command is executed, not just 0,
> > enhancing device performance even without DIM enabled.
> > 
> > So we need to clarify descriptions that don't fit the behavior.
> > 
> > Suggested-by: Jason Wang <jasowang@redhat.com>
> > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > ---
> > v2->v3:
> >   - Clarify description to be more generic. @Parav
> > 
> > v1->v2:
> >   - Update description. @Jason
> > 
> >  device-types/net/description.tex | 20 +++++++++++++-------
> >  1 file changed, 13 insertions(+), 7 deletions(-)
> > 
> > diff --git a/device-types/net/description.tex b/device-
> > types/net/description.tex
> > index 61cce1f..f51e0d8 100644
> > --- a/device-types/net/description.tex
> > +++ b/device-types/net/description.tex
> > @@ -1803,6 +1803,14 @@ \subsubsection{Control
> > Virtqueue}\label{sec:Device Types / Network Device / Devi
> >                                          for an enabled transmit/receive virtqueue whose index is
> > \field{vq_index}.
> >  \end{enumerate}
> > 
> > +If the VIRTIO_NET_F_NOTF_COAL or VIRTIO_NET_F_VQ_NOTF_COAL feature
> > is
> > +negotiated, the device may apply any coalescing parameters to each
> > +transmit/receive virtqueue before the driver successfully performs one of the
> > VIRTIO_NET_CTRL_NOTF_COAL set commands.
> > +
> > +The driver can query the current coalescing parameters of any enabled
> > +transmit/receive virtqueue using the
> > VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET
> > +command, before or after any VIRTIO_NET_CTRL_NOTF_COAL set command
> > is done.
> > +
> >  The device may generate notifications more or less frequently than specified
> > by set commands of the VIRTIO_NET_CTRL_NOTF_COAL class.
> > 
> >  If coalescing parameters are being set, the device applies the last coalescing
> > parameters set for a @@ -1884,18 +1892,16 @@ \subsubsection{Control
> > Virtqueue}\label{sec:Device Types / Network Device / Devi  The device MUST
> > respond to VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET and
> > VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET commands with  VIRTIO_NET_ERR if
> > the designated virtqueue is not an enabled transmit or receive virtqueue.
> > 
> > -Upon disabling and re-enabling a transmit virtqueue, the device MUST set the
> > coalescing parameters of the virtqueue -to those configured through the
> > VIRTIO_NET_CTRL_NOTF_COAL_TX_SET command, or, if the driver did not set
> > any TX coalescing parameters, to 0.
> > -
> > -Upon disabling and re-enabling a receive virtqueue, the device MUST set the
> > coalescing parameters of the virtqueue -to those configured through the
> > VIRTIO_NET_CTRL_NOTF_COAL_RX_SET command, or, if the driver did not set
> > any RX coalescing parameters, to 0.
> > -
> 
> These two normative statements are correct, the change should be only for 0 and only for TX and RX commands.
> (no need to extend that to per VQ level commands).
> 
> There is no need to burden the device to remember the old value for each queue when the q is disabled.
> No point in remember something when the device does not know if it will be ever enabled or not.

The current spec already tells the device to remember the old value.

But I think it's ok to be relaxed for the descriptions.

> 
> So it should be,  
> 
> "or, If the driver did not set any coalescing parameters, the device MAY initialize it to any value."

It works.

> 
> >  The behavior of the device in response to set commands of the
> > VIRTIO_NET_CTRL_NOTF_COAL class is best-effort:
> >  the device MAY generate notifications more or less frequently than specified.
> > 
> >  A device SHOULD NOT send used buffer notifications to the driver if the
> > notifications are suppressed, even if the notification conditions are met.
> > 
> > -Upon reset, a device MUST initialize all coalescing parameters to 0.
> 
> > +Upon disabling and then re-enabling a transmit or receive virtqueue,
> > +the device MUST apply the same coalescing parameters to the virtqueue
> > +as were set before it was disabled.
> > +
> 
> I think it should be, 
> When VIRTIO_NET_F_VQ_NOTF_COAL feature is negotiated, upon disabling and then re-enabling a transmit or receive virtqueue, the device MAY set any coalescing parameters for all transmit or
> receive virtqueues.
> 
> This clarifies the current behavior and also does not burden the device to remember any values related to VQ resource that is disabled (or destroyed).

Makes sense.

Thanks.

> 
> > +Upon reset, a device MAY set any coalescing parameters for all transmit or
> > receive virtqueues.
> > 
> >  \paragraph{Device Statistics}\label{sec:Device Types / Network Device /
> > Device Operation / Control Virtqueue / Device Statistics}
> > 
> > --
> > 2.32.0.3.g01195cf9f
> 

RE: RE: [PATCH v3] virtio-net: clarify coalescing parameters settings

[Parav Pandit]

> From: Heng Qi <hengqi@linux.alibaba.com>
> Sent: Thursday, May 23, 2024 12:06 PM
> 
> On Thu, 23 May 2024 05:07:16 +0000, Parav Pandit <parav@nvidia.com>
> wrote:
> >
> > > From: Heng Qi <hengqi@linux.alibaba.com>
> > > Sent: Thursday, May 23, 2024 8:54 AM
> > >
> > > The device can set any initial coalescing parameters (0 or non-zero)
> > > for the receive/send queue before the setting command is executed,
> > > not just 0, enhancing device performance even without DIM enabled.
> > >
> > > So we need to clarify descriptions that don't fit the behavior.
> > >
> > > Suggested-by: Jason Wang <jasowang@redhat.com>
> > > Signed-off-by: Heng Qi <hengqi@linux.alibaba.com>
> > > ---
> > > v2->v3:
> > >   - Clarify description to be more generic. @Parav
> > >
> > > v1->v2:
> > >   - Update description. @Jason
> > >
> > >  device-types/net/description.tex | 20 +++++++++++++-------
> > >  1 file changed, 13 insertions(+), 7 deletions(-)
> > >
> > > diff --git a/device-types/net/description.tex b/device-
> > > types/net/description.tex index 61cce1f..f51e0d8 100644
> > > --- a/device-types/net/description.tex
> > > +++ b/device-types/net/description.tex
> > > @@ -1803,6 +1803,14 @@ \subsubsection{Control
> > > Virtqueue}\label{sec:Device Types / Network Device / Devi
> > >                                          for an enabled
> > > transmit/receive virtqueue whose index is \field{vq_index}.
> > >  \end{enumerate}
> > >
> > > +If the VIRTIO_NET_F_NOTF_COAL or VIRTIO_NET_F_VQ_NOTF_COAL
> feature
> > > is
> > > +negotiated, the device may apply any coalescing parameters to each
> > > +transmit/receive virtqueue before the driver successfully performs
> > > +one of the
> > > VIRTIO_NET_CTRL_NOTF_COAL set commands.
> > > +
> > > +The driver can query the current coalescing parameters of any
> > > +enabled transmit/receive virtqueue using the
> > > VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET
> > > +command, before or after any VIRTIO_NET_CTRL_NOTF_COAL set
> command
> > > is done.
> > > +
> > >  The device may generate notifications more or less frequently than
> > > specified by set commands of the VIRTIO_NET_CTRL_NOTF_COAL class.
> > >
> > >  If coalescing parameters are being set, the device applies the last
> > > coalescing parameters set for a @@ -1884,18 +1892,16 @@
> > > \subsubsection{Control Virtqueue}\label{sec:Device Types / Network
> > > Device / Devi  The device MUST respond to
> > > VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET and
> > > VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET commands with
> VIRTIO_NET_ERR if the designated virtqueue is not an enabled transmit or
> receive virtqueue.
> > >
> > > -Upon disabling and re-enabling a transmit virtqueue, the device
> > > MUST set the coalescing parameters of the virtqueue -to those
> > > configured through the VIRTIO_NET_CTRL_NOTF_COAL_TX_SET
> command, or,
> > > if the driver did not set any TX coalescing parameters, to 0.
> > > -
> > > -Upon disabling and re-enabling a receive virtqueue, the device MUST
> > > set the coalescing parameters of the virtqueue -to those configured
> > > through the VIRTIO_NET_CTRL_NOTF_COAL_RX_SET command, or, if the
> > > driver did not set any RX coalescing parameters, to 0.
> > > -
> >
> > These two normative statements are correct, the change should be only for 0
> and only for TX and RX commands.
> > (no need to extend that to per VQ level commands).
> >
> > There is no need to burden the device to remember the old value for each
> queue when the q is disabled.
> > No point in remember something when the device does not know if it will be
> ever enabled or not.
> 
> The current spec already tells the device to remember the old value.
> 
It only tells for VIRTIO_NET_F_NOTF_COAL in above two normatives.

For VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET it is not called out explicitly, hence the device do not remember it.

> But I think it's ok to be relaxed for the descriptions.
Yes.
> 
> >
> > So it should be,
> >
> > "or, If the driver did not set any coalescing parameters, the device MAY
> initialize it to any value."
> 
> It works.
> 
> >
> > >  The behavior of the device in response to set commands of the
> > > VIRTIO_NET_CTRL_NOTF_COAL class is best-effort:
> > >  the device MAY generate notifications more or less frequently than
> specified.
> > >
> > >  A device SHOULD NOT send used buffer notifications to the driver if
> > > the notifications are suppressed, even if the notification conditions are
> met.
> > >
> > > -Upon reset, a device MUST initialize all coalescing parameters to 0.
> >
> > > +Upon disabling and then re-enabling a transmit or receive
> > > +virtqueue, the device MUST apply the same coalescing parameters to
> > > +the virtqueue as were set before it was disabled.
> > > +
> >
> > I think it should be,
> > When VIRTIO_NET_F_VQ_NOTF_COAL feature is negotiated, upon disabling
> > and then re-enabling a transmit or receive virtqueue, the device MAY set any
> coalescing parameters for all transmit or receive virtqueues.
> >
> > This clarifies the current behavior and also does not burden the device to
> remember any values related to VQ resource that is disabled (or destroyed).
> 
> Makes sense.
> 
> Thanks.
> 
> >
> > > +Upon reset, a device MAY set any coalescing parameters for all
> > > +transmit or
> > > receive virtqueues.
> > >
> > >  \paragraph{Device Statistics}\label{sec:Device Types / Network
> > > Device / Device Operation / Control Virtqueue / Device Statistics}
> > >
> > > --
> > > 2.32.0.3.g01195cf9f
> >