Re: [PATCH v2] virtio: use shadow_avail_idx while checking number of heads

[Stefan Hajnoczi <stefanha@gmail.com>]

On Wed, 27 Sept 2023 at 09:52, Ilya Maximets <i.maximets@ovn.org> wrote:
>
> We do not need the most up to date number of heads, we only want to
> know if there is at least one.
>
> Use shadow variable as long as it is not equal to the last available
> index checked.  This avoids expensive qatomic dereference of the
> RCU-protected memory region cache as well as the memory access itself.
>
> The change improves performance of the af-xdp network backend by 2-3%.
>
> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
> ---
>
> Version 2:
>   - Changed to not skip error checks and a barrier.
>   - Added comments about the need for a barrier.

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>

>
>  hw/virtio/virtio.c | 18 +++++++++++++++---
>  1 file changed, 15 insertions(+), 3 deletions(-)
>
> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> index 4577f3f5b3..8a4c3e95d2 100644
> --- a/hw/virtio/virtio.c
> +++ b/hw/virtio/virtio.c
> @@ -999,7 +999,12 @@ void virtqueue_push(VirtQueue *vq, const VirtQueueElement *elem,
>  /* Called within rcu_read_lock().  */
>  static int virtqueue_num_heads(VirtQueue *vq, unsigned int idx)
>  {
> -    uint16_t num_heads = vring_avail_idx(vq) - idx;
> +    uint16_t avail_idx, num_heads;
> +
> +    /* Use shadow index whenever possible. */
> +    avail_idx = (vq->shadow_avail_idx != idx) ? vq->shadow_avail_idx
> +                                              : vring_avail_idx(vq);
> +    num_heads = avail_idx - idx;
>
>      /* Check it isn't doing very strange things with descriptor numbers. */
>      if (num_heads > vq->vring.num) {
> @@ -1007,8 +1012,15 @@ static int virtqueue_num_heads(VirtQueue *vq, unsigned int idx)
>                       idx, vq->shadow_avail_idx);
>          return -EINVAL;
>      }
> -    /* On success, callers read a descriptor at vq->last_avail_idx.
> -     * Make sure descriptor read does not bypass avail index read. */
> +    /*
> +     * On success, callers read a descriptor at vq->last_avail_idx.
> +     * Make sure descriptor read does not bypass avail index read.
> +     *
> +     * This is necessary even if we are using a shadow index, since
> +     * the shadow index could have been initialized by calling
> +     * vring_avail_idx() outside of this function, i.e., by a guest
> +     * memory read not accompanied by a barrier.
> +     */
>      if (num_heads) {
>          smp_rmb();
>      }
> --
> 2.41.0
>
>