Re: [PATCH] vhost: make vhost_zerocopy_callback more efficient by poll_queue base on vhost status

[Jason Wang <jasowang@redhat.com>]

----- Original Message -----
> guest kick vhost base on vring flag status and get perfermance improved,
> vhost_zerocopy_callback could do this in the same way, as
> virtqueue_enable_cb need one more check after change the status of
> avail_ring flags, vhost also do the same thing after vhost_enable_notify
> 
> test result list as below:
> guest and host: suse11sp3, netperf, intel 2.4GHz
> +------+----------+--------+----------+--------+--------+---------+
> |      |             old              |            new            |
> +------+----------+--------+----------+--------+--------+---------+
> | UDP  |  Gbit/s  |  PPS   |CPU idle% | Gbit/s |   PPS  |CPU idle%|
> | 256  | 0.74805  | 321309 |  87.16   | 0.77933| 334743 |  90.71  |
> | 512  |   1.42   | 328475 |  87.03   |  1.44  | 333550 |  90.43  |
> | 1024 |   2.79   | 334426 |  89.09   |  2.81  | 336986 |  89.55  |
> | 1460 |   3.71   | 316215 |  87.53   |  4.02  | 342325 |  89.58  |
> +------+----------+--------+----------+--------+--------+---------+
> 
> Signed-off-by: Chuanyu Qin <qinchuanyu@huawei.com>
> ---
>   drivers/vhost/net.c |   12 ++++++++++--
>   1 files changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> index a0fa5de..3c76b89 100644
> --- a/drivers/vhost/net.c
> +++ b/drivers/vhost/net.c
> @@ -314,7 +314,8 @@ static void vhost_zerocopy_callback(struct ubuf_info
> *ubuf, bool success)
>   	vq->heads[ubuf->desc].len = success ?
>   		VHOST_DMA_DONE_LEN : VHOST_DMA_FAILED_LEN;
>   	cnt = vhost_net_ubuf_put(ubufs);
> -
> +	/* make sure len has been updated because handle_tx would use it */
> +	smp_wmb();

Hi Chuanyu:

Apologize first for my typo in the last comment, should be a smp_mb()
here to prevent the used_flags to be checked(read) before updating the
len (writing).

smp_wmb() can only prevent re-ordering between writes.

btw. If you want you patch to be applied for David tree, you need also
cc or to netdev.

>   	/*
>   	 * Trigger polling thread if guest stopped submitting new buffers:
>   	 * in this case, the refcount after decrement will eventually reach 1.
> @@ -322,7 +323,8 @@ static void vhost_zerocopy_callback(struct ubuf_info
> *ubuf, bool success)
>   	 * (the value 16 here is more or less arbitrary, it's tuned to trigger
>   	 * less than 10% of times).
>   	 */
> -	if (cnt <= 1 || !(cnt % 16))
> +	if ((!(vq->used_flags & VRING_USED_F_NO_NOTIFY))
> +			&& (cnt <= 1 || !(cnt % 16)))
>   		vhost_poll_queue(&vq->poll);
> 
>   	rcu_read_unlock_bh();
> @@ -386,6 +388,12 @@ static void handle_tx(struct vhost_net *net)
>   				vhost_disable_notify(&net->dev, vq);
>   				continue;
>   			}
> +			/* there might skb been freed between last
> +			* vhost_zerocopy_signal_used and vhost_enable_notify,
> +			* so one more check is needed.
> +			*/
> +			if (zcopy)
> +				vhost_zerocopy_signal_used(net, vq);
>   			break;
>   		}
>   		if (in) {
> --
> 1.7.3.1.msysgit.0
> 
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>