From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jason Wang Subject: Re: [RFC] VSOCK: The performance problem of vhost_vsock. Date: Wed, 17 Oct 2018 17:51:23 +0800 Message-ID: <8edff784-b311-bfb6-1bf4-1970d564279d@redhat.com> References: <5BC3F0D4.60409@huawei.com> <30d7c370-b206-cdac-dc85-53e9be1e1c63@redhat.com> <5BC42FD8.2070104@huawei.com> <5BC70069.4000600@huawei.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Cc: netdev@vger.kernel.org, kvm@vger.kernel.org, virtualization@lists.linux-foundation.org To: jiangyiwen , stefanha@redhat.com Return-path: Received: from mx1.redhat.com ([209.132.183.28]:35152 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726636AbeJQRqW (ORCPT ); Wed, 17 Oct 2018 13:46:22 -0400 In-Reply-To: Content-Language: en-US Sender: netdev-owner@vger.kernel.org List-ID: On 2018/10/17 下午5:39, Jason Wang wrote: >>> >> Hi Jason and Stefan, >> >> Maybe I find the reason of bad performance. >> >> I found pkt_len is limited to VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE(4K), >> it will cause the bandwidth is limited to 500~600MB/s. And once I >> increase to 64k, it can improve about 3 times(~1500MB/s). > > > Looks like the value was chosen for a balance between rx buffer size > and performance. Allocating 64K always even for small packet is kind > of waste and stress for guest memory. Virito-net try to avoid this by > inventing the merge able rx buffer which allows big packet to be > scattered in into different buffers. We can reuse this idea or revisit > the idea of using virtio-net/vhost-net as a transport of vsock. > > What interesting is the performance is still behind vhost-net. > > Thanks > >> >> By the way, I send to 64K in application once, and I don't use >> sg_init_one and rewrite function to packet sg list because pkt_len >> include multiple pages. >> >> Thanks, >> Yiwen. Btw, if you're using vsock for transferring large files, maybe it's more efficient to implement sendpage() for vsock to allow sendfile()/splice() work. Thanks