From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5ADB6C43381 for ; Tue, 2 Apr 2019 04:19:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2967920857 for ; Tue, 2 Apr 2019 04:19:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="o0oI1hHa" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728211AbfDBET2 (ORCPT ); Tue, 2 Apr 2019 00:19:28 -0400 Received: from mail-wr1-f68.google.com ([209.85.221.68]:40346 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726097AbfDBET2 (ORCPT ); Tue, 2 Apr 2019 00:19:28 -0400 Received: by mail-wr1-f68.google.com with SMTP id h4so14656756wre.7 for ; Mon, 01 Apr 2019 21:19:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=references:user-agent:from:to:cc:subject:in-reply-to:date :message-id:mime-version:content-transfer-encoding; bh=VdkYNC+sTJIBiXITjo35tptTJEiW6bg6UT/l5mSJ3s0=; b=o0oI1hHaBQeqV0E11VZ2T+wM4Q4Z2lfnUerJQlLHMaXZ17GhrLy30v7PfTvURWwTUm Rr1oDpOUTgUvMRq7hahOSrNHV8FmMJa4xLjfmZ4csfDblQSPn18zJdeGldPDem91mA/g 85BIRvS37l8/Dqb8eYsFu9pMsQ5tV5IVnBlbtIgNbiWpNEt2lq0GTrlylO4682Qk+gYo XNky/AmLJ+ZmKl0nEYdYSdzq9zigO6ZHDS6G/3QOz1oik6CBi/9Rq7n0j2cCirkYFC+d 1GQWaFQSbS062pooP6TqYJyoy9Sj56l3zlnR2tvrfK91JW0GC7Gp8QpHFR2IYoA4QCfr K7vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:references:user-agent:from:to:cc:subject :in-reply-to:date:message-id:mime-version:content-transfer-encoding; bh=VdkYNC+sTJIBiXITjo35tptTJEiW6bg6UT/l5mSJ3s0=; b=rMD6AWb+g58vOIdo+0fnBnGFHvAwOJLX9vAI37LiajZD2i1wDV9KbFt0BWfZ+HEP7b HvyMgulw2ClcTo1nB+xWbzt499UVIsgs3e26aee/AgTwLCF09BO49iCpqltcUHBqlLOc fK5U7S77lO66NqixRQEDMFvL4ye1ewL7F680y3OYj0Da1nuApjIelw5GMNRU4kzgmHcc wXvD8fnURZkq0Kb4/npt3aDx7KHCaVggBPpTBcWc0ekg7KAmE+4yHQxLIA9dDdgBkqvz 6JFo7uCW+wdTx/hgx/ciKB2qXoAdCPIO2sHzId6/T6kVV148RJ1DWhYLzA55LfNvfhiS smnw== X-Gm-Message-State: APjAAAW5hnna/KFLtT5tf/rtF+bmgm/jYU16CRXXbQKDlE0u8HNtpAuR tDn3Z9V6i2CFSpdbFvE99hMfCA== X-Google-Smtp-Source: APXvYqxCwgmelWSDT05p7xfrU2otdpRB0tswO2IfF4ssTWkvcj8okEd0O32/Jetx4/+ztjJP6Y7Asg== X-Received: by 2002:adf:e889:: with SMTP id d9mr20013048wrm.162.1554178766483; Mon, 01 Apr 2019 21:19:26 -0700 (PDT) Received: from zen.linaroharston ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id r6sm10155735wrt.38.2019.04.01.21.19.25 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 01 Apr 2019 21:19:25 -0700 (PDT) Received: from zen (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 6298D1FF87; Tue, 2 Apr 2019 04:19:25 +0000 (UTC) References: <20190401163240.xw24ezsloy5ds2hz@steredhat> User-agent: mu4e 1.1.0; emacs 26.1 From: Alex =?utf-8?Q?Benn=C3=A9e?= To: Stefano Garzarella Cc: qemu devel list , netdev@vger.kernel.org Subject: Re: VSOCK benchmark and optimizations In-reply-to: <20190401163240.xw24ezsloy5ds2hz@steredhat> Date: Tue, 02 Apr 2019 04:19:25 +0000 Message-ID: <87zhp9599u.fsf@zen.linaroharston> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Stefano Garzarella writes: > Hi Alex, > I'm sending you some benchmarks and information about VSOCK CCing qemu-de= vel > and linux-netdev (maybe this info could be useful for others :)) > > One of the VSOCK advantages is the simple configuration: you don't need t= o set > up IP addresses for guest/host, and it can be used with the standard POSIX > socket API. [1] > > I'm currently working on it, so the "optimized" values are still work in > progress and I'll send the patches upstream (Linux) as soon as possible. > (I hope in 1 or 2 weeks) > > Optimizations: > + reducing the number of credit update packets > - RX side sent, on every packet received, an empty packet only to infor= m the > TX side about the space in the RX buffer. > + increase RX buffers size to 64 KB (from 4 KB) > + merge packets to fill RX buffers > > As benchmark tool I used iperf3 [2] modified with VSOCK support: > > host -> guest [Gbps] guest -> host [Gbps] > pkt_size before opt. optimized before opt. optimized > 1K 0.5 1.6 1.4 1.4 > 2K 1.1 3.1 2.3 2.5 > 4K 2.0 5.6 4.2 4.4 > 8K 3.2 10.2 7.2 7.5 > 16K 6.4 14.2 9.4 11.3 > 32K 9.8 18.9 9.2 17.8 > 64K 13.8 22.9 8.8 25.0 > 128K 17.6 24.5 7.7 25.7 > 256K 19.0 24.8 8.1 25.6 > 512K 20.8 25.1 8.1 25.4 > > > How to reproduce: > > host$ modprobe vhost_vsock > host$ qemu-system-x86_64 ... -device vhost-vsock-pci,guest-cid=3D3 > # Note: Guest CID should be >=3D 3 > # (0, 1 are reserved and 2 identify the host) > > guest$ iperf3 --vsock -s > > host$ iperf3 --vsock -c 3 -l ${pkt_size} # host -> guest > host$ iperf3 --vsock -c 3 -l ${pkt_size} -R # guest -> host > > > If you want, I can do a similar benchmark (with iperf3) using a networking > card (do you have a specific configuration?). My main interest is how it stacks up against: --device virtio-net-pci and I guess the vhost equivalent AIUI one of the motivators was being able to run something like NFS for a guest FS over vsock instead of the overhead from UDP and having to deal with the additional complication of having a working network setup. > > Let me know if you need more details! > > Thanks, > Stefano > > [1] https://wiki.qemu.org/Features/VirtioVsock > [2] https://github.com/stefano-garzarella/iperf/ --=20 Alex Benn=C3=A9e