From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52166) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XW3bG-0003La-UV for qemu-devel@nongnu.org; Mon, 22 Sep 2014 09:24:25 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XW3b8-0004j0-5E for qemu-devel@nongnu.org; Mon, 22 Sep 2014 09:24:18 -0400 Received: from [58.251.49.30] (port=32944 helo=mail.sangfor.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XW3b7-0004iM-FK for qemu-devel@nongnu.org; Mon, 22 Sep 2014 09:24:10 -0400 Date: Mon, 22 Sep 2014 21:23:24 +0800 From: "Zhang Haoyu" References: <201408291545282753855@sangfor.com>, <20140829143849.GA8909@grmbl.mre>, <201409012038178763909@sangfor.com>, <201409012052442706397@sangfor.com>, <20140902063637.GG14171@grmbl.mre>, <20140919055333.GA6889@fam-t430.nay.redhat.com> Message-ID: <201409222123220404614@sangfor.com> Mime-Version: 1.0 Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [question] virtio-blk performancedegradationhappened with virito-serial List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Fam Zheng , Amit Shah Cc: Paolo Bonzini , qemu-devel , kvm >> > >>> Hi, all >> > >>> >> > >>> I start a VM with virtio-serial (default ports number: 31), and found that virtio-blk performance degradation happened, about 25%, this problem can be reproduced 100%. >> > >>> without virtio-serial: >> > >>> 4k-read-random 1186 IOPS >> > >>> with virtio-serial: >> > >>> 4k-read-random 871 IOPS >> > >>> >> > >>> but if use max_ports=2 option to limit the max number of virio-serial ports, then the IO performance degradation is not so serious, about 5%. >> > >>> >> > >>> And, ide performance degradation does not happen with virtio-serial. >> > >> >> > >>Pretty sure it's related to MSI vectors in use. It's possible that >> > >>the virtio-serial device takes up all the avl vectors in the guests, >> > >>leaving old-style irqs for the virtio-blk device. >> > >> >> > >I don't think so, >> > >I use iometer to test 64k-read(or write)-sequence case, if I disable the virtio-serial dynamically via device manager->virtio-serial => disable, >> > >then the performance get promotion about 25% immediately, then I re-enable the virtio-serial via device manager->virtio-serial => enable, >> > >the performance got back again, very obvious. >> > add comments: >> > Although the virtio-serial is enabled, I don't use it at all, the degradation still happened. >> >> Using the vectors= option as mentioned below, you can restrict the >> number of MSI vectors the virtio-serial device gets. You can then >> confirm whether it's MSI that's related to these issues. > >Amit, > >It's related to the big number of ioeventfds used in virtio-serial-pci. With >virtio-serial-pci's ioeventfd=off, the performance is not affected no matter if >guest initializes it or not. > >In my test, there are 12 fds to poll in qemu_poll_ns before loading guest >virtio_console.ko, whereas 76 once modprobe virtio_console. > >Looks like the ppoll takes more time to poll more fds. > >Some trace data with systemtap: > >12 fds: > >time rel_time symbol >15 (+1) qemu_poll_ns [enter] >18 (+3) qemu_poll_ns [return] > >76 fd: > >12 (+2) qemu_poll_ns [enter] >18 (+6) qemu_poll_ns [return] > >I haven't looked at virtio-serial code, I'm not sure if we should reduce the >number of ioeventfds in virtio-serial-pci or focus on lower level efficiency. > Does ioeventfd=off hamper the performance of virtio-serial? In my opinion, virtio-serial's use scenario is not so high throughput rate, so ioventfd=off has slight impaction on the performance. Thanks, Zhang Haoyu >Haven't compared with g_poll but I think the underlying syscall should be the >same. > >Any ideas? > >Fam > > >> >> > >So, I think it has no business with legacy interrupt mode, right? >> > > >> > >I am going to observe the difference of perf top data on qemu and perf kvm stat data when disable/enable virtio-serial in guest, >> > >and the difference of perf top data on guest when disable/enable virtio-serial in guest, >> > >any ideas? >> > > >> > >Thanks, >> > >Zhang Haoyu >> > >>If you restrict the number of vectors the virtio-serial device gets >> > >>(using the -device virtio-serial-pci,vectors= param), does that make >> > >>things better for you? >> >> >> >> Amit