From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51103) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c6IbY-0007us-WF for qemu-devel@nongnu.org; Mon, 14 Nov 2016 09:51:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c6IbW-00068U-C0 for qemu-devel@nongnu.org; Mon, 14 Nov 2016 09:51:29 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:58703) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1c6IbW-000689-4H for qemu-devel@nongnu.org; Mon, 14 Nov 2016 09:51:26 -0500 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id uAEEpNhw039280 for ; Mon, 14 Nov 2016 09:51:24 -0500 Received: from e06smtp14.uk.ibm.com (e06smtp14.uk.ibm.com [195.75.94.110]) by mx0a-001b2d01.pphosted.com with ESMTP id 26qfda001j-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Mon, 14 Nov 2016 09:51:23 -0500 Received: from localhost by e06smtp14.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 14 Nov 2016 14:51:21 -0000 Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by d06dlp01.portsmouth.uk.ibm.com (Postfix) with ESMTP id B05D117D805A for ; Mon, 14 Nov 2016 14:53:42 +0000 (GMT) Received: from d06av06.portsmouth.uk.ibm.com (d06av06.portsmouth.uk.ibm.com [9.149.37.217]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id uAEEpJk423134276 for ; Mon, 14 Nov 2016 14:51:19 GMT Received: from d06av06.portsmouth.uk.ibm.com (localhost [127.0.0.1]) by d06av06.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id uAEEpJli008358 for ; Mon, 14 Nov 2016 09:51:19 -0500 References: <1478711602-12620-1-git-send-email-stefanha@redhat.com> From: Christian Borntraeger Date: Mon, 14 Nov 2016 15:51:18 +0100 MIME-Version: 1.0 In-Reply-To: <1478711602-12620-1-git-send-email-stefanha@redhat.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Message-Id: <1bca4a16-5a2c-8de7-b66d-a8ceb3e5a756@de.ibm.com> Subject: Re: [Qemu-devel] [RFC 0/3] aio: experimental virtio-blk polling mode List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi , qemu-devel@nongnu.org Cc: Paolo Bonzini , Fam Zheng , Karl Rister On 11/09/2016 06:13 PM, Stefan Hajnoczi wrote: > Recent performance investigation work done by Karl Rister shows that the > guest->host notification takes around 20 us. This is more than the "overhead" > of QEMU itself (e.g. block layer). > > One way to avoid the costly exit is to use polling instead of notification. > The main drawback of polling is that it consumes CPU resources. In order to > benefit performance the host must have extra CPU cycles available on physical > CPUs that aren't used by the guest. > > This is an experimental AioContext polling implementation. It adds a polling > callback into the event loop. Polling functions are implemented for virtio-blk > virtqueue guest->host kick and Linux AIO completion. > > The QEMU_AIO_POLL_MAX_NS environment variable sets the number of nanoseconds to > poll before entering the usual blocking poll(2) syscall. Try setting this > variable to the time from old request completion to new virtqueue kick. > > By default no polling is done. The QEMU_AIO_POLL_MAX_NS must be set to get any > polling! > > Karl: I hope you can try this patch series with several QEMU_AIO_POLL_MAX_NS > values. If you don't find a good value we should double-check the tracing data > to see if this experimental code can be improved. > > Stefan Hajnoczi (3): > aio-posix: add aio_set_poll_handler() > virtio: poll virtqueues for new buffers > linux-aio: poll ring for completions > > aio-posix.c | 133 ++++++++++++++++++++++++++++++++++++++++++++++++++++ > block/linux-aio.c | 17 +++++++ > hw/virtio/virtio.c | 19 ++++++++ > include/block/aio.h | 16 +++++++ > 4 files changed, 185 insertions(+) Hmm, I see all affected threads using more CPU power, but the performance numbers are somewhat inconclusive on s390. I have no proper test setup (only a shared LPAR), but all numbers are in the same ballpark of 3-5Gbyte/sec for 5 disks for 4k random reads with iodepth=8. What I find interesting is that the guest still does a huge amount of exits for the guest->host notifications. I think if we could combine this with some notification suppression, then things could be even more interesting. Christian