From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:39262) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RujkF-0000R1-NP for qemu-devel@nongnu.org; Tue, 07 Feb 2012 07:02:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Rujk7-0004kT-R5 for qemu-devel@nongnu.org; Tue, 07 Feb 2012 07:01:59 -0500 Received: from mx1.redhat.com ([209.132.183.28]:39374) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Rujk7-0004kI-KQ for qemu-devel@nongnu.org; Tue, 07 Feb 2012 07:01:51 -0500 Message-ID: <4F3112A4.3090407@redhat.com> Date: Tue, 07 Feb 2012 14:01:40 +0200 From: Avi Kivity MIME-Version: 1.0 References: <4F2AB552.2070909@redhat.com> <4F2E80A7.5040908@redhat.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC] Next gen kvm api List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Rob Earhart Cc: linux-kernel , KVM list , qemu-devel On 02/06/2012 07:41 PM, Rob Earhart wrote: > >> > >> I like the ioctl() interface. If the overhead matters in your hot path, > > > > I can't say that it's a pressing problem, but it's not negligible. > > > >> I suspect you're doing it wrong; > > > > What am I doing wrong? > > "You the vmm" not "you the KVM maintainer" :-) > > To be a little more precise: If a VCPU thread is going all the way out > to host usermode in its hot path, that's probably a performance > problem regardless of how fast you make the transitions between host > user and host kernel. Why? > That's why ioctl() doesn't bother me. I think it'd be more useful to > focus on mechanisms which don't require the VCPU thread to exit at all > in its hot paths, so the overhead of the ioctl() really becomes lost > in the noise. irq fds and ioevent fds are great for that, and I > really like your MMIO-over-socketpair idea. I like them too, but they're not suitable for all cases. An ioeventfd, or unordered write-over-mmio-socketpair can take one of two paths: - waking up an idle mmio service thread on a different core, involving a double context switch on that remote core - scheduling the idle mmio service thread on the current core, involving both a double context switch and a heavyweight exit An ordered write-over-mmio-socketpair, or a read-over-mmio-socketpair can also take one of two paths - waking up an idle mmio service thread on a different core, involving a double context switch on that remote core, and also invoking two context switches on the current core (while we wait for a reply); if the current core schedules a user task we might also have a heavyweight exit - scheduling the idle mmio service thread on the current core, involving both a double context switch and a heavyweight exit As you can see the actual work is greater for threaded io handlers than the synchronous ones. The real advantage is that you can perform more work in parallel if you have the spare cores (not a given in consolidation environments) and if you actually have a lot of work to do (like virtio-net in a throughput load). It doesn't quite fit a "read hpet register" load. > > >> This would allow for a variety of different programming styles for the > >> VMM--I'm a fan of CSP model myself, but that's hard to do with the > >> current API. > > > > Just convert the synchronous API to an RPC over a pipe, in the vcpu > > thread, and you have the asynchronous model you asked for. > > Yup. But you still get multiple threads in your process. It's not a > disaster, though. > You have multiple threads anyway, even if it's the kernel that creates them. -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain.