From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Williamson Subject: Re: [PATCH] kvm:virtio-net: Run TX from the I/O thread Date: Thu, 22 Jan 2009 11:36:08 -0700 Message-ID: <1232649368.20605.52.camel@bling> References: <20090121230642.10404.65372.stgit@kvm.aw> <497862A7.8040408@redhat.com> <1232628508.4533.16.camel@blaa> Mime-Version: 1.0 Content-Type: text/plain Content-Transfer-Encoding: 7bit Cc: Avi Kivity , kvm@vger.kernel.org To: Mark McLoughlin Return-path: Received: from g5t0006.atlanta.hp.com ([15.192.0.43]:44313 "EHLO g5t0006.atlanta.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753237AbZAVSgO (ORCPT ); Thu, 22 Jan 2009 13:36:14 -0500 In-Reply-To: <1232628508.4533.16.camel@blaa> Sender: kvm-owner@vger.kernel.org List-ID: On Thu, 2009-01-22 at 12:48 +0000, Mark McLoughlin wrote: > On Thu, 2009-01-22 at 14:12 +0200, Avi Kivity wrote: > > > My worry with this change is that increases cpu utilization even more > > than it increases bandwidth, so that our bits/cycle measure decreases. > > Yep, agreed it's important to watch out for this. > > > The descriptors (and perhaps data) are likely on the same cache as the > > vcpu, and moving the transmit to the iothread will cause them to move to > > the iothread's cache. > > We flush from the I/O thread right now. > > We only ever flush from the vcpu thread when the ring fills up, which > rarely happens from what I've observed. Sorry to have come in late to the discussion, but it seems like maybe it needed another kick after a couple months anyway. As noted, we are mostly (almost exclusively?) doing TX from the timer anyway, so this change or Mark's previous patch series don't really change current cache effects. I am curious what happens to latency with Mark's series since that isn't really addressed by the charts, hopefully good things without the tx_timer. A thread per device or perhaps even a thread per RX/TX stream seems like a logical goal, but these current patches do provide a worthwhile incremental improvement. Perhaps we could affinitize the guest to do I/O on a specific vcpu via _PXM methods in ACPI so we can provide hints to the scheduler to keep a vcpu thread and it's associated I/O threads nearby. Thanks, Alex -- Alex Williamson HP Open Source & Linux Org.