From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=54298 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OLhH6-0006CP-7v for qemu-devel@nongnu.org; Mon, 07 Jun 2010 14:42:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OLhH4-0001I9-RY for qemu-devel@nongnu.org; Mon, 07 Jun 2010 14:42:16 -0400 Received: from mx1.redhat.com ([209.132.183.28]:14878) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OLhH4-0001I3-Kt for qemu-devel@nongnu.org; Mon, 07 Jun 2010 14:42:14 -0400 Message-ID: <4C0D3D80.5060208@redhat.com> Date: Mon, 07 Jun 2010 21:42:08 +0300 From: Avi Kivity MIME-Version: 1.0 References: <4C0D0FB7.80709@redhat.com> <4C0D26B5.9030708@codemonkey.ws> In-Reply-To: <4C0D26B5.9030708@codemonkey.ws> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: [Qemu-devel] Re: [RFC] Moving the kvm ioapic, pic, and pit back to userspace List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Anthony Liguori Cc: qemu-devel , KVM list On 06/07/2010 08:04 PM, Anthony Liguori wrote: > > I think we could also move the local APIC. I'm not even sure we can safely move the ioapic/pic (mostly due to churn). But the local APIC is so heavily accessed by the guest that it's impossible to move it. Run an ftrace one day, especially on an smp guest. Every IPI requires several APIC accesses. Before a halt a tickless kernel sets the wakeup timer. EOIs. > > To optimize device models, we've tended to put the full device model > in the kernel whereas the hardware vendors have tended to put only the > fast paths of the devices models in hardware. > > For instance, we could introduce a userspace interface similar to > vapic support whereas a shared page that mapped the APIC's layout was > used with a mask to select which registers trapped on read/write. That leads to very problematic interfaces. When you separate along a device boundary, you have a spec that defines the software interfaces. When you separate along a boundary that you define, it's up to you to get everything right. In fact with the ioapic/pic/lapic one of the problems is that the interconnection between the devices that is not well defined, and that's where we have bugs. > > That said, I can understand an argument that the local APIC is part of > the CPU state since it's a very special type of device. > > A better example would be a generic counter kernel mechanism. I can > envision such a device as doing nothing more than providing a > read-only view of a counter with a userspace configurable divider and > width. Any write to the counter or read of any other byte outside the > counter register would result in a trap to userspace. What about latches? byte access to word registers? There will be as many special cases as there are timers. If the kernel supported a bytecode/jit facility I'd happily use that to download portions of the device model into the kernel. > > That should allow both the PIT and the HPET to be accelerated with > minimal effort in the kernel. IMO it's probably more effort than porting HPET to the kernel. Try outlining an interface that supports PIT, HPET, RTC, and ACPI PMTIMER. > > I'd be in favor of a straight port to userspace. We already have the > interfaces to communicate with an external device model for these > devices so let's just take the kernel code and stick it into dedicated > threads in userspace. Currently we support an all-or-nothing approach. I don't think local APIC in userspace is worthwhile. Esp. as it will slow down vhost and assigned devices significantly - interrupts will have to be mediated by userspace. > > I think it's easier to then work to merge the two bits of code in the > same tree than it is to try and take out-of-tree code and merge it > incrementally. Are you talking about qemu.git/qemu-kvm.git? That's the least of my concerns, I'm worried about kvm.git. > >> 5. Risk >> >> We may find out after all this is implemented that performance is not >> acceptable and all the work will have to be dropped. > > That's another advantage to a straight port to userspace. We can > collect performance data with only a modest amount of engineering effort. Port what exactly? We have a userspace irqchip implementation. What we don't have is just the ioapic/pic/pit in userspace, and the only way to try it out is to implement the whole thing. -- I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain.