From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sheng Yang Subject: Re: [PATCH 0/8][v2] MSI-X mask emulation support for assigned device Date: Thu, 21 Oct 2010 15:41:20 +0800 Message-ID: <201010211541.20518.sheng@linux.intel.com> References: <1287563192-29685-1-git-send-email-sheng@linux.intel.com> <4CBEBB85.4000706@redhat.com> Mime-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: Marcelo Tosatti , kvm@vger.kernel.org, "Michael S. Tsirkin" , Alex Williamson To: Avi Kivity Return-path: Received: from mga09.intel.com ([134.134.136.24]:53196 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751884Ab0JUHkI (ORCPT ); Thu, 21 Oct 2010 03:40:08 -0400 In-Reply-To: <4CBEBB85.4000706@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: On Wednesday 20 October 2010 17:51:01 Avi Kivity wrote: > On 10/20/2010 10:26 AM, Sheng Yang wrote: > > Here is v2. > > > > Changelog: > > > > v1->v2 > > > > The major change from v1 is I've added the in-kernel MSI-X mask emulation > > support, as well as adding shortcuts for reading MSI-X table. > > > > I've taken Michael's advice to use mask/unmask directly, but unsure about > > exporting irq_to_desc() for module... > > > > Also add flush_work() according to Marcelo's comments. > > Any performance numbers? What are the affected guests? just RHEL 4, or > any others? At least current RHEL5 series would be affected. I have done an simple benchmark on RHEL5u5 guest with 512m memory and 1 cpu. Device is one 10G NIC with SRIOV. One VF was assigned to guest to communicate the PF in the host. 3 threads had been used in iperf of guest to push the CPU utilization to 100%. In this condition, QEmu method's bandwidth is about 20% lower than in-kernel one(~7.5G vs ~9G). Interrupt rate in this condition is about 20k/sec. The reason is 2.6.18 kernel used mask_msi in ack() for MSI chip, caused significant mask bit operations when interrupt rate is high. We have also reproduced the issue on some large scale benchmark on the guest with newer kernel like 2.6.30 on Xen, under very high interrupt rate, due to some interrupt rate limitation mechanism in kernel. > > Alex, Michael, how would you do this with vfio? -- regards Yang, Sheng