From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:39549) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TYJFJ-0007h3-0X for qemu-devel@nongnu.org; Tue, 13 Nov 2012 11:21:56 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TYJFF-0000Cj-Tj for qemu-devel@nongnu.org; Tue, 13 Nov 2012 11:21:52 -0500 Received: from ssl.dlhnet.de ([91.198.192.8]:59476 helo=ssl.dlh.net) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TYJFF-0000Cb-Jh for qemu-devel@nongnu.org; Tue, 13 Nov 2012 11:21:49 -0500 Message-ID: <50A2739E.1060107@dlhnet.de> Date: Tue, 13 Nov 2012 17:21:50 +0100 From: Peter Lieven MIME-Version: 1.0 References: <5098E288.9060505@web.de> <24E144B8C0207547AD09C467A8259F75577EC4B2@lisa.maurer-it.com> <509B7DC4.3010802@web.de> <24E144B8C0207547AD09C467A8259F75577EDCC2@lisa.maurer-it.com> <24E144B8C0207547AD09C467A8259F75577EE707@lisa.maurer-it.com> <6c17c45c1439f217b36df2c78f0a04e6.squirrel@ssl.dlhnet.de> <50A233AF.70107@dlhnet.de> <20121113162256.GA554@redhat.com> In-Reply-To: <20121113162256.GA554@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] slow virtio network with vhost=on and multiple cores List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Michael S. Tsirkin" Cc: Stefan Hajnoczi , Peter Lieven , Jan Kiszka , Dietmar Maurer , "qemu-devel@nongnu.org" On 13.11.2012 17:22, Michael S. Tsirkin wrote: > On Tue, Nov 13, 2012 at 12:49:03PM +0100, Peter Lieven wrote: >> On 09.11.2012 19:03, Peter Lieven wrote: >>> Remark: >>> If i disable interrupts on CPU1-3 for virtio the performance is ok again. >>> >>> Now we need someone with deeper knowledge of the in-kernel irqchip and the >>> virtio/vhost driver development to say if this is a regression in qemu-kvm >>> or a problem with the old virtio drivers if they receive the interrupt on >>> different CPUs. >> anyone? > Looks like the problem is not in the guest: I tried ubuntu guest > on a rhel host, I got 8GB/s with vhost and 4GB/s without > on a host to guest banchmark. which ubuntu version in the guest? Peter > > > >>> Peter Lieven wrote: >>>> it seems that with in-kernel irqchip the interrupts are distributed across >>>> all vpcus. without in-kernel irqchip all interrupts are on cpu0. maybe >>>> this is related. >>>> >>>> without inkernel irqchip >>>> CPU0 CPU1 CPU2 CPU3 >>>> 0: 16 0 0 0 IO-APIC-edge timer >>>> 1: 23 0 0 0 IO-APIC-edge i8042 >>>> 4: 1 0 0 0 IO-APIC-edge >>>> 6: 4 0 0 0 IO-APIC-edge >>>> floppy >>>> 7: 0 0 0 0 IO-APIC-edge >>>> parport0 >>>> 8: 0 0 0 0 IO-APIC-edge rtc0 >>>> 9: 0 0 0 0 IO-APIC-fasteoi acpi >>>> 11: 76 0 0 0 IO-APIC-fasteoi >>>> uhci_hcd:usb1 >>>> 12: 102 0 0 0 IO-APIC-edge i8042 >>>> 14: 0 0 0 0 IO-APIC-edge >>>> ata_piix >>>> 15: 16881 0 0 0 IO-APIC-edge >>>> ata_piix >>>> 24: 0 0 0 0 PCI-MSI-edge >>>> virtio1-config >>>> 25: 5225 0 0 0 PCI-MSI-edge >>>> virtio1-requests >>>> 26: 0 0 0 0 PCI-MSI-edge >>>> virtio0-config >>>> 27: 72493 0 0 0 PCI-MSI-edge >>>> virtio0-input >>>> ... >>>> >>>> with inkernel irqchip >>>> CPU0 CPU1 CPU2 CPU3 >>>> 0: 16 0 0 0 IO-APIC-edge timer >>>> 1: 0 3 3 1 IO-APIC-edge i8042 >>>> 4: 0 0 1 0 IO-APIC-edge >>>> 6: 1 0 1 2 IO-APIC-edge >>>> floppy >>>> 7: 0 0 0 0 IO-APIC-edge >>>> parport0 >>>> 8: 0 0 0 0 IO-APIC-edge rtc0 >>>> 9: 0 0 0 0 IO-APIC-fasteoi acpi >>>> 11: 7 9 4 1 IO-APIC-fasteoi >>>> uhci_hcd:usb1 >>>> 12: 30 27 29 34 IO-APIC-edge i8042 >>>> 14: 0 0 0 0 IO-APIC-edge >>>> ata_piix >>>> 15: 943 937 950 943 IO-APIC-edge >>>> ata_piix >>>> 24: 0 0 0 0 PCI-MSI-edge >>>> virtio0-config >>>> 25: 930 978 980 947 PCI-MSI-edge >>>> virtio0-input >>>> 26: 0 0 1 0 PCI-MSI-edge >>>> virtio0-output >>>> 27: 0 0 0 0 PCI-MSI-edge >>>> virtio1-config >>>> 28: 543 541 542 553 PCI-MSI-edge >>>> virtio1-requests >>>> ... >>>> >>>>