From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:39863) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TYJH2-0000FN-E0 for qemu-devel@nongnu.org; Tue, 13 Nov 2012 11:23:43 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TYJGz-0000Sg-Bd for qemu-devel@nongnu.org; Tue, 13 Nov 2012 11:23:40 -0500 Received: from mx1.redhat.com ([209.132.183.28]:64812) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TYJGz-0000SU-3p for qemu-devel@nongnu.org; Tue, 13 Nov 2012 11:23:37 -0500 Date: Tue, 13 Nov 2012 18:26:11 +0200 From: "Michael S. Tsirkin" Message-ID: <20121113162611.GB554@redhat.com> References: <509B7DC4.3010802@web.de> <24E144B8C0207547AD09C467A8259F75577EDCC2@lisa.maurer-it.com> <24E144B8C0207547AD09C467A8259F75577EE707@lisa.maurer-it.com> <6c17c45c1439f217b36df2c78f0a04e6.squirrel@ssl.dlhnet.de> <50A233AF.70107@dlhnet.de> <20121113162256.GA554@redhat.com> <50A2739E.1060107@dlhnet.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <50A2739E.1060107@dlhnet.de> Subject: Re: [Qemu-devel] slow virtio network with vhost=on and multiple cores List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Peter Lieven Cc: Stefan Hajnoczi , Peter Lieven , Jan Kiszka , Dietmar Maurer , "qemu-devel@nongnu.org" On Tue, Nov 13, 2012 at 05:21:50PM +0100, Peter Lieven wrote: > On 13.11.2012 17:22, Michael S. Tsirkin wrote: > >On Tue, Nov 13, 2012 at 12:49:03PM +0100, Peter Lieven wrote: > >>On 09.11.2012 19:03, Peter Lieven wrote: > >>>Remark: > >>>If i disable interrupts on CPU1-3 for virtio the performance is ok again. > >>> > >>>Now we need someone with deeper knowledge of the in-kernel irqchip and the > >>>virtio/vhost driver development to say if this is a regression in qemu-kvm > >>>or a problem with the old virtio drivers if they receive the interrupt on > >>>different CPUs. > >>anyone? > >Looks like the problem is not in the guest: I tried ubuntu guest > >on a rhel host, I got 8GB/s with vhost and 4GB/s without > >on a host to guest banchmark. > which ubuntu version in the guest? > > Peter ubuntu-10.04.4-server-amd64.iso > > > > > > > >>>Peter Lieven wrote: > >>>>it seems that with in-kernel irqchip the interrupts are distributed across > >>>>all vpcus. without in-kernel irqchip all interrupts are on cpu0. maybe > >>>>this is related. > >>>> > >>>>without inkernel irqchip > >>>> CPU0 CPU1 CPU2 CPU3 > >>>> 0: 16 0 0 0 IO-APIC-edge timer > >>>> 1: 23 0 0 0 IO-APIC-edge i8042 > >>>> 4: 1 0 0 0 IO-APIC-edge > >>>> 6: 4 0 0 0 IO-APIC-edge > >>>>floppy > >>>> 7: 0 0 0 0 IO-APIC-edge > >>>>parport0 > >>>> 8: 0 0 0 0 IO-APIC-edge rtc0 > >>>> 9: 0 0 0 0 IO-APIC-fasteoi acpi > >>>> 11: 76 0 0 0 IO-APIC-fasteoi > >>>>uhci_hcd:usb1 > >>>> 12: 102 0 0 0 IO-APIC-edge i8042 > >>>> 14: 0 0 0 0 IO-APIC-edge > >>>>ata_piix > >>>> 15: 16881 0 0 0 IO-APIC-edge > >>>>ata_piix > >>>> 24: 0 0 0 0 PCI-MSI-edge > >>>>virtio1-config > >>>> 25: 5225 0 0 0 PCI-MSI-edge > >>>>virtio1-requests > >>>> 26: 0 0 0 0 PCI-MSI-edge > >>>>virtio0-config > >>>> 27: 72493 0 0 0 PCI-MSI-edge > >>>>virtio0-input > >>>>... > >>>> > >>>>with inkernel irqchip > >>>> CPU0 CPU1 CPU2 CPU3 > >>>> 0: 16 0 0 0 IO-APIC-edge timer > >>>> 1: 0 3 3 1 IO-APIC-edge i8042 > >>>> 4: 0 0 1 0 IO-APIC-edge > >>>> 6: 1 0 1 2 IO-APIC-edge > >>>>floppy > >>>> 7: 0 0 0 0 IO-APIC-edge > >>>>parport0 > >>>> 8: 0 0 0 0 IO-APIC-edge rtc0 > >>>> 9: 0 0 0 0 IO-APIC-fasteoi acpi > >>>> 11: 7 9 4 1 IO-APIC-fasteoi > >>>>uhci_hcd:usb1 > >>>> 12: 30 27 29 34 IO-APIC-edge i8042 > >>>> 14: 0 0 0 0 IO-APIC-edge > >>>>ata_piix > >>>> 15: 943 937 950 943 IO-APIC-edge > >>>>ata_piix > >>>> 24: 0 0 0 0 PCI-MSI-edge > >>>>virtio0-config > >>>> 25: 930 978 980 947 PCI-MSI-edge > >>>>virtio0-input > >>>> 26: 0 0 1 0 PCI-MSI-edge > >>>>virtio0-output > >>>> 27: 0 0 0 0 PCI-MSI-edge > >>>>virtio1-config > >>>> 28: 543 541 542 553 PCI-MSI-edge > >>>>virtio1-requests > >>>>... > >>>> > >>>>