From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36359) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XQZ3d-0007ud-ST for qemu-devel@nongnu.org; Sun, 07 Sep 2014 05:47:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XQZ3U-00076t-Oa for qemu-devel@nongnu.org; Sun, 07 Sep 2014 05:46:53 -0400 Received: from mail-pd0-x233.google.com ([2607:f8b0:400e:c02::233]:55372) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XQZ3U-00076n-CM for qemu-devel@nongnu.org; Sun, 07 Sep 2014 05:46:44 -0400 Received: by mail-pd0-f179.google.com with SMTP id g10so5969093pdj.10 for ; Sun, 07 Sep 2014 02:46:42 -0700 (PDT) Message-ID: <540C2972.5070707@gmail.com> Date: Sun, 07 Sep 2014 17:46:26 +0800 From: Zhang Haoyu MIME-Version: 1.0 References: <201408291545282753855@sangfor.com>, <20140829143849.GA8909@grmbl.mre>, <201409012038178763909@sangfor.com>, <201409012052442706397@sangfor.com>, <5404701F.5070302@de.ibm.com>, <540470CD.9060800@redhat.com>, <5404731A.8020405@de.ibm.com> <201409041556259426013@sangfor.com> In-Reply-To: <201409041556259426013@sangfor.com> Content-Type: text/plain; charset=GB2312 Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [question] virtio-blk performance degradationhappenedwith virito-serial List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Zhang Haoyu , Paolo Bonzini , Christian Borntraeger , Amit Shah Cc: qemu-devel , kvm Hi, Paolo, Amit, any ideas? Thanks, Zhang Haoyu On 2014-9-4 15:56, Zhang Haoyu wrote: >>>>> If virtio-blk and virtio-serial share an IRQ, the guest operating s= ystem has to check each virtqueue for activity. Maybe there is some ineff= iciency doing that. >>>>> AFAIK virtio-serial registers 64 virtqueues (on 31 ports + console)= even if everything is unused. >>>> That could be the case if MSI is disabled. >>> Do the windows virtio drivers enable MSIs, in their inf file? >> It depends on the version of the drivers, but it is a reasonable guess= >> at what differs between Linux and Windows. Haoyu, can you give us the= >> output of lspci from a Linux guest? >> > I made a test with fio on rhel-6.5 guest, the same degradation happened= too, this degradation can be reproduced on rhel6.5 guest 100%. > virtio_console module installed: > 64K-write-sequence: 285 MBPS, 4380 IOPS > virtio_console module uninstalled: > 64K-write-sequence: 370 MBPS, 5670 IOPS > > And, virio-blk's interrupt mode always is MSI, no matter if virtio_cons= ole module is installed or uninstalled. > 25: 2245933 PCI-MSI-edge virtio1-requests > > fio command: > fio -filename /dev/vda -direct=3D1 -iodepth=3D1 -thread -rw=3Dwrite -io= engine=3Dpsync -bs=3D64k -size=3D30G -numjobs=3D1 -name=3Dmytest > > QEMU comamnd: > /usr/bin/kvm -id 5497356709352 -chardev socket,id=3Dqmp,path=3D/var/run= /qemu-server/5497356709352.qmp,server,nowait -mon chardev=3Dqmp,mode=3Dco= ntrol -vnc :0,websocket,to=3D200 -enable-kvm -pidfile /var/run/qemu-serve= r/5497356709352.pid -daemonize -name io-test-rhel-6.5 -smp sockets=3D1,co= res=3D1 -cpu core2duo -nodefaults -vga cirrus -no-hpet -k en-us -boot men= u=3Don,splash-time=3D8000 -m 4096 -usb -drive file=3D/sf/data/local/zhang= haoyu/rhel-server-6.5-x86_64-dvd.iso,if=3Dnone,id=3Ddrive-ide0,media=3Dcd= rom,aio=3Dnative,forecast=3Ddisable -device ide-cd,bus=3Dide.0,unit=3D0,d= rive=3Ddrive-ide0,id=3Dide0,bootindex=3D200 -drive file=3D/sf/data/local/= images/host-1051721dff13/io-test-rhel-6.5.vm/vm-disk-1.qcow2,if=3Dnone,id= =3Ddrive-virtio1,cache=3Dnone,aio=3Dnative -device virtio-blk-pci,drive=3D= drive-virtio1,id=3Dvirtio1,bus=3Dpci.0,addr=3D0xb -drive file=3D/sf/data/= local/images/host-1051721dff13/io-test-rhel-6.5.vm/vm-disk-2.qcow2,if=3Dn= one,id=3Ddrive-virtio2,cache=3Dnone,aio=3Dnative -device virtio-blk-pci,d= rive=3Ddrive-virtio2,id=3Dvirtio2,bus=3Dpci > .0,addr=3D0xc,bootindex=3D101 -netdev type=3Dtap,id=3Dnet0,ifname=3D16= 4922379979200,script=3D/sf/etc/kvm/vtp-bridge,vhost=3Don,vhostforce=3Don = -device virtio-net-pci,mac=3DFE:FC:FE:C6:47:F6,netdev=3Dnet0,bus=3Dpci.0,= addr=3D0x12,id=3Dnet0,bootindex=3D300 -rtc driftfix=3Dslew,clock=3Drt -gl= obal kvm-pit.lost_tick_policy=3Ddiscard -global PIIX4_PM.disable_s3=3D1 -= global PIIX4_PM.disable_s4=3D1 -chardev socket,path=3D/run/virtser/164922= 3799792.sock,server,nowait,id=3Dchannelser -device virtio-serial,vectors=3D= 4 -device virtserialport,chardev=3Dchannelser,name=3Dchannelser.virtseria= l0.0 > > [environment] > Host:linux-3.10(RHEL7-rc1) > QEMU: qemu-2.0.1 > Guest: RHEL6.5 > > # lspci -tv > -[0000:00]-+-00.0 Intel Corporation 440FX - 82441FX PMC [Natoma] > +-01.0 Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton I= I] > +-01.1 Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton I= I] > +-01.2 Intel Corporation 82371SB PIIX3 USB [Natoma/Triton I= I] > +-01.3 Intel Corporation 82371AB/EB/MB PIIX4 ACPI > +-02.0 Cirrus Logic GD 5446 > +-03.0 Red Hat, Inc Virtio console > +-0b.0 Red Hat, Inc Virtio block device > +-0c.0 Red Hat, Inc Virtio block device > \-12.0 Red Hat, Inc Virtio network device > > # lspci -vvv > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (re= v 02) > Subsystem: Red Hat, Inc Qemu virtual machine > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- Par= Err- Stepping- SERR+ FastB2B- DisINTx- > Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort= - SERR- > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton = II] > Subsystem: Red Hat, Inc Qemu virtual machine > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- Par= Err- Stepping- SERR+ FastB2B- DisINTx- > Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dmedium >TAbo= rt- SERR- > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Trit= on II] (prog-if 80 [Master]) > Subsystem: Red Hat, Inc Qemu virtual machine > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- Par= Err- Stepping- SERR+ FastB2B- DisINTx- > Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=3Dmedium >TAbo= rt- SERR- Latency: 0 > Region 0: [virtual] Memory at 000001f0 (32-bit, non-prefetchabl= e) [size=3D8] > Region 1: [virtual] Memory at 000003f0 (type 3, non-prefetchabl= e) > Region 2: [virtual] Memory at 00000170 (32-bit, non-prefetchabl= e) [size=3D8] > Region 3: [virtual] Memory at 00000370 (type 3, non-prefetchabl= e) > Region 4: I/O ports at c0e0 [size=3D16] > Kernel driver in use: ata_piix > Kernel modules: ata_generic, pata_acpi, ata_piix > > 00:01.2 USB controller: Intel Corporation 82371SB PIIX3 USB [Natoma/Tri= ton II] (rev 01) (prog-if 00 [UHCI]) > Subsystem: Red Hat, Inc Qemu virtual machine > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- Par= Err- Stepping- SERR+ FastB2B- DisINTx- > Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort= - SERR- Latency: 0 > Interrupt: pin D routed to IRQ 11 > Region 4: I/O ports at c080 [size=3D32] > Kernel driver in use: uhci_hcd > > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) > Subsystem: Red Hat, Inc Qemu virtual machine > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- Par= Err- Stepping- SERR+ FastB2B- DisINTx- > Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=3Dmedium >TAbo= rt- SERR- Interrupt: pin A routed to IRQ 9 > Kernel driver in use: piix4_smbus > Kernel modules: i2c-piix4 > > 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 (prog-if 00 [VG= A controller]) > Subsystem: Red Hat, Inc Device 1100 > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- Par= Err- Stepping- SERR+ FastB2B- DisINTx- > Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort= - SERR- Region 0: Memory at fc000000 (32-bit, prefetchable) [size=3D32M= ] > Region 1: Memory at febd0000 (32-bit, non-prefetchable) [size=3D= 4K] > Expansion ROM at febc0000 [disabled] [size=3D64K] > Kernel modules: cirrusfb > > 00:03.0 Communication controller: Red Hat, Inc Virtio console > Subsystem: Red Hat, Inc Device 0003 > Physical Slot: 3 > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- Par= Err- Stepping- SERR+ FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort= - SERR- Interrupt: pin A routed to IRQ 10 > Region 0: I/O ports at c0a0 [size=3D32] > Region 1: Memory at febd1000 (32-bit, non-prefetchable) [size=3D= 4K] > Capabilities: [40] MSI-X: Enable- Count=3D4 Masked- > Vector table: BAR=3D1 offset=3D00000000 > PBA: BAR=3D1 offset=3D00000800 > Kernel driver in use: virtio-pci > Kernel modules: virtio_pci > > 00:0b.0 SCSI storage controller: Red Hat, Inc Virtio block device > Subsystem: Red Hat, Inc Device 0002 > Physical Slot: 11 > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- Par= Err- Stepping- SERR+ FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort= - SERR- Interrupt: pin A routed to IRQ 10 > Region 0: I/O ports at c000 [size=3D64] > Region 1: Memory at febd2000 (32-bit, non-prefetchable) [size=3D= 4K] > Capabilities: [40] MSI-X: Enable+ Count=3D2 Masked- > Vector table: BAR=3D1 offset=3D00000000 > PBA: BAR=3D1 offset=3D00000800 > Kernel driver in use: virtio-pci > Kernel modules: virtio_pci > > 00:0c.0 SCSI storage controller: Red Hat, Inc Virtio block device > Subsystem: Red Hat, Inc Device 0002 > Physical Slot: 12 > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- Par= Err- Stepping- SERR+ FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort= - SERR- Interrupt: pin A routed to IRQ 11 > Region 0: I/O ports at c040 [size=3D64] > Region 1: Memory at febd3000 (32-bit, non-prefetchable) [size=3D= 4K] > Capabilities: [40] MSI-X: Enable+ Count=3D2 Masked- > Vector table: BAR=3D1 offset=3D00000000 > PBA: BAR=3D1 offset=3D00000800 > Kernel driver in use: virtio-pci > Kernel modules: virtio_pci > > 00:12.0 Ethernet controller: Red Hat, Inc Virtio network device > Subsystem: Red Hat, Inc Device 0001 > Physical Slot: 18 > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- Par= Err- Stepping- SERR+ FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort= - SERR- Interrupt: pin A routed to IRQ 10 > Region 0: I/O ports at c0c0 [size=3D32] > Region 1: Memory at febd4000 (32-bit, non-prefetchable) [size=3D= 4K] > Expansion ROM at feb80000 [disabled] [size=3D256K] > Capabilities: [40] MSI-X: Enable+ Count=3D3 Masked- > Vector table: BAR=3D1 offset=3D00000000 > PBA: BAR=3D1 offset=3D00000800 > Kernel driver in use: virtio-pci > Kernel modules: virtio_pci > > Thanks, > Zhang Haoyu > >> Paolo > >