From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: virtio_net sometimes didn't work Date: Wed, 16 Feb 2011 10:19:16 +0200 Message-ID: <20110216081916.GB16906@redhat.com> References: <20101125150634.GA17597@redhat.com> <20101209132246.GA22460@redhat.com> <20110216051036.GB10461@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Avi Kivity , kvm@vger.kernel.org To: lidong chen Return-path: Received: from mx1.redhat.com ([209.132.183.28]:34032 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755742Ab1BPITp (ORCPT ); Wed, 16 Feb 2011 03:19:45 -0500 Content-Disposition: inline In-Reply-To: Sender: kvm-owner@vger.kernel.org List-ID: On Wed, Feb 16, 2011 at 04:00:15PM +0800, lidong chen wrote: > how to info pci in qemu? just type it at the monitor prompt. > the usb controller also used irq 11, i think the interrupt maybe caus= e by this. > i will modify qemu, ignore -usb,and try again. >=20 > i add some debug code, printk the ioaddr of vdev. >=20 > static int i; > /* reading the ISR has the effect of also clearing it so it's= very > * important to save off the value. */ > isr =3D ioread8(vp_dev->ioaddr + VIRTIO_PCI_ISR); >=20 > /* It's definitely not us if the ISR was not high */ > if (!isr) { > i++; > if( i=3D=3D10000 ) { > printk(KERN_EMERG "22222"); > printk(KERN_EMERG "ioaddr %p", vp_dev->ioaddr= ); > i=3D0; > } > return IRQ_NONE; > } OK, so we seem to get it right. Try to stick the printout in qemu case VIRTIO_PCI_ISR: /* reading from the ISR also clears it. */ ret =3D vdev->isr; vdev->isr =3D 0; qemu_set_irq(proxy->pci_dev.irq[0], 0); see whether it's the baloon device or the virtio net device that gets to handle the reads. > ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11 > ACPI: PCI Interrupt 0000:00:03.0[A] -> Link [LNKC] -> GSI 11 (level, > high) -> IRQ 11 > 22222<0>ioaddr 0001c040<0>22222<0>ioaddr 0001c040<0>22222<0>ioaddr > 0001c040<0>22222<0>ioaddr 0001c040<0>22222<0>ioaddr > 0001c040<0>22222<0>ioaddr 0001c040<0>22222<0>ioaddr > 0001c040<0>22222<0>ioaddr 0001c040<0>22222<0>ioaddr > 0001c040<0>22222<0>ioaddr 0001c040<3>irq 11: nobody cared (try bootin= g > with the "irqpoll" option) > [] __report_bad_irq+0x2b/0x69 > [] note_interrupt+0x18b/0x1b2 > [] handle_IRQ_event+0x26/0x51 > [] __do_IRQ+0xab/0xdc > [] do_IRQ+0x46/0x53 > [] common_interrupt+0x1a/0x20 > [] __do_softirq+0x4f/0xc2 > [] do_softirq+0x2e/0x32 > [] apic_timer_interrupt+0x1c/0x30 > [] _spin_unlock_irqrestore+0x6/0x7 > [] setup_irq+0xab/0x108 > [] vp_interrupt+0x0/0x114 [virtio_pci] > [] request_irq+0x74/0x90 > [] virtio_pci_probe+0x14c/0x1c2 [virtio_pci] > [] pci_device_probe+0x36/0x57 > [] driver_probe_device+0x42/0x8b > [] __driver_attach+0x4a/0x71 > [] __driver_attach+0x0/0x71 > [] bus_for_each_dev+0x39/0x5b > [] driver_attach+0x11/0x13 > [] __driver_attach+0x0/0x71 > [] bus_add_driver+0x64/0xfd > [] __pci_register_driver+0x6c/0x8e > [] virtio_pci_init+0x20/0x34 [virtio_pci] > [] sys_init_module+0x1749/0x18c2 > [] generic_file_read+0x9a/0xaf > [] vfs_read+0xa8/0x150 > [] sysenter_past_esp+0x54/0x79 > handlers: > [] (vp_interrupt+0x0/0x114 [virtio_pci]) > Disabling IRQ #11 >=20 >=20 > the output of lspci -vv > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (= rev 02) > Subsystem: Unknown device 1af4:1100 > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- > Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort- > SERR- =20 > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Trito= n II] > Subsystem: Unknown device 1af4:1100 > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- > Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dmedium >TAbort- > SERR- Latency: 0 >=20 > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE > [Natoma/Triton II] (prog-if 80 [Master]) > Subsystem: Unknown device 1af4:1100 > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- > Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=3Dmedium >TAbort- > SERR- Latency: 0 > Region 4: I/O ports at c000 [size=3D16] >=20 > 00:01.2 USB Controller: Intel Corporation 82371SB PIIX3 USB > [Natoma/Triton II] (rev 01) (prog-if 00 [UHCI]) > Subsystem: Unknown device 1af4:1100 > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- > Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort- > SERR- Latency: 0 > Interrupt: pin D routed to IRQ 11 > Region 4: I/O ports at c020 [size=3D32] >=20 > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03) > Subsystem: Unknown device 1af4:1100 > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- > Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=3Dmedium >TAbort- > SERR- Interrupt: pin A routed to IRQ 9 >=20 > 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 (prog-if 00 > [VGA controller]) > Subsystem: Unknown device 1af4:1100 > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- > Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort- > SERR- Region 0: Memory at f0000000 (32-bit, prefetchable) [size=3D32M] > Region 1: Memory at f2000000 (32-bit, non-prefetchable) [size=3D4K] > Expansion ROM at f2010000 [disabled] [size=3D64K] >=20 > 00:03.0 RAM memory: Unknown device 1af4:1002 > Subsystem: Unknown device 1af4:0005 > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- > Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort- > SERR- Interrupt: pin A routed to IRQ 11 > Region 0: I/O ports at c040 [size=3D32] >=20 > 00:04.0 Ethernet controller: Unknown device 1af4:1000 > Subsystem: Unknown device 1af4:0001 > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort- > SERR- Latency: 0 > Interrupt: pin A routed to IRQ 10 > Region 0: I/O ports at c060 [size=3D32] > Region 1: Memory at f2020000 (32-bit, non-prefetchable) [size=3D4K] > Expansion ROM at f2030000 [disabled] [size=3D64K] > Capabilities: [40] MSI-X: Enable- Mask- TabSize=3D3 > Vector table: BAR=3D1 offset=3D00000000 > PBA: BAR=3D1 offset=3D00000800 >=20 > 00:05.0 Ethernet controller: Unknown device 1af4:1000 > Subsystem: Unknown device 1af4:0001 > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort- > SERR- Interrupt: pin A routed to IRQ 10 > Region 0: I/O ports at c080 [size=3D32] > Region 1: Memory at f2040000 (32-bit, non-prefetchable) [size=3D4K] > Expansion ROM at f2050000 [disabled] [size=3D64K] > Capabilities: [40] MSI-X: Enable- Mask- TabSize=3D3 > Vector table: BAR=3D1 offset=3D00000000 > PBA: BAR=3D1 offset=3D00000800 >=20 > 00:06.0 Ethernet controller: Unknown device 1af4:1000 > Subsystem: Unknown device 1af4:0001 > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort- > SERR- Interrupt: pin A routed to IRQ 11 > Region 0: I/O ports at c0a0 [size=3D32] > Region 1: Memory at f2060000 (32-bit, non-prefetchable) [size=3D4K] > Expansion ROM at f2070000 [disabled] [size=3D64K] > Capabilities: [40] MSI-X: Enable- Mask- TabSize=3D3 > Vector table: BAR=3D1 offset=3D00000000 > PBA: BAR=3D1 offset=3D00000800 >=20 > 00:07.0 Ethernet controller: Unknown device 1af4:1000 > Subsystem: Unknown device 1af4:0001 > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort- > SERR- Interrupt: pin A routed to IRQ 11 > Region 0: I/O ports at c0c0 [size=3D32] > Region 1: Memory at f2080000 (32-bit, non-prefetchable) [size=3D4K] > Expansion ROM at f2090000 [disabled] [size=3D64K] > Capabilities: [40] MSI-X: Enable- Mask- TabSize=3D3 > Vector table: BAR=3D1 offset=3D00000000 > PBA: BAR=3D1 offset=3D00000800 >=20 > 00:08.0 Ethernet controller: Unknown device 1af4:1000 > Subsystem: Unknown device 1af4:0001 > Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=3Dfast >TAbort- > SERR- Interrupt: pin A routed to IRQ 10 > Region 0: I/O ports at c0e0 [size=3D32] > Region 1: Memory at f20a0000 (32-bit, non-prefetchable) [size=3D4K] > Expansion ROM at f20b0000 [disabled] [size=3D64K] > Capabilities: [40] MSI-X: Enable- Mask- TabSize=3D3 > Vector table: BAR=3D1 offset=3D00000000 > PBA: BAR=3D1 offset=3D00000800 >=20 > 2011/2/16 Michael S. Tsirkin : > > On Wed, Feb 16, 2011 at 09:53:25AM +0800, lidong chen wrote: > >> because of some other work, i could not focus on this problem last= month. > >> > >> now i find the wrong vaule of isr cause this problem. > >> in function vp_interrupt, the isr is 0, and the virtio_balloon pci > >> device have 100000 times initerrupts. then IRQ #11 disabled. > >> > >> static irqreturn_t vp_interrupt(int irq, void *opaque) > >> { > >> =A0 =A0 =A0 struct virtio_pci_device *vp_dev =3D opaque; > >> =A0 =A0 =A0 struct virtio_pci_vq_info *info; > >> =A0 =A0 =A0 irqreturn_t ret =3D IRQ_NONE; > >> =A0 =A0 =A0 unsigned long flags; > >> =A0 =A0 =A0 u8 isr; > >> > >> =A0 =A0 =A0 /* reading the ISR has the effect of also clearing it = so it's very > >> =A0 =A0 =A0 =A0* important to save off the value. */ > >> =A0 =A0 =A0 isr =3D ioread8(vp_dev->ioaddr + VIRTIO_PCI_ISR); > >> > >> =A0 =A0 =A0 /* It's definitely not us if the ISR was not high */ > >> =A0 =A0 =A0 if (!isr) > >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 return IRQ_NONE; =A0 //return from her= e > >> > > > > This implies that io addr values got swapped between the > > devices. Try lspci -vv in guest and info pci in qemu and compare > > the io address values. > > > >> > >> > >> 2010/12/9 Michael S. Tsirkin : > >> > On Fri, Nov 26, 2010 at 10:38:33AM +0800, lidong chen wrote: > >> >> Does this message appear on boot, or after some stress? > >> >> on boot, and only appear when boot from network. > >> >> > >> >> Which qemu-kvm version? > >> >> [root@kvm-4slot ~]# /usr/libexec/qemu-kvm --version > >> >> QEMU PC emulator version 0.12.1 (qemu-kvm-0.12.1.2), Copyright = (c) > >> >> 2003-2008 Fabrice Bellard > >> > > >> > what happens with the latest qemu? > >> > > >> >> Does cherry-picking 3fff0179e33cd7d0a688dab65700c46ad089e934 he= lp? > >> >> the virtio_pci have already used this patch, still have this pr= oblem. > >> >> > >> >> What does info irqs show in qemu? > >> >> how to collect this information? > >> >> > >> >> but I found if modify the slot number of balloon device from 0x= 03 to > >> >> 0x09, the problem solved. > >> >> > >> >> =A0 =A0 > >> >> =A0 =A0 =A0 > >> >> =A0 =A0 =A0
>> >> function=3D'0x0'/> > >> >> =A0 =A0 > >> > > >> > Interesting. Is it possible that even after baloon is moved, > >> > there's still the message in guest, only this time > >> > things keep going afterwards? > >> > > >> >> and i found someone else also meet this problem. > >> >> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/584675 > >> >> > >> >> > >> >> > >> >> 2010/11/25 Michael S. Tsirkin : > >> >> > On Thu, Nov 25, 2010 at 10:21:24PM +0800, lidong chen wrote: > >> >> >> [version] > >> >> >> the host os version is 2.6.32 > >> >> > > >> >> > Which qemu-kvm version? > >> >> > > >> >> >> the guest os version is 2.6.16 > >> >> >> > >> >> >> [dmesg] > >> >> >> ACPI: (supports S3 S4 S5) > >> >> >> Freeing unused kernel memory: 200k freed > >> >> >> input: ImExPS/2 Generic Explorer Mouse as /class/input/input= 2 > >> >> >> ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11 > >> >> >> ACPI: PCI Interrupt 0000:00:03.0[A] -> Link [LNKC] -> GSI 11= (level, > >> >> >> high) -> IRQ 11 > >> >> >> io address 0001c040ACPI: PCI Interrupt Link [LNKD] enabled a= t IRQ 10 > >> >> >> ACPI: PCI Interrupt 0000:00:04.0[A] -> Link [LNKD] -> GSI 10= (level, > >> >> >> high) -> IRQ 10 > >> >> >> io address 0001c060ACPI: PCI Interrupt Link [LNKA] enabled a= t IRQ 10 > >> >> >> ACPI: PCI Interrupt 0000:00:05.0[A] -> Link [LNKA] -> GSI 10= (level, > >> >> >> high) -> IRQ 10 > >> >> >> io address 0001c080ACPI: PCI Interrupt Link [LNKB] enabled a= t IRQ 11 > >> >> >> ACPI: PCI Interrupt 0000:00:06.0[A] -> Link [LNKB] -> GSI 11= (level, > >> >> >> high) -> IRQ 11 > >> >> >> io address 0001c0a0<6>ACPI: PCI Interrupt 0000:00:07.0[A] ->= Link > >> >> >> [LNKC] -> GSI 11 (level, high) -> IRQ 11 > >> >> >> io address 0001c0c0<6>ACPI: PCI Interrupt 0000:00:08.0[A] ->= Link > >> >> >> [LNKD] -> GSI 10 (level, high) -> IRQ 10 > >> >> >> io address 0001c0e0 > >> >> >> irq 11: nobody cared (try booting with the "irqpoll" option) > >> >> >> =A0[] __report_bad_irq+0x2b/0x69 > >> >> >> =A0[] note_interrupt+0x18b/0x1b2 > >> >> >> =A0[] handle_IRQ_event+0x26/0x51 > >> >> >> =A0[] __do_IRQ+0xab/0xdc > >> >> >> =A0[] do_IRQ+0x46/0x53 > >> >> >> =A0[] common_interrupt+0x1a/0x20 > >> >> >> =A0[] __do_softirq+0x4f/0xc2 > >> >> >> =A0[] do_softirq+0x2e/0x32 > >> >> >> =A0[] apic_timer_interrupt+0x1c/0x30 > >> >> >> =A0[] default_idle+0x2e/0x5c > >> >> >> =A0[] cpu_idle+0x91/0xad > >> >> >> =A0[] start_kernel+0x34c/0x353 > >> >> >> handlers: > >> >> >> [] (vp_interrupt+0x0/0x3e [virtio_pci]) > >> >> >> Disabling IRQ #11 > >> >> > > >> >> > Does this message appear on boot, or after some stress? > >> >> > Does cherry-picking 3fff0179e33cd7d0a688dab65700c46ad089e934 = help? > >> >> > Happens with a newer kernel as guest? > >> >> > What does info irqs show in qemu? > >> >> > > >> >> > > >> >> > -- > >> >> > MST > >> >> > -- > >> >> > To unsubscribe from this list: send the line "unsubscribe kvm= " in > >> >> > the body of a message to majordomo@vger.kernel.org > >> >> > More majordomo info at =A0http://vger.kernel.org/majordomo-in= fo.html > >> >> > > >> > > >