From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:43277) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TldnI-0000TL-0n for qemu-devel@nongnu.org; Thu, 20 Dec 2012 05:56:12 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TldnA-0004Hf-HV for qemu-devel@nongnu.org; Thu, 20 Dec 2012 05:56:03 -0500 Received: from indium.canonical.com ([91.189.90.7]:35831) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TldnA-0004HX-BK for qemu-devel@nongnu.org; Thu, 20 Dec 2012 05:55:56 -0500 Received: from loganberry.canonical.com ([91.189.90.37]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1Tldn7-0002o3-Ft for ; Thu, 20 Dec 2012 10:55:53 +0000 Received: from loganberry.canonical.com (localhost [127.0.0.1]) by loganberry.canonical.com (Postfix) with ESMTP id 741D12E807F for ; Thu, 20 Dec 2012 10:55:53 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Date: Thu, 20 Dec 2012 10:46:11 -0000 From: Vlastimil Holer <1091766@bugs.launchpad.net> Sender: bounces@canonical.com References: <20121218171248.25658.59752.malonedeb@gac.canonical.com> <20121219224628.2558.12785.malone@chaenomeles.canonical.com> Message-Id: Errors-To: bounces@canonical.com Subject: Re: [Qemu-devel] [Bug 1091766] Re: Physical host crash with Mellanox IB PCI passthrough Reply-To: Bug 1091766 <1091766@bugs.launchpad.net> List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org On Wed, Dec 19, 2012 at 5:46 PM, Alex Williamson wrote: > You're right, I'm thinking of MSI where lspci reports x/y vectors. The > only way I know to get this is to grep /proc/interrupts on host and > guest. Look for kvm or vfio in the host and likely some device specific > identifier in the guest. You were absolutely right with vector counts. With VFIO I can see 4 and 4 vectors in host and guest and with legacy PCI assign. (on qemu-1.0.1) I can see 30 in host and 4 in guest. On Wed, Dec 19, 2012 at 11:46 PM, Alex Williamson wrote: > Here's another version of the legacy pci-assign patch. This should also > only use 4 vectors on the host, like vfio. I'm a little uneasy about > setting up an MSIMessage with unknown data, but I guess we did it for a > long time previously. Please test. Thanks I have physical machine in production back again, so it'll take few days to test your new patch. I'll let you know. -- = You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1091766 Title: Physical host crash with Mellanox IB PCI passthrough Status in QEMU: New Bug description: (from http://permalink.gmane.org/gmane.comp.emulators.kvm.devel/100736) We have been using PCI passthrough with the Mellanox IB interface (MT27500 Family [ConnectX-3]) on Debian 6.0.6, kernel 3.2.23 and qemu-kvm-1.0 (both from backports). It worked fine until latest update in backports to qemu-kvm-1.1.2. With newer qemu-kvm versions IB device probe in guest fails leaving firmware to kill whole physical ma= chine. I have then compiled qemu-kvm from source, 1.0.1 was OK, 1.1.2 fails and even 1.2.0 fails as well. Our setup is based on IBM System X iDataPlex dx360 M4 Server. Note: Now I have also tested latest qemu-1.3.0 with linux 3.7.1 and new VFIO mechanism and behaves the same way. On guest the mlx4_core fails to probe device: | mlx4_core 0000:00:08.0: irq 74 for MSI/MSI-X | mlx4_core 0000:00:08.0: irq 75 for MSI/MSI-X | mlx4_core 0000:00:08.0: irq 76 for MSI/MSI-X | mlx4_core 0000:00:08.0: irq 77 for MSI/MSI-X | mlx4_core 0000:00:08.0: NOP command failed to generate MSI-X interrupt = IRQ 51). | mlx4_core 0000:00:08.0: Trying again without MSI-X. | mlx4_core 0000:00:08.0: NOP command failed to generate interrupt (IRQ 5= 1), aborting. | mlx4_core 0000:00:08.0: BIOS or ACPI interrupt routing problem? | mlx4_core 0000:00:08.0: PCI INT A disabled | mlx4_core: probe of 0000:00:08.0 failed with error -16 Which immediately results in reset of the whole physical machine: | Uhhuh. NMI received for unknown reason 3d on CPU 0. | Do you have a strange power saving mode enabled? | Dazed and confused, but trying to continue Followed by events in hardware management module: | A software NMI has occurred on system "SN# xxxxxxx" | Fault in slot "All PCI Err" on system "SN# xxxxxxx" | Fault in slot "PCI 1" on system "SN# xxxxxxx" | A Uncorrectable Bus Error has occurred on system "SN# xxxxxxx" | "Host Power" has been Power Cycled | System "SN# xxxxxxx" has recovered from an NMI Kernel logs for both host/guest machines and different qemu-kvm versions are attached. PCI passthrough for e.g. Intel e1000 works fine with all tested qemu-kvm versions. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1091766/+subscriptions