From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alex Williamson Subject: Re: VT-d not working for FreeBSD 9.0 guest Date: Wed, 16 May 2012 12:22:09 -0600 Message-ID: <1337192529.6954.249.camel@bling.home> References: Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Cc: kvm@vger.kernel.org To: Shesha Sreenivasamurthy Return-path: Received: from mx1.redhat.com ([209.132.183.28]:31133 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756227Ab2EPSWL (ORCPT ); Wed, 16 May 2012 14:22:11 -0400 In-Reply-To: Sender: kvm-owner@vger.kernel.org List-ID: On Wed, 2012-05-16 at 17:49 +0000, Shesha Sreenivasamurthy wrote: > I'm using FreeBSD 9.0 (FreeBSD freebsd9-i386 9.0-RELEASE FreeBSD 9.0-RELEASE) > as my guest OS with qemu-kvm (qemu-kvm-1.1-rc2/x86_64-softmmu/ > qemu-system-x86_64) running on centos 6.2 (CentOS release 6.2) with > 2.6.32-220.el6.x86_64 kernel. I am using Intel's 82599EB 10-Gigabit hardware > on the host, which I am trying to export into the FreeBSD guest OS. The guest > driver (version 2.4.4) will claim it. However, the status is shown as > "no carrier" an therefore cannot ping any other system. > > $ ifconfig ix0 > ix0: flags=8843 metric 0 mtu 1500 > > options=401bb VLAN_HWCSUM,TSO4,VLAN_HWTSO> > ether 90:e2:ba:0d:3d:b0 > inet 4.4.4.1 netmask 0xffffff00 broadcast 4.4.4.255 > nd6 options=29 > media: Ethernet autoselect > status: no carrier > > On the host side, before the ixgbe is loaded in the guest, /proc/interrupts > show the following entry for kvm > > # cat /proc/interrupts | grep kvm > 30: 0 0 0 0 0 0 > 0 0 0 0 0 0 0 > 0 0 0 IR-IO-APIC-fasteoi kvm_assigned_intx_device > > However, after the driver is loaded, the above entry is gone but two new > entries show up. > > # cat /proc/interrupts | grep kvm > 76: 0 0 0 0 0 0 > 0 0 0 0 0 0 0 > 0 0 0 IR-PCI-MSI-edge kvm_assigned_msix_device > 77: 0 0 0 0 0 0 > 0 0 0 0 0 0 0 > 0 0 0 IR-PCI-MSI-edge kvm_assigned_msix_device. This is expected, the driver enabled MSI-X. If you have an equivalent of lspci, you should be able to see before that MSI/MSI-X is disabled and after MSI-X is enabled with 2 vectors. Do you ever get interrupts on these lines? > To make sure, the my host configurations are fine, I loaded linux guest, with > exact same qemu params and host configurations. It works just fine. > > What I noticed in case linux guest is, to start with same entry is seen in > /proc/interrupts. However, after the ixgbe is loaded in the guest, I see only > one entry instead of two as in case of freebsd. > > # cat /proc/interrupts | grep kvm > 30: 0 0 0 0 0 0 > 0 0 0 0 0 0 0 > 0 0 0 IR-IO-APIC-fasteoi kvm_assigned_intx_device > > # cat /proc/interrupts | grep kvm > 76: 0 0 0 0 0 0 > 0 0 0 0 0 0 0 > 0 0 0 IR-PCI-MSI-edg kvm_assigned_msi_device Linux may be enabling MSI instead of MSI-X, that would be Ok. Check lspci in the guest to see what Linux thinks it's using. > Similar problem was reported earlier too: > http://comments.gmane.org/gmane.comp.emulators.kvm.devel/83806 > > Any ideas or advice will be highly valued Right, we fixed how we do MSI-X vector allocation so we can better track how this driver works. That allows an 82576 (igb) device to work, but there might still be a FreeBSD ixgbe driver issue that's not allowing it to see the link state. Thanks, Alex