From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: vhost-net patches Date: Tue, 20 Oct 2009 19:33:00 +0200 Message-ID: <20091020173259.GA14639@redhat.com> References: <1255721369.10322.38.camel@w-sridhar.beaverton.ibm.com> <20091018105356.GA8374@redhat.com> <20091018173202.GA32441@redhat.com> <1255993014.15949.167.camel@w-sridhar.beaverton.ibm.com> <20091020113409.GA11444@redhat.com> <20091020171832.GE13378@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: David Stevens , kvm@vger.kernel.org, sri@linux.vnet.ibm.com To: Shirley Ma Return-path: Received: from mx1.redhat.com ([209.132.183.28]:12828 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752546AbZJTRfM (ORCPT ); Tue, 20 Oct 2009 13:35:12 -0400 Content-Disposition: inline In-Reply-To: Sender: kvm-owner@vger.kernel.org List-ID: On Tue, Oct 20, 2009 at 10:27:38AM -0700, Shirley Ma wrote: > > Hello Michael, > > Here are the output, I am using guest 2.6.32-rc3 kernel. It doesn't use > MSIX. So which guest kernel I should use?? > > [xma@localhost ~]$ cat /proc/interrupts. > CPU0 > 0: 299 IO-APIC-edge timer. > 1: 2 IO-APIC-edge i8042. > 2: 0 XT-PIC-XT cascade > 4: 76 IO-APIC-edge serial > 11: 2126 IO-APIC-edge virtio1, virtio0 <--------- here is the > virtio for both disk and network i/o?? Yes, this is regular shared IRQ, no good. I think your guest is too old, please use kernel 2.6.31 and up in guest. I will work to improve the error message as well. > 12: 89 IO-APIC-edge i8042 > NMI: 0 Non-maskable interrupts > LOC: 5146 Local timer interrupts > SPU: 0 Spurious interrupts > CNT: 0 Performance counter interrupts > PND: 0 Performance pending work > RES: 0 Rescheduling interrupts > CAL: 0 Function call interrupts > TLB: 0 TLB shootdowns > TRM: 0 Thermal event interrupts > MCE: 0 Machine check exceptions > MCP: 1 Machine check polls > ERR: 0 > MIS: 0 > [xma@localhost ~]$ uname -r > 2.6.32-rc3 > > > Shirley Ma > IBM Linux Technology Center > 15300 SW Koll Parkway > Beaverton, OR 97006-6063 > Phone(Fax): (503) 578-7638 > > > > > > "Michael S. > Tsirkin" > To > Shirley Ma/Beaverton/IBM@IBMUS > 10/20/2009 10:18 cc > AM David Stevens/Beaverton/IBM@IBMUS, > kvm@vger.kernel.org, > sri@linux.vnet.ibm.com > Subject > Re: vhost-net patches > > > > > > > > > > > On Tue, Oct 20, 2009 at 10:14:55AM -0700, Shirley Ma wrote: > > > > Hello Michael, > > > > >what is vnet->vector? > > >And what do you mean by not defined? > > > > In funcation: > > > > static int vhost_virtqueue_init() > > { > > ...... > > r = vdev->binding->irqfd(vdev->binding_opaque, q->vector, > > vq->call); > > ...... > > }..... > > > > q->vector is 65535, > > Thanks for debugging this. > I think this means that guest does not use MSI-X. > > You can verify this by booting guest without vhost, > and performing the following command: > cat /proc/interrupts > > Please note that you currently need recent kernel in guest, > so that it uses MSI-X. I plan on implementing regular IRQ, > but not yet, and it will be slower anyway. > > > in static int virtio_pci_irqfd() > > { > > ...... > > if (vector >= proxy->pci_dev.msix_entries_nr) { > > fprintf(stderr, " pci irq fd returned vector %d, msix_entries_nr > %d > > \n", > > vector, proxy->pci_dev.msix_entries_nr); <------- I added > > one output line here. > > return -EINVAL; > > .... > > }... > > The output is: > > > > pci irq fd returned vector 65535, msix_entries_nr 3, EINVAL is > > returned. > > > > thanks > > Shirley Ma > > IBM Linux Technology Center > > 15300 SW Koll Parkway > > Beaverton, OR 97006-6063 > > Phone(Fax): (503) 578-7638 > > > > > > > > > > > > "Michael S. > > Tsirkin" > > > To > > Shirley Ma/Beaverton/IBM@IBMUS > > 10/20/2009 04:34 > cc > > AM sri@linux.vnet.ibm.com, David > > Stevens/Beaverton/IBM@IBMUS, > > kvm@vger.kernel.org > > > Subject > > Re: vhost-net patches > > > > > > > > > > > > > > > > > > > > > > On Mon, Oct 19, 2009 at 04:08:24PM -0700, Shirley Ma wrote: > > > Hello Michael, > > > > > > >They all failed with the following error > > > > vhost_net_init returned -7 > > > >This is an error message from hw/virtio-net.c:virtio_net_driver_ok() > > when > > > >vhost_net_start() fails. It looks like dev->binding->irqfd() is > failing > > in > > > >vhost_virtqueue_init(). Haven't yet debugged further. I have > > CONFIG_EVENTFD > > > >enabled in the host kernel. > > > > > > >From the debug output, looks like the vnet->vector is not defined, > > > > what is vnet->vector? > > And what do you mean by not defined? > > > > > and the > > > default msix_entries_nr is 3, so it returned EINVAL from > > virtio_pci_irqfd. > > > Looks we need to either disable QEMU_PCI_CAP_MSIX or define vector in > > QEMU > > > configuration? > > > > You shouldn't have to do anything. > > > > > I am not familiar with MSIX stuffs. > > > > > > Thanks > > > Shirley > > > > > > > > > Inactive hide details for sri---10/19/2009 03:56:57 PM---On Sun, > > 2009-10-18 at > > > 19:32 +0200, Michael S. Tsirkin wrote:sri---10/19/2009 03:56:57 PM---On > > Sun, > > > 2009-10-18 at 19:32 +0200, Michael S. Tsirkin wrote: > > > > > > sri@linux.vnet.ibm.com [cid] * > > > To "Michael S. Tsirkin" > > , > > > 10/19/2009 03:56 PM kvm@vger.kernel.org > > > [cid] * > > > cc David > > Stevens/Beaverton/IBM@IBMUS, Shirley Ma/ > > > Beaverton/IBM@IBMUS > > > [cid] * > > > Subject Re: vhost-net patches > > > * * > > > > > > On Sun, 2009-10-18 at 19:32 +0200, Michael S. Tsirkin wrote: > > > > On Sun, Oct 18, 2009 at 12:53:56PM +0200, Michael S. Tsirkin wrote: > > > > > On Fri, Oct 16, 2009 at 12:29:29PM -0700, Sridhar Samudrala wrote: > > > > > > Hi Michael, > > > > > > > > > > > > We are trying out your vhost-net patches from your git trees on > > > kernel.org. > > > > > > I am using mst/vhost.git as host kernel and mst/qemu-kvm.git for > > qemu. > > > > > > > > > > > > I am using the following qemu script to start the guest using > > userspace > > > tap backend. > > > > > > > > > > > > > > home/sridhar/git/mst/qemu-kvm/x86_64-softmmu/qemu-system-x86_64 /home/ > > > sridhar/kvm_images/fedora10-1-vm -m 512 -drive > > file=/home/sridhar/kvm_images/ > > > fedora10-1-vm,if=virtio,index=0,boot=on -net nic,macaddr= > > > 54:52:00:35:e3:73,model=virtio -net > > tap,ifname=vnet0,script=no,downscript=no > > > > > > > > > > > > Now that i got the default backend to work, i wanted to try vhost > > in > > > kernel. But > > > > > > could not figure out the right -net option to pass to qemu. > > > > > > > > > > > > Can you let me know the right syntax to start a guest using > vhost. > > > > > > > > > > > > Thanks > > > > > > Sridhar > > > > > > > > > > Here's an example with raw socket: > > > > > > > > > > /root/kvm-test/bin/qemu-system-x86_64 -m 1G -kernel \ > > > > > /boot/vmlinuz-$release -append \ > > > > > 'root=UUID=d5d2d201-d086-42ad-bb1d-32fbe40eda71 ro quiet nosplash \ > > > > > console=tty0 console=ttyS0,9600n8' -initrd /boot/guest-initrd.img \ > > > > > $HOME/disk.raw.copy -net raw,ifname=eth3 -net > nic,model=virtio,vhost > > \ > > > > > -balloon none -redir tcp:8023::22 > > > > > > > > > > As you see, I changed the command line. > > > > > You now simply add ",vhost" after model, and it will locate > > > > > host network interface specified earlier and attach to it. > > > > > This should have been clear from running qemu with -help > > > > > flag. Could you please suggest how can that text > > > > > be clarified? > > > > > > I updated to your latest git trees and the default user-space tap > backend > > using > > > the > > > following -net options worked fine. > > > -net tap,ifname=vnet0,script=no,downscript=no -net nic,model=virtio > > > > > > But i could not get vhost to work with either raw or tap backends. > > > I tried the following combinations. > > > 1) -net raw,ifname=eth0 -net nic,model=virtio,vhost > > > 2) -net raw,ifname=vnet0, -net nic,model=virtio,vhost > > > 3) -net tap,ifname=vnet0,script=no,downscript=no -net > > nic,model=virtio,vhost > > > > Yes, should work. > > > > > > > > They all failed with the following error > > > vhost_net_init returned -7 > > > This is an error message from hw/virtio-net.c:virtio_net_driver_ok() > when > > > vhost_net_start() fails. It looks like dev->binding->irqfd() is failing > > in > > > vhost_virtqueue_init(). > > > > what parameters are passed in? > > > > > Haven't yet debugged further. > > > > this calls into virtio_pci_irqfd. > > > > > I have CONFIG_EVENTFD > > > enabled in the host kernel. > > > > Note you need to also enable eventfd support under kvm menu. > > > > > Are all the above -net options supposed to work? > > > > > > In your descriptions, you say that checksum/tso offload is not > supported. > > Isn't > > > it > > > possible to send/receive large packets without checksum using AF_PACKET > > sockets > > > if > > > the attached interface supports these offloads. > > > Do you see the same offload issue even when using tap backend via > vhost? > > > > > > Thanks > > > Sridhar > > > > > > > > > > > > > > > > > > > > > > > > > > >