Re: vhost-net patches

["Michael S. Tsirkin" <mst@redhat.com>]

On Tue, Oct 20, 2009 at 10:27:38AM -0700, Shirley Ma wrote:
> 
> Hello Michael,
> 
> Here are the output, I am using guest 2.6.32-rc3 kernel. It doesn't use
> MSIX. So which guest kernel I should use??
> 
> [xma@localhost ~]$ cat /proc/interrupts.
>            CPU0
>   0:        299   IO-APIC-edge      timer.
>   1:          2   IO-APIC-edge      i8042.
>   2:          0    XT-PIC-XT        cascade
>   4:         76   IO-APIC-edge      serial
>  11:       2126   IO-APIC-edge      virtio1, virtio0 <--------- here is the
> virtio for both disk and network i/o??

Yes, this is regular shared IRQ, no good.
I think your guest is too old, please use kernel 2.6.31 and up in guest.
I will work to improve the error message as well.

>  12:         89   IO-APIC-edge      i8042
> NMI:          0   Non-maskable interrupts
> LOC:       5146   Local timer interrupts
> SPU:          0   Spurious interrupts
> CNT:          0   Performance counter interrupts
> PND:          0   Performance pending work
> RES:          0   Rescheduling interrupts
> CAL:          0   Function call interrupts
> TLB:          0   TLB shootdowns
> TRM:          0   Thermal event interrupts
> MCE:          0   Machine check exceptions
> MCP:          1   Machine check polls
> ERR:          0
> MIS:          0
> [xma@localhost ~]$ uname -r
> 2.6.32-rc3
> 
> 
> Shirley Ma
> IBM Linux Technology Center
> 15300 SW Koll Parkway
> Beaverton, OR 97006-6063
> Phone(Fax): (503) 578-7638
> 
> 
> 
> 
>                                                                        
>              "Michael S.                                               
>              Tsirkin"                                                  
>              <mst@redhat.com>                                           To
>                                        Shirley Ma/Beaverton/IBM@IBMUS  
>              10/20/2009 10:18                                           cc
>              AM                        David Stevens/Beaverton/IBM@IBMUS,
>                                        kvm@vger.kernel.org,            
>                                        sri@linux.vnet.ibm.com          
>                                                                    Subject
>                                        Re: vhost-net patches           
>                                                                        
>                                                                        
>                                                                        
>                                                                        
>                                                                        
>                                                                        
> 
> 
> 
> 
> On Tue, Oct 20, 2009 at 10:14:55AM -0700, Shirley Ma wrote:
> >
> > Hello Michael,
> >
> > >what is vnet->vector?
> > >And what do you mean by not defined?
> >
> > In funcation:
> >
> > static int vhost_virtqueue_init()
> > {
> > ......
> >               r = vdev->binding->irqfd(vdev->binding_opaque, q->vector,
> > vq->call);
> > ......
> > }.....
> >
> > q->vector is 65535,
> 
> Thanks for debugging this.
> I think this means that guest does not use MSI-X.
> 
> You can verify this by booting guest without vhost,
> and performing the following command:
>               cat /proc/interrupts
> 
> Please note that you currently need recent kernel in guest,
> so that it uses MSI-X. I plan on implementing regular IRQ,
> but not yet, and it will be slower anyway.
> 
> > in   static int virtio_pci_irqfd()
> > {
> > ......
> >     if (vector >= proxy->pci_dev.msix_entries_nr) {
> >         fprintf(stderr, " pci irq fd returned vector %d, msix_entries_nr
> %d
> > \n",
> >                 vector, proxy->pci_dev.msix_entries_nr); <------- I added
> > one output line here.
> >         return -EINVAL;
> > ....
> > }...
> > The output is:
> >
> >        pci irq fd returned vector 65535, msix_entries_nr 3, EINVAL is
> > returned.
> >
> > thanks
> > Shirley Ma
> > IBM Linux Technology Center
> > 15300 SW Koll Parkway
> > Beaverton, OR 97006-6063
> > Phone(Fax): (503) 578-7638
> >
> >
> >
> >
> >
> >              "Michael S.
> >              Tsirkin"
> >              <mst@redhat.com>
> To
> >                                        Shirley Ma/Beaverton/IBM@IBMUS
> >              10/20/2009 04:34
> cc
> >              AM                        sri@linux.vnet.ibm.com, David
> >                                        Stevens/Beaverton/IBM@IBMUS,
> >                                        kvm@vger.kernel.org
> >
> Subject
> >                                        Re: vhost-net patches
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Mon, Oct 19, 2009 at 04:08:24PM -0700, Shirley Ma wrote:
> > > Hello Michael,
> > >
> > > >They all failed with the following error
> > > >    vhost_net_init returned -7
> > > >This is an error message from hw/virtio-net.c:virtio_net_driver_ok()
> > when
> > > >vhost_net_start() fails. It looks like dev->binding->irqfd() is
> failing
> > in
> > > >vhost_virtqueue_init(). Haven't yet debugged further. I have
> > CONFIG_EVENTFD
> > > >enabled in the host kernel.
> > >
> > > >From the debug output, looks like the vnet->vector is not defined,
> >
> > what is vnet->vector?
> > And what do you mean by not defined?
> >
> > > and the
> > > default msix_entries_nr is 3, so it returned EINVAL from
> > virtio_pci_irqfd.
> > > Looks we need to either disable QEMU_PCI_CAP_MSIX or define vector in
> > QEMU
> > > configuration?
> >
> > You shouldn't have to do anything.
> >
> > > I am not familiar with MSIX stuffs.
> > >
> > > Thanks
> > > Shirley
> > >
> > >
> > > Inactive hide details for sri---10/19/2009 03:56:57 PM---On Sun,
> > 2009-10-18 at
> > > 19:32 +0200, Michael S. Tsirkin wrote:sri---10/19/2009 03:56:57 PM---On
> > Sun,
> > > 2009-10-18 at 19:32 +0200, Michael S. Tsirkin wrote:
> > >
> > >                 sri@linux.vnet.ibm.com [cid]   *
> > >                                             To "Michael S. Tsirkin"
> > <mst@redhat.com>,
> > >                 10/19/2009 03:56 PM            kvm@vger.kernel.org
> > >                                        [cid]   *
> > >                                             cc David
> > Stevens/Beaverton/IBM@IBMUS, Shirley Ma/
> > >                                                Beaverton/IBM@IBMUS
> > >                                        [cid]   *
> > >                                        Subject Re: vhost-net patches
> > >                                        *        *
> > >
> > > On Sun, 2009-10-18 at 19:32 +0200, Michael S. Tsirkin wrote:
> > > > On Sun, Oct 18, 2009 at 12:53:56PM +0200, Michael S. Tsirkin wrote:
> > > > > On Fri, Oct 16, 2009 at 12:29:29PM -0700, Sridhar Samudrala wrote:
> > > > > > Hi Michael,
> > > > > >
> > > > > > We are trying out your vhost-net patches from your git trees on
> > > kernel.org.
> > > > > > I am using mst/vhost.git as host kernel and mst/qemu-kvm.git for
> > qemu.
> > > > > >
> > > > > > I am using the following qemu script to start the guest using
> > userspace
> > > tap backend.
> > > > > >
> > > > > >
> > home/sridhar/git/mst/qemu-kvm/x86_64-softmmu/qemu-system-x86_64 /home/
> > > sridhar/kvm_images/fedora10-1-vm -m 512 -drive
> > file=/home/sridhar/kvm_images/
> > > fedora10-1-vm,if=virtio,index=0,boot=on -net nic,macaddr=
> > > 54:52:00:35:e3:73,model=virtio -net
> > tap,ifname=vnet0,script=no,downscript=no
> > > > > >
> > > > > > Now that i got the default backend to work, i wanted to try vhost
> > in
> > > kernel. But
> > > > > > could not figure out the right -net option to pass to qemu.
> > > > > >
> > > > > > Can you let me know the right syntax to start a guest using
> vhost.
> > > > > >
> > > > > > Thanks
> > > > > > Sridhar
> > > > >
> > > > > Here's an example with raw socket:
> > > > >
> > > > > /root/kvm-test/bin/qemu-system-x86_64 -m 1G -kernel \
> > > > > /boot/vmlinuz-$release -append \
> > > > > 'root=UUID=d5d2d201-d086-42ad-bb1d-32fbe40eda71 ro quiet nosplash \
> > > > > console=tty0 console=ttyS0,9600n8' -initrd /boot/guest-initrd.img \
> > > > > $HOME/disk.raw.copy -net raw,ifname=eth3 -net
> nic,model=virtio,vhost
> > \
> > > > > -balloon none -redir tcp:8023::22
> > > > >
> > > > > As you see, I changed the command line.
> > > > > You now simply add ",vhost" after model, and it will locate
> > > > > host network interface specified earlier and attach to it.
> > > > > This should have been clear from running  qemu with -help
> > > > > flag. Could you please suggest how can that text
> > > > > be clarified?
> > >
> > > I updated to your latest git trees and the default user-space tap
> backend
> > using
> > > the
> > > following -net options worked fine.
> > > -net tap,ifname=vnet0,script=no,downscript=no -net nic,model=virtio
> > >
> > > But i could not get vhost to work with either raw or tap backends.
> > > I tried the following combinations.
> > > 1) -net raw,ifname=eth0 -net nic,model=virtio,vhost
> > > 2) -net raw,ifname=vnet0, -net nic,model=virtio,vhost
> > > 3) -net tap,ifname=vnet0,script=no,downscript=no -net
> > nic,model=virtio,vhost
> >
> > Yes, should work.
> >
> > >
> > > They all failed with the following error
> > >    vhost_net_init returned -7
> > > This is an error message from hw/virtio-net.c:virtio_net_driver_ok()
> when
> > > vhost_net_start() fails. It looks like dev->binding->irqfd() is failing
> > in
> > > vhost_virtqueue_init().
> >
> > what parameters are passed in?
> >
> > > Haven't yet debugged further.
> >
> > this calls into virtio_pci_irqfd.
> >
> > > I have CONFIG_EVENTFD
> > > enabled in the host kernel.
> >
> > Note you need to also enable eventfd support under kvm menu.
> >
> > > Are all the above -net options supposed to work?
> > >
> > > In your descriptions, you say that checksum/tso offload is not
> supported.
> > Isn't
> > > it
> > > possible to send/receive large packets without checksum using AF_PACKET
> > sockets
> > > if
> > > the attached interface supports these offloads.
> > > Do you see the same offload issue even when using tap backend via
> vhost?
> > >
> > > Thanks
> > > Sridhar
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
>