NIC limit

NIC limit

[linux_kvm]

Hi again everybody,
 
One of the admins at the ProxmoxVE project was gracious enough to
quickly release a package including the previously discussed change to
allow up to 32 NICs in qemu.
 
For future reference the .deb is here:
ftp://download.proxmox.com/debian/dists/lenny/pvetest/binary-amd64/pve-qemu-kvm_0.12.5-2_amd64.deb
 
Upon creating & running the VM with the newly patched qemu-kvm app
installed, I found a NIC limitation remained in place, presumably
imposed by some other aspect of the environment.
 
The machine would start when it had 33 PCI devices, as long as no more
than 28 of them were NICs.
 
This is still a vast improvement compared to the previous limit of 8
NICs, and is very good news for my project. I post here in hopes that
maybe someone will come across the link in a search and have a solution.
 
More likely however the new API will be in place and widely in use by
then, but whatever.
 
Either way, thanks for your help yesterday.

Re: NIC limit

[Chris Wright]

* linux_kvm@proinbox.com (linux_kvm@proinbox.com) wrote:
> Hi again everybody,
>  
> One of the admins at the ProxmoxVE project was gracious enough to
> quickly release a package including the previously discussed change to
> allow up to 32 NICs in qemu.

You mean they patched qemu to increase the MAX_NICS constant?  Nice to
get the quick turn around.

Te better choice is to use a newer command line.  Not only does it avoid
the MAX_NICS limitation, but it also enables standard virtio-net offload
accelerations.

> For future reference the .deb is here:
> ftp://download.proxmox.com/debian/dists/lenny/pvetest/binary-amd64/pve-qemu-kvm_0.12.5-2_amd64.deb
>  
> Upon creating & running the VM with the newly patched qemu-kvm app
> installed, I found a NIC limitation remained in place, presumably
> imposed by some other aspect of the environment.
>  
> The machine would start when it had 33 PCI devices, as long as no more
> than 28 of them were NICs.

The PCI bus has only 32 slots (devices), 3 taken by chipset + vga, and
a 4th if you have, for example, a virtio disk.  Are you sure these are
33 PCI devices and not 33 PCI functions?

thanks,
-chris

Re: NIC limit

[linux_kvm]

>The PCI bus has only 32 slots (devices), 3 taken by chipset + vga, and
>a 4th if you have, for example, a virtio disk.  Are you sure these are
>33 PCI devices and not 33 PCI functions?

No, not sure.
Apparently my statement was based on an uninformed assumption.

I tested using a VM that had 30 (removable-per-web-interface)
attachments, and added 3x IDE HDDs to bring it above what I thought was
32 devices:
28 virtio NICs
 1 IDE CD-ROM
 1 virtio HDD
+3 IDE HDDs

I could add IDE up past 32 and it would start, as soon as there were
more than 28 NICs with or without the 3 IDE HDDs, start would fail.


On Wed, 06 Oct 2010 10:18 -0700, "Chris Wright" <chrisw@sous-sol.org>
wrote:
> * linux_kvm@proinbox.com (linux_kvm@proinbox.com) wrote:
> > Hi again everybody,
> >  
> > One of the admins at the ProxmoxVE project was gracious enough to
> > quickly release a package including the previously discussed change to
> > allow up to 32 NICs in qemu.
> 
> You mean they patched qemu to increase the MAX_NICS constant?  Nice to
> get the quick turn around.
> 
> Te better choice is to use a newer command line.  Not only does it avoid
> the MAX_NICS limitation, but it also enables standard virtio-net offload
> accelerations.
> 
> > For future reference the .deb is here:
> > ftp://download.proxmox.com/debian/dists/lenny/pvetest/binary-amd64/pve-qemu-kvm_0.12.5-2_amd64.deb
> >  
> > Upon creating & running the VM with the newly patched qemu-kvm app
> > installed, I found a NIC limitation remained in place, presumably
> > imposed by some other aspect of the environment.
> >  
> > The machine would start when it had 33 PCI devices, as long as no more
> > than 28 of them were NICs.
> 
> The PCI bus has only 32 slots (devices), 3 taken by chipset + vga, and
> a 4th if you have, for example, a virtio disk.  Are you sure these are
> 33 PCI devices and not 33 PCI functions?
> 
> thanks,
> -chris
>