8 NIC limit

8 NIC limit

[linux_kvm]

Hello list:

I'm working on a project that calls for the creation of a firewall in
KVM.
While adding a 20-interface trunk of virtio adapters to bring in a dual
10GB bond, I've discovered an 8 NIC limit in QEMU.

I found the following thread in the list archives detailing a similar
problem:
http://kerneltrap.org/mailarchive/linux-kvm/2009/1/29/4848304

It includes a patch for the file qemu/net.h to allow 24 NICs:
https://bugs.launchpad.net/ubuntu/+source/qemu-kvm">qemu-kvm/+bug/595873/+attachment/1429544/+files/max_nics.patch

In my case I want to attach 29, and have simply changed line 8 to 30
from 24.

This will be the first patch I've ever had to do, and so far my internet
search yields results that don't seem to apply.

Would someone like to recommend a pertinent tutorial?

Many thanks

Re: 8 NIC limit

[Dustin Kirkland]

On Tue, Oct 5, 2010 at 7:48 AM,  <linux_kvm@proinbox.com> wrote:
> Hello list:
>
> I'm working on a project that calls for the creation of a firewall in
> KVM.
> While adding a 20-interface trunk of virtio adapters to bring in a dual
> 10GB bond, I've discovered an 8 NIC limit in QEMU.
>
> I found the following thread in the list archives detailing a similar
> problem:
> http://kerneltrap.org/mailarchive/linux-kvm/2009/1/29/4848304
>
> It includes a patch for the file qemu/net.h to allow 24 NICs:
> https://bugs.launchpad.net/ubuntu/+source/qemu-kvm">qemu-kvm/+bug/595873/+attachment/1429544/+files/max_nics.patch
>
> In my case I want to attach 29, and have simply changed line 8 to 30
> from 24.
>
> This will be the first patch I've ever had to do, and so far my internet
> search yields results that don't seem to apply.
>
> Would someone like to recommend a pertinent tutorial?

Hi there,

I commented on the original bug in Launchpad.  We're willing and able
to carry the patch against qemu-kvm in Ubuntu, I just asked that the
reporter at least submit the patch upstream for discussion.  I don't
see where that has happened yet.  It's a trivial patch to submit.
Please note in that bug a pointer to the mailing list thread, if you
start one.

To your specific question, different communities have different
requirements on patch submission, so you do need to consult each
community.  A good place to start might be the
Documentation/SubmittingPatches how-to in the kernel tree:
 * http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob_plain;f=Documentation/SubmittingPatches;hb=HEAD

In this case, I think you're going to want to send your patch to the
qemu-devel (on CC) mailing list (perhaps in addition to sending it
here, to the kvm list).

:-Dustin

Re: 8 NIC limit - patch - places limit at 32

[linux_kvm]

[-- Attachment #1: Type: text/plain, Size: 99 bytes --]

Attached is a patch that allows qemu to have up to 32 NICs, without
using the qdev -device method.

[-- Attachment #2: max_nics.patch --]
[-- Type: application/octet-stream, Size: 225 bytes --]

--- a/net.h	2010-02-26 03:34:00.000000000 +0100
+++ b/net.h.my	2010-06-10 18:48:53.215317795 +0200
@@ -119,7 +119,7 @@
 
 /* NIC info */
 
-#define MAX_NICS 8
+#define MAX_NICS 32 
 enum {
 	NIC_NVECTORS_UNSPECIFIED = -1
 };

Re: 8 NIC limit - patch - places limit at 32

[linux_kvm]

It's 8 otherwise- and after the patch is applied, it still only goes to
28 for some reason.
28's acceptable for my needs, so I'll step aside from here & leave it to
the experts.

As for the new -device method, that's all fine & good but AFAIK it's not
implemented on my platform, so this was the answer.

On Wed, 06 Oct 2010 07:54 -0500, "Anthony Liguori"
<anthony@codemonkey.ws> wrote:
> On 10/06/2010 12:46 AM, linux_kvm@proinbox.com wrote:
> > Attached is a patch that allows qemu to have up to 32 NICs, without
> > using the qdev -device method.
> >    
> 
> I'd rather there be no fixed limit and we validate that when add fails 
> because there isn't a TCP slot available, we do the right thing.
> 
> BTW, using -device, it should be possible to add a very high number of 
> nics because you can specify the PCI address including a function.  If 
> this doesn't Just Work today, we should make it work.
> 
> Regards,
> 
> Anthony Liguori
> 

Re: 8 NIC limit - patch - places limit at 32

[Chris Wright]

* Anthony Liguori (anthony@codemonkey.ws) wrote:
> BTW, using -device, it should be possible to add a very high number of 
> nics because you can specify the PCI address including a function.  If 
> this doesn't Just Work today, we should make it work.

Should work...test...mostly[1], but I don't actually know of any tools that
make use of it.

thanks,
-chris

[1] 40 worked, 48 caused guest kernel stack corruption, didn't dig in to
see why yet.

Here's my simple wrapper to build up the command line:

QEMU=/home/chrisw/git/kvm/qemu-kvm/x86_64-softmmu/qemu-system-x86_64
BIOS=/home/chrisw/git/kvm/qemu-kvm/pc-bios
DISK=/home/chrisw/disk-snap1.img
SCRIPT=/home/chrisw/git/kvm/qemu-kvm/kvm/scripts/qemu-ifup

unset NETARGS
i=0
dev=4
func=0
max_dev=40
while [ $i -lt $max_dev ]
do
  unset MULTIFUNC
  if [ $(($i + 1)) -lt $max_dev -a $func -eq 0 ]; then
    MULTIFUNC=",multifunction=on"
  fi

  NETARGS="${NETARGS} -netdev type=tap,id=netdev$i,script=$SCRIPT -device virtio-net-pci,mac=52:54:00:12:34:$(printf "%.2x\n" $i),netdev=netdev$i,bus=pci.0,addr=$dev.$func$MULTIFUNC"

  i=$(($i+1))
  func=$(($func+1))
  if [ $func -eq 8 ]; then
    func=0
    dev=$(($dev+1))
  fi
done

$QEMU -L $BIOS -m 1024 -drive file=$DISK,if=virtio,boot=on $NETARGS -vnc :0

Re: 8 NIC limit

[Markus Armbruster]

linux_kvm@proinbox.com writes:

> Hello list:
>
> I'm working on a project that calls for the creation of a firewall in
> KVM.
> While adding a 20-interface trunk of virtio adapters to bring in a dual
> 10GB bond, I've discovered an 8 NIC limit in QEMU.

Have you tried creating NICs with -device?  The limit shouldn't apply
there.

Re: 8 NIC limit

[linux_kvm]

> Have you tried creating NICs with -device?

I'm not sure what that is, will look into it, thanks.

I'm using ProxmoxVE, and currently add them via a web interface.

Someone happens to host a screenshot of that part here:
http://c-nergy.be/blog/wp-content/uploads/Proxmox_Net2.png

On Tue, 05 Oct 2010 17:57 +0200, "Markus Armbruster" <armbru@redhat.com>
wrote:
> linux_kvm@proinbox.com writes:
> 
> > Hello list:
> >
> > I'm working on a project that calls for the creation of a firewall in
> > KVM.
> > While adding a 20-interface trunk of virtio adapters to bring in a dual
> > 10GB bond, I've discovered an 8 NIC limit in QEMU.
> 
> Have you tried creating NICs with -device?  The limit shouldn't apply
> there.
> 

Re: 8 NIC limit

[Brian Jackson]

  On 10/5/2010 9:48 AM, linux_kvm@proinbox.com wrote:
> Hello list:
>
> I'm working on a project that calls for the creation of a firewall in
> KVM.
> While adding a 20-interface trunk of virtio adapters to bring in a dual
> 10GB bond, I've discovered an 8 NIC limit in QEMU.
>
> I found the following thread in the list archives detailing a similar
> problem:
> http://kerneltrap.org/mailarchive/linux-kvm/2009/1/29/4848304
>
> It includes a patch for the file qemu/net.h to allow 24 NICs:
> https://bugs.launchpad.net/ubuntu/+source/qemu-kvm">qemu-kvm/+bug/595873/+attachment/1429544/+files/max_nics.patch
>
> In my case I want to attach 29, and have simply changed line 8 to 30
> from 24.


I'd guess you'll bump into a pci device number limit (I believe it is 32 
at the moment).


> This will be the first patch I've ever had to do, and so far my internet
> search yields results that don't seem to apply.
>
> Would someone like to recommend a pertinent tutorial?
>
> Many thanks
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Fwd: Re: 8 NIC limit

[linux_kvm]

Forgot to cc list, forwarding.

> In this case, I think you're going to want to send your patch to the
> qemu-devel (on CC) mailing list (perhaps in addition to sending it
> here, to the kvm list).

Will do, thanks for the pointer.

Before I do so, I'd like to bring up one thing that comes to mind.

I don't know how to make the determination, but it makes sense to me for
the limit defined here to be indicitive of an actual limitation, rather
than what seems an arbitrary best-guess as to the most someone might
need.

If the change ends up being permanent, then I would hope it would be a
large enough value to provide a degree of extensibility and prevent the
necessity of bumping it up again later when someone else comes along
with even greater bandwidth requirements.

Perhaps someone could provide some guidance as to a sane, higher number,
as opposed to an arbitrary '65000' which would surely prevent this from
happening again (knock on wood).

For the time being I still have to find something to help learn how to
implement the change locally.
I rarely have to compile let alone deal with patches, so to me at least
this is a considerable obstacle.

-Thanks

On Tue, 05 Oct 2010 08:24 -0700, "Dustin Kirkland"
<kirkland@canonical.com> wrote:
> On Tue, Oct 5, 2010 at 7:48 AM,  <linux_kvm@proinbox.com> wrote:
> > Hello list:
> >
> > I'm working on a project that calls for the creation of a firewall in
> > KVM.
> > While adding a 20-interface trunk of virtio adapters to bring in a dual
> > 10GB bond, I've discovered an 8 NIC limit in QEMU.
> >
> > I found the following thread in the list archives detailing a similar
> > problem:
> > http://kerneltrap.org/mailarchive/linux-kvm/2009/1/29/4848304
> >
> > It includes a patch for the file qemu/net.h to allow 24 NICs:
> > https://bugs.launchpad.net/ubuntu/+source/qemu-kvm">qemu-kvm/+bug/595873/+attachment/1429544/+files/max_nics.patch
> >
> > In my case I want to attach 29, and have simply changed line 8 to 30
> > from 24.
> >
> > This will be the first patch I've ever had to do, and so far my internet
> > search yields results that don't seem to apply.
> >
> > Would someone like to recommend a pertinent tutorial?
> 
> Hi there,
> 
> I commented on the original bug in Launchpad.  We're willing and able
> to carry the patch against qemu-kvm in Ubuntu, I just asked that the
> reporter at least submit the patch upstream for discussion.  I don't
> see where that has happened yet.  It's a trivial patch to submit.
> Please note in that bug a pointer to the mailing list thread, if you
> start one.
> 
> To your specific question, different communities have different
> requirements on patch submission, so you do need to consult each
> community.  A good place to start might be the
> Documentation/SubmittingPatches how-to in the kernel tree:
>  *
>  http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob_plain;f=Documentation/SubmittingPatches;hb=HEAD
> 
> In this case, I think you're going to want to send your patch to the
> qemu-devel (on CC) mailing list (perhaps in addition to sending it
> here, to the kvm list).
> 
> :-Dustin
> 

Re: 8 NIC limit

[Chris Wright]

* linux_kvm@proinbox.com (linux_kvm@proinbox.com) wrote:
> I don't know how to make the determination, but it makes sense to me for
> the limit defined here to be indicitive of an actual limitation, rather
> than what seems an arbitrary best-guess as to the most someone might
> need.

The actual limitation is that we have only a single PCI bus.  The
network devices are single function devices.  While there are 32 slots
on the PCI bus, a few are already taken (typically the first 3, leaving
29 free slots).

But, as Markus mentioned, this limitation is only for legacy command
line option.  The modern qdev way[1] should not hit this same limitation.

thanks,
-chris

[1] http://www.linux-kvm.org/page/Qdev_status

8 nic limit

[Richard Weinberger]

Hi!

qemu-kvm seems to support only 8 network interfaces.
I found this posting from 2009:
http://www.mail-archive.com/kvm@vger.kernel.org/msg10104.html

Why is MAX_NICS still only 8?
Please raise this limit in future releases.

I have some use cases where much more than 8 nics are needed.
(Lot's of VLANs...)

Thanks,
//richard

P.s: Please CC me.

Re: 8 nic limit

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 513 bytes --]

On 2011-09-03 13:54, Richard Weinberger wrote:
> Hi!
> 
> qemu-kvm seems to support only 8 network interfaces.
> I found this posting from 2009:
> http://www.mail-archive.com/kvm@vger.kernel.org/msg10104.html
> 
> Why is MAX_NICS still only 8?
> Please raise this limit in future releases.
> 
> I have some use cases where much more than 8 nics are needed.
> (Lot's of VLANs...)

If you avoid the legacy -net and create networks via "-device <nic>" and
-netdev, you won't face this limit.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]