Trouble understanding net config options

Trouble understanding net config options

[Michael Jinks]

On my desktop I have KVM working and one guest running, with the command line:

 # kvm -m 512M -net nic -net tap -hda /dev/mapper/pile-evil64 -boot c
-vnc :2 -smp 2 -nographic

Next I need to set up a virtual network for testing.  The plan calls
for four guest systems, and two virtual networks, one connected to the
outside world through eth0, and the other with no gateway, just
passing traffic back and forth between the VM's.  All the VM's wil
need to have two NICs each, one connected to each of the two virtual
networks.

So, having the OS automatically assign tap interfaces to guest NICs
isn't going to work.  I need to specify which tap goes to which
virtual bridge, and I'd like that to be consistent across VM reboots.

I have the bridges and several tap interfaces configured.  Here's my
'brctl show' output:

bridge name	bridge id		STP enabled	interfaces
br0		8000.00144fa1f17a	no		eth0
							tap0
							tap1
							tap12
							tap14
							tap16
							tap18
br1		8000.deadbeef3200	no		tap11
							tap13
							tap15
							tap17
							tap9

Now I want to bring up a VM with two NICs, one attached to tap12
(bridge 0), the other on tap11 (bridge 1), but I think I've
misunderstood the versious "-net" options as described in the kvm man
page.  It *seems* to say that the way to map a specific tap to a
specific guest NIC is to say, "-net nic" followed by "-net
tap,name=<name of the tap from the host OS's perspective>", and repeat
that sequence for each nic to appear within the VM.

So, here's the full command line I tried, based on that reading:

  # kvm -m 512M -net nic -net tap,name=tap11 -net nic -net
tap,name=tap12 -hda /vmstore/wee -vnc :11 -cdrom
/path/to/my/Windows.iso -boot d

But when I try that, the host OS brings up two new tap interfaces
(tap2 and tap3), the guest OS never boots (at least, nothing pops up
on its console through VNC), and networking on my system completely
freezes up until I kill -9 the kvm process (kill -15 is ignored) and
restart br0.

I tried "fd=" instead of "name=", but that looks for a file descriptor
instead of a network interface name, and I didn't even know that Linux
had file descriptors for network interfaces let alone how to map them
to a tap.  Nothing under /dev looks promising.

Clearly, I am confused.

Re: Trouble understanding net config options

[Stephane Bakhos]

> bridge name	bridge id		STP enabled	interfaces
> br0		8000.00144fa1f17a	no		eth0
> 							tap0
> 							tap1
> 							tap12
> 							tap14
> 							tap16
> 							tap18
> br1		8000.deadbeef3200	no		tap11
> 							tap13
> 							tap15
> 							tap17
> 							tap9
>
> Now I want to bring up a VM with two NICs, one attached to tap12
> (bridge 0), the other on tap11 (bridge 1), but I think I've
> misunderstood the versious "-net" options as described in the kvm man
> page.  It *seems* to say that the way to map a specific tap to a
> specific guest NIC is to say, "-net nic" followed by "-net
> tap,name=<name of the tap from the host OS's perspective>", and repeat
> that sequence for each nic to appear within the VM.
>
> So, here's the full command line I tried, based on that reading:
>
>  # kvm -m 512M -net nic -net tap,name=tap11 -net nic -net
> tap,name=tap12 -hda /vmstore/wee -vnc :11 -cdrom
> /path/to/my/Windows.iso -boot d
>
> But when I try that, the host OS brings up two new tap interfaces
> (tap2 and tap3), the guest OS never boots (at least, nothing pops up
> on its console through VNC), and networking on my system completely
> freezes up until I kill -9 the kvm process (kill -15 is ignored) and
> restart br0.
>
> I tried "fd=" instead of "name=", but that looks for a file descriptor
> instead of a network interface name, and I didn't even know that Linux
> had file descriptors for network interfaces let alone how to map them
> to a tap.  Nothing under /dev looks promising.

You need to add a vlan option to one of them, for example vlan=2
Otherwise kvm will bridge the interfaces together, and it's going to 
create a packet storm.

Re: Trouble understanding net config options

[Michael Jinks]

On Wed, Jul 15, 2009 at 6:05 PM, Stephane
Bakhos<nuitari@melchior.nuitari.net> wrote:

> You need to add a vlan option to one of them, for example vlan=2
> Otherwise kvm will bridge the interfaces together, and it's going to create
> a packet storm.

I wondered about that -- but what's the relationship of a KVM vlan to
my existing bridge interfaces, and how can I control which one gets
mapped to, say "vlan 1" or "vlan 2"?

Are these redundant?  Should I get rid of the bridges?  Question still
remains about how to control which one connects to a physical NIC...

Re: Trouble understanding net config options

[Stephane Bakhos]

>> You need to add a vlan option to one of them, for example vlan=2
>> Otherwise kvm will bridge the interfaces together, and it's going to create
>> a packet storm.
>
> I wondered about that -- but what's the relationship of a KVM vlan to
> my existing bridge interfaces, and how can I control which one gets
> mapped to, say "vlan 1" or "vlan 2"?
>
> Are these redundant?  Should I get rid of the bridges?  Question still
> remains about how to control which one connects to a physical NIC...

It's not redundant, it just ensures that each tap is treated as it's own 
lan by kvm and that it isn't bridged together by kvm.

You need to keep the bridges as the kvm process doesn't talk to other kvm 
processes by itself.

Re: Trouble understanding net config options

[Andreas Plesner Jacobsen]

On Wed, Jul 15, 2009 at 07:05:48PM -0400, Stephane Bakhos wrote:
>>
>> I tried "fd=" instead of "name=", but that looks for a file descriptor
>> instead of a network interface name, and I didn't even know that Linux
>> had file descriptors for network interfaces let alone how to map them
>> to a tap.  Nothing under /dev looks promising.
>
> You need to add a vlan option to one of them, for example vlan=2
> Otherwise kvm will bridge the interfaces together, and it's going to  
> create a packet storm.

Not if the tap-interfaces are connected to different bridges.

-- 
Andreas

Re: Trouble understanding net config options

[Michael Jinks]

On Wed, Jul 15, 2009 at 6:12 PM, Andreas Plesner Jacobsen<apj@mutt.dk> wrote:
> On Wed, Jul 15, 2009 at 07:05:48PM -0400, Stephane Bakhos wrote:
>>>
>> You need to add a vlan option to one of them, for example vlan=2
>> Otherwise kvm will bridge the interfaces together, and it's going to
>> create a packet storm.
>
> Not if the tap-interfaces are connected to different bridges.

In that case... How do I make a guest use a specific tap?  Quoting
from my initial post, my -net options are:

 -net nic -net tap,name=tap11 -net nic -net tap,name=tap12

I think I am getting a packet storm -- which explains why my br0 and
everything on it crashes -- but that's probably because the "name="
options are being ignored, and instead KVM is bringing up two new
taps, probably both attached to br0.

So, what's wrong with my "name=" options?

Re: Trouble understanding net config options

[Chris Webb]

Michael Jinks <michael.jinks@gmail.com> writes:

> How do I make a guest use a specific tap?  Quoting
> from my initial post, my -net options are:
> 
>  -net nic -net tap,name=tap11 -net nic -net tap,name=tap12

You want

  -net nic,vlan=0 -net tap,vlan=0,ifname=tap11 -net nic,vlan=1 -net tap,vlan=1,ifname=tap12

to get the effect that (I think) you're looking for: one nic connected to
tap11 using vlan0 and one nic connected to tap12 using vlan1.

Without the vlan parameters, everything's on vlan0 so you get two nics and
two tap interfaces all connected together inside qemu on a single virtual
switch.

Best wishes,

Chris.

Re: Trouble understanding net config options

[Michael Jinks]

On Wed, Jul 15, 2009 at 6:24 PM, Chris Webb<chris@arachsys.com> wrote:
> You want
>
>  -net nic,vlan=0 -net tap,vlan=0,ifname=tap11 -net nic,vlan=1 -net tap,vlan=1,ifname=tap12

Progress!  This works, I can bring up the guest and watch it boot, but
both of its NICs came up bound to the first bridge on the system.

I can work around that using 'brctl delif' and 'brctl addif' on the
host system, but how would I automate it so that, say, tap11 always
goes to br1, and tap12 always goes to br0?  I guess by taking out the
brctl in the kvm ifup script, and pre-setting all my bridge/tap
connections?

I think that brctl command in the kvm-ifup script is a (distro) bug.

Re: Trouble understanding net config options

[Charles Duffy]

Michael Jinks wrote:
> On Wed, Jul 15, 2009 at 6:24 PM, Chris Webb<chris@arachsys.com> wrote:
>> You want
>>
>>  -net nic,vlan=0 -net tap,vlan=0,ifname=tap11 -net nic,vlan=1 -net tap,vlan=1,ifname=tap12
> 
> Progress!  This works, I can bring up the guest and watch it boot, but
> both of its NICs came up bound to the first bridge on the system.
> 
> I can work around that using 'brctl delif' and 'brctl addif' on the
> host system, but how would I automate it so that, say, tap11 always
> goes to br1, and tap12 always goes to br0?  I guess by taking out the
> brctl in the kvm ifup script, and pre-setting all my bridge/tap
> connections?

Use the script= argument on the -net tap,vlan=1 to use a qemu-ifup 
script which connects the tap device to the bridge you'd prefer be used.

Re: Trouble understanding net config options

[Andreas Plesner Jacobsen]

On Thu, Jul 16, 2009 at 01:12:19AM +0200, Andreas Plesner Jacobsen wrote:
> >>
> >> I tried "fd=" instead of "name=", but that looks for a file descriptor
> >> instead of a network interface name, and I didn't even know that Linux
> >> had file descriptors for network interfaces let alone how to map them
> >> to a tap.  Nothing under /dev looks promising.
> >
> > You need to add a vlan option to one of them, for example vlan=2
> > Otherwise kvm will bridge the interfaces together, and it's going to  
> > create a packet storm.
> 
> Not if the tap-interfaces are connected to different bridges.

I'm sorry, I misunderstood your point. You're right.

-- 
Andreas

Re: Trouble understanding net config options

[Andreas Plesner Jacobsen]

On Wed, Jul 15, 2009 at 05:54:14PM -0500, Michael Jinks wrote:
> 
> Now I want to bring up a VM with two NICs, one attached to tap12
> (bridge 0), the other on tap11 (bridge 1), but I think I've
> misunderstood the versious "-net" options as described in the kvm man
> page.  It *seems* to say that the way to map a specific tap to a
> specific guest NIC is to say, "-net nic" followed by "-net
> tap,name=<name of the tap from the host OS's perspective>", and repeat
> that sequence for each nic to appear within the VM.
> 
> So, here's the full command line I tried, based on that reading:
> 
>   # kvm -m 512M -net nic -net tap,name=tap11 -net nic -net
> tap,name=tap12 -hda /vmstore/wee -vnc :11 -cdrom
> /path/to/my/Windows.iso -boot d

The parameter is "ifname", not "name".

-- 
Andreas

Re: Trouble understanding net config options

[Michael Jinks]

>>   # kvm -m 512M -net nic -net tap,name=tap11 -net nic -net
>> tap,name=tap12 -hda /vmstore/wee -vnc :11 -cdrom
>> /path/to/my/Windows.iso -boot d
>
> The parameter is "ifname", not "name".

In that case, what does the "name" parameter mean?  Quoting from the
manpage on my system:

       -net tap[,vlan=n][,name=name][,fd=h][,ifname=name][,script=file][,down-
       script=dfile]
           Connect the host TAP network interface name to VLAN n, use the net-
           work script file to configure it and the network script dfile to
           deconfigure it. If name is not provided, the OS automatically pro-
           vides one. fd=h can be used to specify the handle of an already
           opened host TAP interface.

I tried again, substituting "ifname" for "name" and leaving everything
else as-is,  That draws an error:

  device tap11 is already a member of a bridge; can't enslave it to bridge br0.
  /etc/kvm/kvm-ifup: could not launch network script
  Could not initialize device 'tap'

I suppose that's a good sign, but it still leaves me wondering how to
control which tap connects to which bridge, if I can't attach a guest
to an existing tap.

Re: Trouble understanding net config options

[Stephane Bakhos]

>  device tap11 is already a member of a bridge; can't enslave it to bridge br0.
>  /etc/kvm/kvm-ifup: could not launch network script
>  Could not initialize device 'tap'

That's because your kvm-ifup scripts tries to connect the tap to the 
bridge and it's already there. You should either remove it from the bridge 
before hand or remove the brctl addif line from kvm-ifup

Re: Trouble understanding net config options

[Andreas Plesner Jacobsen]

On Wed, Jul 15, 2009 at 06:22:56PM -0500, Michael Jinks wrote:
> >>   # kvm -m 512M -net nic -net tap,name=tap11 -net nic -net
> >> tap,name=tap12 -hda /vmstore/wee -vnc :11 -cdrom
> >> /path/to/my/Windows.iso -boot d
> >
> > The parameter is "ifname", not "name".
> 
> In that case, what does the "name" parameter mean?  Quoting from the
> manpage on my system:
> 
>        -net tap[,vlan=n][,name=name][,fd=h][,ifname=name][,script=file][,down-
>        script=dfile]

No idea, it's not a supported parameter in the kvm-72 I have on this
machine, maybe it's the name in the qemu console for manipulating the
device.

> I tried again, substituting "ifname" for "name" and leaving everything
> else as-is,  That draws an error:
> 
>   device tap11 is already a member of a bridge; can't enslave it to bridge br0.
>   /etc/kvm/kvm-ifup: could not launch network script
>   Could not initialize device 'tap'
> 
> I suppose that's a good sign, but it still leaves me wondering how to
> control which tap connects to which bridge, if I can't attach a guest
> to an existing tap.

You can. Try adding script=no, so the kvm-ifup script does not get run.

-- 
Andreas

Re: Trouble understanding net config options

[Michael Jinks]

On Wed, Jul 15, 2009 at 6:35 PM, Andreas Plesner Jacobsen<apj@mutt.dk> wrote:
> On Wed, Jul 15, 2009 at 06:22:56PM -0500, Michael Jinks wrote:


>> I suppose that's a good sign, but it still leaves me wondering how to
>> control which tap connects to which bridge, if I can't attach a guest
>> to an existing tap.
>
> You can. Try adding script=no, so the kvm-ifup script does not get run.

Aha!  That answers my last question too.

Thanks!