Re: Multiple TAP Interfaces, with multiple bridges

[Tom Lendacky <tahm@linux.vnet.ibm.com>]

On Wednesday 03 February 2010 10:56:43 am J L wrote:
> Hi,
> 
> I am having an odd networking issue. It is one of those "it used to
> work, and now it doesn't" kind of things. I can't work out what I am
> doing differently.
> 
> I have a virtual machine, started with (among other things):
>   -net nic,macaddr=fa:9e:0b:53:d2:7d,model=rtl8139 -net
> tap,script=/images/1/ifup-eth0,downscript=/images/1/ifdown-eth0
>   -net nic,macaddr=fa:02:4e:86:ed:ce,model=e1000 -net
> tap,script=/images/1/ifup-eth1,downscript=/images/1/ifdown-eth1
> 

I believe this has to do with the qemu vlan support. If you don't specify the 
vlan= option you end up with nics on the same vlan. You need to assign the two 
nics to separate vlans using vlan= on each net parameter, eg:


   -net nic,vlan=0,macaddr=fa:9e:0b:53:d2:7d,model=rtl8139 -net
 tap,vlan=0,script=/images/1/ifup-eth0,downscript=/images/1/ifdown-eth0
   -net nic,vlan=1,macaddr=fa:02:4e:86:ed:ce,model=e1000 -net
 tap,vlan=1,script=/images/1/ifup-eth1,downscript=/images/1/ifdown-eth1

Try that and see if you get the results you expect.

Tom

> The ifup-ethX script inserts the tap interface into the correct bridge
> (of which there are multiple.)
> 
> The Virtual Machine is Centos 5.3, with a 2.6.27.21 kernel. The Host
> is Ubuntu 9.10 with a 2.6.31 kernel.
> 
> 
> My network then looks like:
> 
> The Virtual Machine has an eth0 interface, which is matched with tap0
> on the host.
> The Virtual Machine has an eth1 interface, which is matched with tap1
> on the host.
> 
> The host has a bridge br0, which contains tap0 and eth0.
> The host has a bridge br1, which contains tap1.
> 
> There is a server on the same network as the Host's eth0.
> 
> The Virtual Machines eth0 interface is down.
> The Virtual Machines eth1 interface has an IP address of 192.168.1.10/24.
> The Virtual Machine has a default gateway of 192.168.1.1.
> 
> The host's br0 has an IP address of 192.168.0.1/24.
> The host's br1 has an IP address of 192.168.1.1/24.
> 
> The server has an IP address of 192.168.0.20/24, and a default gateway
> of 192.168.0.1.
> 
> Firewalling is disabled everywhere. I have allowed time for the
> bridges and STP to settle.
> 
> 
> 
> If I go to the Virtual Machine, and ping 192.168.0.20 (the server), I
> would expect tcpdumps to show:
>   * VM: eth1, dest MAC of Host's tap1/br0
>   * Host: tap1, dest MAC of Host's tap1/br0
>   * Host: br1, dest MAC of Host's tap1/br0
>   * Host now routes from br1 to br0
>   * Host: tap0, no packet
>   * Host: br0, dest MAC of Server
>   * Host: eth0, dest MAC of Server
>   * Server: eth0, dest MAC of Server
> 
> What I actually get:
>   * VM: eth1, dest MAC of Host's tap1/br0
>   * Host: tap1, dest MAC of Host's tap1/br0
>   * Host: br1, dest MAC of Host's tap1/br0
>   * Host should, but does not route from br0 to br1
>   * Host: tap0, dest MAC of ***Host's tap1/br0***
>   * Host: br0, dest MAC of ***Host's tap1/br0**
>   * Host: eth0, no packet
>   * Server: eth0, no packet
> 
> As you can see, the packet has egressed *both* tap interfaces! Is this
> expected behaviour? What can I do about this?
> 
> 
> 
> 
> If I remove tap0 from the bridge, I then get:
>   * VM: eth1, dest MAC of Host's tap1/br0
>   * Host: tap1, dest MAC of Host's tap1/br0
>   * Host: br1, dest MAC of Host's tap1/br0
>   * Host should, but does not, route from br0 to br1
>   * Host: tap0, no packet
>   * Host: br0, no packet
>   * Host: eth0, no packet
>   * Server: eth0, no packet
> 
> This is the other half of my problem: in this case, with effectively
> only one tap, the host is not routing between br1 and br0. The packet
> just gets silently dropped. Does anyone know what I am doing wrong?
> 
> I hope I have managed to explain this well enough!
> 
> Thanks,
> --
> Jarrod Lowe
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>