From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tom Lendacky Subject: Re: Multiple TAP Interfaces, with multiple bridges Date: Wed, 3 Feb 2010 11:10:50 -0600 Message-ID: <201002031110.50628.tahm@linux.vnet.ibm.com> References: <15f314a41002030856o70066267pc2e8f2b768fd3d83@mail.gmail.com> Mime-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Cc: kvm@vger.kernel.org To: J L Return-path: Received: from e8.ny.us.ibm.com ([32.97.182.138]:49217 "EHLO e8.ny.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757402Ab0BCRK7 (ORCPT ); Wed, 3 Feb 2010 12:10:59 -0500 Received: from d01relay07.pok.ibm.com (d01relay07.pok.ibm.com [9.56.227.147]) by e8.ny.us.ibm.com (8.14.3/8.13.1) with ESMTP id o13D41eL009450 for ; Wed, 3 Feb 2010 08:04:01 -0500 Received: from d01av02.pok.ibm.com (d01av02.pok.ibm.com [9.56.224.216]) by d01relay07.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id o13HAq7A2048020 for ; Wed, 3 Feb 2010 12:10:52 -0500 Received: from d01av02.pok.ibm.com (loopback [127.0.0.1]) by d01av02.pok.ibm.com (8.14.3/8.13.1/NCO v10.0 AVout) with ESMTP id o13HAqZi024585 for ; Wed, 3 Feb 2010 15:10:52 -0200 In-Reply-To: <15f314a41002030856o70066267pc2e8f2b768fd3d83@mail.gmail.com> Sender: kvm-owner@vger.kernel.org List-ID: On Wednesday 03 February 2010 10:56:43 am J L wrote: > Hi, > > I am having an odd networking issue. It is one of those "it used to > work, and now it doesn't" kind of things. I can't work out what I am > doing differently. > > I have a virtual machine, started with (among other things): > -net nic,macaddr=fa:9e:0b:53:d2:7d,model=rtl8139 -net > tap,script=/images/1/ifup-eth0,downscript=/images/1/ifdown-eth0 > -net nic,macaddr=fa:02:4e:86:ed:ce,model=e1000 -net > tap,script=/images/1/ifup-eth1,downscript=/images/1/ifdown-eth1 > I believe this has to do with the qemu vlan support. If you don't specify the vlan= option you end up with nics on the same vlan. You need to assign the two nics to separate vlans using vlan= on each net parameter, eg: -net nic,vlan=0,macaddr=fa:9e:0b:53:d2:7d,model=rtl8139 -net tap,vlan=0,script=/images/1/ifup-eth0,downscript=/images/1/ifdown-eth0 -net nic,vlan=1,macaddr=fa:02:4e:86:ed:ce,model=e1000 -net tap,vlan=1,script=/images/1/ifup-eth1,downscript=/images/1/ifdown-eth1 Try that and see if you get the results you expect. Tom > The ifup-ethX script inserts the tap interface into the correct bridge > (of which there are multiple.) > > The Virtual Machine is Centos 5.3, with a 2.6.27.21 kernel. The Host > is Ubuntu 9.10 with a 2.6.31 kernel. > > > My network then looks like: > > The Virtual Machine has an eth0 interface, which is matched with tap0 > on the host. > The Virtual Machine has an eth1 interface, which is matched with tap1 > on the host. > > The host has a bridge br0, which contains tap0 and eth0. > The host has a bridge br1, which contains tap1. > > There is a server on the same network as the Host's eth0. > > The Virtual Machines eth0 interface is down. > The Virtual Machines eth1 interface has an IP address of 192.168.1.10/24. > The Virtual Machine has a default gateway of 192.168.1.1. > > The host's br0 has an IP address of 192.168.0.1/24. > The host's br1 has an IP address of 192.168.1.1/24. > > The server has an IP address of 192.168.0.20/24, and a default gateway > of 192.168.0.1. > > Firewalling is disabled everywhere. I have allowed time for the > bridges and STP to settle. > > > > If I go to the Virtual Machine, and ping 192.168.0.20 (the server), I > would expect tcpdumps to show: > * VM: eth1, dest MAC of Host's tap1/br0 > * Host: tap1, dest MAC of Host's tap1/br0 > * Host: br1, dest MAC of Host's tap1/br0 > * Host now routes from br1 to br0 > * Host: tap0, no packet > * Host: br0, dest MAC of Server > * Host: eth0, dest MAC of Server > * Server: eth0, dest MAC of Server > > What I actually get: > * VM: eth1, dest MAC of Host's tap1/br0 > * Host: tap1, dest MAC of Host's tap1/br0 > * Host: br1, dest MAC of Host's tap1/br0 > * Host should, but does not route from br0 to br1 > * Host: tap0, dest MAC of ***Host's tap1/br0*** > * Host: br0, dest MAC of ***Host's tap1/br0** > * Host: eth0, no packet > * Server: eth0, no packet > > As you can see, the packet has egressed *both* tap interfaces! Is this > expected behaviour? What can I do about this? > > > > > If I remove tap0 from the bridge, I then get: > * VM: eth1, dest MAC of Host's tap1/br0 > * Host: tap1, dest MAC of Host's tap1/br0 > * Host: br1, dest MAC of Host's tap1/br0 > * Host should, but does not, route from br0 to br1 > * Host: tap0, no packet > * Host: br0, no packet > * Host: eth0, no packet > * Server: eth0, no packet > > This is the other half of my problem: in this case, with effectively > only one tap, the host is not routing between br1 and br0. The packet > just gets silently dropped. Does anyone know what I am doing wrong? > > I hope I have managed to explain this well enough! > > Thanks, > -- > Jarrod Lowe > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html >