From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=45905 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Ps8Na-0002Gd-NY for qemu-devel@nongnu.org; Wed, 23 Feb 2011 01:39:20 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Ps8NZ-0001MF-3k for qemu-devel@nongnu.org; Wed, 23 Feb 2011 01:39:18 -0500 Received: from chello084112167138.7.11.vie.surfer.at ([84.112.167.138]:51684 helo=wiesinger.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Ps8NY-0001LW-IL for qemu-devel@nongnu.org; Wed, 23 Feb 2011 01:39:17 -0500 Date: Wed, 23 Feb 2011 07:38:28 +0100 (CET) From: Gerhard Wiesinger Subject: Re: [Qemu-devel] Re: Network bridging without adding bridge with brctl, possible? In-Reply-To: <201102211307.38714.arnd@arndb.de> Message-ID: References: <4D622602.7070009@web.de> <201102211307.38714.arnd@arndb.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Arnd Bergmann Cc: Jan Kiszka , qemu-devel@nongnu.org On Mon, 21 Feb 2011, Arnd Bergmann wrote: > On Monday 21 February 2011, Jan Kiszka wrote: >>> Now I think I tried all useful possible combinations: >>> 1.) macvtap0 and macvlan0 in bridged and non bridge mode >>> 2.) macvlan0 based on eth0 or based on macvtap0 >>> 3.) Using and ip address on macvlan0 or not (tried both for 7b mac and >>> 7c mac) >>> 4.) Using 7b and 7c mac address in KVM (MAC in KVM is always the command >>> line configured) and used tap interface from macvtap0 (as no second tap >>> devices shows up) >>> >>> Summary: >>> 1.) Using macvtap0 only with 7b mac on interface and also at qemu works >>> well in bridged mode as well as non bridged mode but with limitation no >>> guest/host communication possible, used interface in KVM is tap >>> interface created by macvtap0. Quite logically that it doesn't work with >>> guest/host because of missing hairpin mode on the switch. But according >>> to http://virt.kernelnewbies.org/MacVTap bridge mode should work even >>> without hairpinning available on the switch. >>> 2.) macvlan0 doesn't create any tap interface therefore it can't be used >>> as a device on KVM. >>> 3.) Using 7c mac address in KVM doesn't work at all regardsless of >>> setting ip addresses on macvlan0 or any other setup. >>> >>> @Arnd: Can you explain a setup in detail which should work also with >>> host/guest communication. Thnx. >>> >>> Any further ideas? >>> Which kernel is needed to work? >>> (current: 2.6.34.7-56.fc13.x86_64) >> >> Actually, I tried this successfully over a 2.6.38-rcX, but I have no >> idea what changes related to macvlan/tap went in since .34 and if this >> is required for this. > > We had a few bugs in .34, but it should work in general. > > One thing to make sure is that the host has connectivity through the > macvlan interface and the lower interface (eth0) has no IP address assigned > but is 'up'. > > It could also be a bug in the NIC, you could try to set the NIC into > promiscious mode using ethtool to work around that. Hello, After some further tests and looking at the iproute ip and kernel code I finally gave up because I thing such a setup it is not possible without breaking up/reconfiguring eth0. When I have to reconfigure eth0 I think a better approach is to configure a bridge which I finally did and works well. I tried to explain/document the macvtap/macvlan concepts and limitations below. Please comment on it whether this is true or false. macvtap/macvlan driver concepts and limitations: 1.) macvlan driver adds a MAC address to a lower interface device where the actual macvlanx device is based on 2.) macvtap driver is based on macvlan driver and macvtap driver adds additional functionality of interface <=> external program communication with stdin/stdout channel. 3.) Limitations: macvtap/macvlan based devices can only communicate with childs based on the same lower device (e.g. eth0 in this sample) but not to the lower device itself, only to the outside world of the interface (I guess limitations are derived from function macvlan_queue_xmit: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=drivers/net/macvlan.c;h=6ed577b065df136a18af5ee483810773ca4f43f8;hb=HEAD#l217) Example: # Lower device eth0 must not have an IP address assigned, but when also no communication is possible with macv* devices /root/download/iproute2/iproute2-2.6.37/ip/ip link add link eth0 name macvtap0 type macvtap mode bridge /root/download/iproute2/iproute2-2.6.37/ip/ip link set macvtap0 address 1a:46:0b:ca:bc:7b up /root/download/iproute2/iproute2-2.6.37/ip/ip link add link eth0 name macvtap1 type macvtap mode bridge /root/download/iproute2/iproute2-2.6.37/ip/ip link set macvtap1 address 1a:46:0b:ca:bc:7c up /root/download/iproute2/iproute2-2.6.37/ip/ip link add link eth0 name macvlan0 type macvtap mode bridge /root/download/iproute2/iproute2-2.6.37/ip/ip link set macvlan0 address 1a:46:0b:ca:bc:7d up Example usage: eth0 should not be assigned any IP. macvlan0 should be used as the main host interface and the main IP address is configured. macvtap0 and macvtap1 can be used in e.g. VMs. Bridge communication is possible between the 3 interfaces and the outer world: macvtap0, macvtap1, macvlan0 E.g.: 1.) macvtap0 <=> macvtap1, macvlan0, eth0 external 2.) macvtap1 <=> macvtap0, macvlan0, eth0 external 3.) macvlan0 <=> macvtap0, macvtap1, eth0 external But no communication is possible between lower device eth0 and the child interfaces macvtap0, macvtap1, macvlan0: 1.) eth0 <=> macvtap0 2.) eth0 <=> macvtap1 3.) eth0 <=> macvlan0 finally this makes the macvlan/macvtap approach useless because main eth0 interface must still be broken in the chain and reconfigured which was against the requirements that eth0 should not be touched and reconfigured! Bridge Setup for Fedora 13 is below (reconfigure with service network restart, be carefull on remote operation or have a serial console :-)!). Nevertheless, thank you for your help and I'm ready for any other approach :-). Ciao, Gerhard -- http://www.wiesinger.com/ ############################################################ cat /etc/sysconfig/network-scripts/ifcfg-br0 # Bridge config DEVICE=br0 TYPE=Bridge BOOTPROTO=static BROADCAST=192.168.0.255 IPADDR=192.168.0.9 NETMASK=255.255.255.0 NETWORK=192.168.0.0 ONBOOT=yes DNS1=192.168.0.2 SEARCH="mydomain" NM_CONTROLLED= ############################################################ cat /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE=eth0 # Removed for bridge config #BOOTPROTO=static #BROADCAST=192.168.0.255 ######################### # Onboard one HWADDR=00:1f:c6:8c:59:1f ######################### # Removed for bridge config #IPADDR=192.168.0.9 #NETMASK=255.255.255.0 #NETWORK=192.168.0.0 ONBOOT=yes #DNS1=192.168.0.2 #SEARCH="mydomain" NM_CONTROLLED= # New to add for bridge # Bridge membership BRIDGE=br0 BOOTPROTO=none ############################################################ cat /etc/sysconfig/network-scripts/_ifcfg-eth0.nonbridge # Old config without bridge DEVICE=eth0 BOOTPROTO=static BROADCAST=192.168.0.255 ######################### # Onboard one HWADDR=00:1f:c6:8c:59:1f ######################### IPADDR=192.168.0.9 NETMASK=255.255.255.0 NETWORK=192.168.0.0 ONBOOT=yes DNS1=192.168.0.2 SEARCH="mydomain" NM_CONTROLLED= ############################################################