[Qemu-devel] Connecting vde and LAN

[Qemu-devel] Connecting vde and LAN

[Oliver Gerlich]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,
what is the best solution to connect the vde "switch" to my real LAN so
that Qemu guests get IPs from my LAN-wide DHCP server?

So far I've experimented with bridging tap0 and eth0 so that the Qemu
guests are transparently on my LAN and get IPs from the LAN-wide DHCP
server. But this has the drawback that my host network interface is now
br0 instead of eth0 (that causes confusion at least for samba).
Also I tried to set up IP forwarding between tap0 and eth0, but then the
Qemu guests aren't transparently on my LAN (and don't get the correct
config from the DHCP server).

So, are there other solutions that offer the power of bridging with the
simpleness of IP forwarding? What do you recommend to establish a LAN
with real hosts and Qemu instances side by side? Such a setup could be
interesting for all users who want more than user-net and who have a
"SOHO router" at home which provides DHCP and Dynamic DNS!

Thanks in advance,
Oliver Gerlich
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFC0RQUTFOM6DcNJ6cRAgFJAJ9gfaQmw9E06h9Mr1C2Ugin/PcFagCfTh39
kc0bKVpqT4YFfx9FptEQaN8=
=hHCt
-----END PGP SIGNATURE-----

Re: [Qemu-devel] Connecting vde and LAN

[Henrik Nordstrom]

On Sun, 10 Jul 2005, Oliver Gerlich wrote:

> what is the best solution to connect the vde "switch" to my real LAN so
> that Qemu guests get IPs from my LAN-wide DHCP server?

bridgeing of your ethernet interface and the TAP interface connecting to 
vde is undoubtly the best if you want to provide full access to the LAN.

> So far I've experimented with bridging tap0 and eth0 so that the Qemu
> guests are transparently on my LAN and get IPs from the LAN-wide DHCP
> server. But this has the drawback that my host network interface is now
> br0 instead of eth0 (that causes confusion at least for samba).

Fix the Samba problem?

Why does this cause problems for Samba? I run my qemu stations like this 
all the time, but not using Samba..

> Also I tried to set up IP forwarding between tap0 and eth0, but then the
> Qemu guests aren't transparently on my LAN (and don't get the correct
> config from the DHCP server).

Correct.

DHCP is Ethernet broadcast based, not IP.

You could in theory run a DHCP proxy agent on your tap0 interface, and 
have the DHCP server configured with a suitable scope. But this does not 
give you addresses from your LAN.

You could also run a local DHCP server on tap0, configured with the 
address scope you have set up proxy-arp for. But these addresses better be 
statically assigned to you, never handed out by the LAN DHCP server to 
anyone else in the LAN. But this doesn't provide true LAN access, only 
routed acces to the LAN (no broadcasts).

Regards
Henrik

Re: [Qemu-devel] Connecting vde and LAN

[Jim C. Brown]

On Sun, Jul 10, 2005 at 06:42:30PM +0200, Henrik Nordstrom wrote:
> On Sun, 10 Jul 2005, Oliver Gerlich wrote:
> 
> >what is the best solution to connect the vde "switch" to my real LAN so
> >that Qemu guests get IPs from my LAN-wide DHCP server?
> 
> bridgeing of your ethernet interface and the TAP interface connecting to 
> vde is undoubtly the best if you want to provide full access to the LAN.
> 
> >Also I tried to set up IP forwarding between tap0 and eth0, but then the
> >Qemu guests aren't transparently on my LAN (and don't get the correct
> >config from the DHCP server).

The best solution is to use a vde_switch with libpcap and libnet. This allows
intercepting packets meant for the guests (via libnet) and forwarding them thru
vde. libpcap allows for packets in vde to be passed on to the lan. This also
allows for a sort of seamless bridging between host and guest lan. (Technically
a tap device is still required to be able to talk to the host, but my understanding
is that libpcap handles this automatically.) Plus, the nic remains eth0.

I am not sure how libpcap and libnet would work with DHCP or ARP, but since
they work on the ethernet level I suppose a single dhcpd server on eth0 would
work in this case, but I am not 100% sure.

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.

Re: [Qemu-devel] Connecting vde and LAN

[Oliver Gerlich]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Henrik Nordstrom schrieb:
> On Sun, 10 Jul 2005, Oliver Gerlich wrote:
> 
>> what is the best solution to connect the vde "switch" to my real LAN so
>> that Qemu guests get IPs from my LAN-wide DHCP server?
> 
> 
> bridgeing of your ethernet interface and the TAP interface connecting to
> vde is undoubtly the best if you want to provide full access to the LAN.
> 
>> So far I've experimented with bridging tap0 and eth0 so that the Qemu
>> guests are transparently on my LAN and get IPs from the LAN-wide DHCP
>> server. But this has the drawback that my host network interface is now
>> br0 instead of eth0 (that causes confusion at least for samba).
> 
> 
> Fix the Samba problem?
> 
> Why does this cause problems for Samba? I run my qemu stations like this
> all the time, but not using Samba..

The "problem" is that I start vde_switch and the bridging not at boot,
but when I want to run Qemu. So then I have to restart Samba to bind to
to br0 instead of eth0. Not so much of a problem though... Only I don't
know what other services already rely on eth0 as my network interface :)

Could you explain how you integrated vde_switch and bridging with your
current system? Do you start everything at boot and accept that this
changes the system-wide network configuration, or do you use some
bridging trick to avoid this?

> 
>> Also I tried to set up IP forwarding between tap0 and eth0, but then the
>> Qemu guests aren't transparently on my LAN (and don't get the correct
>> config from the DHCP server).
> 
> 
> Correct.
> 
> DHCP is Ethernet broadcast based, not IP.
> 
> You could in theory run a DHCP proxy agent on your tap0 interface, and
> have the DHCP server configured with a suitable scope. But this does not
> give you addresses from your LAN.
> 
> You could also run a local DHCP server on tap0, configured with the
> address scope you have set up proxy-arp for. But these addresses better
> be statically assigned to you, never handed out by the LAN DHCP server
> to anyone else in the LAN. But this doesn't provide true LAN access,
> only routed acces to the LAN (no broadcasts).

So it looks indeed that vde and bridging is the way to go.
Thanks for your explanations :)

> 
> Regards
> Henrik
> 
> 
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel
> 
> 

Greetings,
Oliver
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFC0V+KTFOM6DcNJ6cRAtA2AKC7yADYol1HiZ8yGUlg7H1qZSOpqwCgtR4l
G/5XEwjQH6Ov1bXfkgkHPs4=
=V9ii
-----END PGP SIGNATURE-----

Re: [Qemu-devel] Connecting vde and LAN

[Oliver Gerlich]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jim C. Brown schrieb:
> On Sun, Jul 10, 2005 at 06:42:30PM +0200, Henrik Nordstrom wrote:
> 
>>On Sun, 10 Jul 2005, Oliver Gerlich wrote:
>>
>>
>>>what is the best solution to connect the vde "switch" to my real LAN so
>>>that Qemu guests get IPs from my LAN-wide DHCP server?
>>
>>bridgeing of your ethernet interface and the TAP interface connecting to 
>>vde is undoubtly the best if you want to provide full access to the LAN.
>>
>>
>>>Also I tried to set up IP forwarding between tap0 and eth0, but then the
>>>Qemu guests aren't transparently on my LAN (and don't get the correct
>>>config from the DHCP server).
> 
> 
> The best solution is to use a vde_switch with libpcap and libnet. This allows
> intercepting packets meant for the guests (via libnet) and forwarding them thru
> vde. libpcap allows for packets in vde to be passed on to the lan. This also
> allows for a sort of seamless bridging between host and guest lan. (Technically
> a tap device is still required to be able to talk to the host, but my understanding
> is that libpcap handles this automatically.) Plus, the nic remains eth0.
> 
> I am not sure how libpcap and libnet would work with DHCP or ARP, but since
> they work on the ethernet level I suppose a single dhcpd server on eth0 would
> work in this case, but I am not 100% sure.
> 

Sounds interesting! How do I set this up? Couldn't find anything on
Google, and personally I have no idea how to do this :)

Oliver
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFC0WeTTFOM6DcNJ6cRAlH6AKCfIo99S6MMihOY5DUOpdOFxQaHdgCdG0+B
8POxg7Ufhc5QBU0Rg4WfyT8=
=EC4W
-----END PGP SIGNATURE-----

Re: [Qemu-devel] Connecting vde and LAN

[Bakul Shah]

> DHCP is Ethernet broadcast based, not IP.

Not that it matters in this case but strictly speaking DHCP
is an IP protocol, using UDP as its transport and the request
goes to IP broadcast address of 255.255.255.255.

> You could also run a local DHCP server on tap0, configured with the 
> address scope you have set up proxy-arp for.

You can run a local DHCP server and NAT.

On FreeBSD I assign an IP address like 172.20.0.1 to the
*bridge* interface on the host on startup so that samba and
dhcpd can bind to it.  When qemu is started /etc/qemu-ifup
makes the new tap device a bridge member.  I imagine Linux
has something similar....

Re: [Qemu-devel] Connecting vde and LAN

[Jim C. Brown]

On Sun, Jul 10, 2005 at 08:23:17PM +0200, Oliver Gerlich wrote:
> > The best solution is to use a vde_switch with libpcap and libnet. This allows
> > intercepting packets meant for the guests (via libnet) and forwarding them thru
> > vde. libpcap allows for packets in vde to be passed on to the lan. This also
> > allows for a sort of seamless bridging between host and guest lan. (Technically
> > a tap device is still required to be able to talk to the host, but my understanding
> > is that libpcap handles this automatically.) Plus, the nic remains eth0.
> > 
> > I am not sure how libpcap and libnet would work with DHCP or ARP, but since
> > they work on the ethernet level I suppose a single dhcpd server on eth0 would
> > work in this case, but I am not 100% sure.
> > 
> 
> Sounds interesting! How do I set this up? Couldn't find anything on
> Google, and personally I have no idea how to do this :)
> 
> Oliver

I should have stated that VDE doesn't actually have a version of vde_switch
which uses libpcap or libnet - so it will have to be written first. I'm working
toward that end, but right now its been a slow start.

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.

Re: [Qemu-devel] Connecting vde and LAN

[Henrik Nordstrom]

On Sun, 10 Jul 2005, Oliver Gerlich wrote:

> Could you explain how you integrated vde_switch and bridging with your
> current system? Do you start everything at boot and accept that this
> changes the system-wide network configuration, or do you use some
> bridging trick to avoid this?

I am not currently using vde_switch, instead I use named tun/tap devices 
(see archives) with statically assigned addresses.

The bridges is created at boot time. The host is a Fedora Core 4, and the 
following changes were done to cerate the bridge:

/etc/sysconfig/network-scripts

-- ifcfg-eth0 --
DEVICE=eth0
HWADDR=<the ethernet address>
ONBOOT=yes
TYPE=Ethernet
BRIDGE=br0
USERCTL=no
PEERDNS=yes
IPV6INIT=no

-- ifcfg-br0 --
DEVICE=br0
BOOTPROTO=static
IPADDR=192.168.1.53
NETMASK=255.255.255.0
GATEWAY=192.168.1.254
ONBOOT=yes
TYPE=Bridge

-- ifcfg-qemu0 ---
DEVICE=qemu0
BOOTPROTO=static
ONBOOT=yes
TYPE=Tap
BRIDGE=br0
tunctl -t ${DEVICE} -u 507

where 507 is my user ID allowing me to access the pre-created TUN/TAP 
device. tunctl can be found from user-mode-linux.


In theory it should be possible to set the IP address of br0 via DHCP, but 
the bridge support in the Fedora network scripts is a bit simpleminded and 
can't yet handle this proper (they attempt to dhcp before adding the 
physical interfaces...).



I suppose you could fool Fedora into thinking that the Bridge interface is 
in fact a physical Ethernet interface, and script the creation of the 
bridge in ifcfg-br0. Maybe something like the following will work:


-- ifcfg-br0 --
DEVICE=br0
ONBOOT=yes
TYPE=Ethernet
BOOTPROTO=dhcp
USERCTL=no
PEERDNS=yes
IPV6INIT=no
brctl addbr br0 2>/dev/null || true
brctl stp br0 off
brctl setfd br0 0
brctl sethello br0 0
brctl addif br0 eth0 2>/dev/null || true

-- ifcfg-eth0 --
DEVICE=eth0
ONBOOT=no
TYPE=Ethernet
USERCTL=no

Regards
Henrik

Re: [Qemu-devel] Connecting vde and LAN

[Henrik Nordstrom]

On Sun, 10 Jul 2005, Jim C. Brown wrote:

> The best solution is to use a vde_switch with libpcap and libnet. This allows
> intercepting packets meant for the guests (via libnet) and forwarding them thru
> vde.

Interesting idea. Would make full qemu networking truly a no-brainer.

Should even be possible to write just the libpcap+libnet "driver", 
avoiding the need of a full vde_switch in such case.

> libpcap allows for packets in vde to be passed on to the lan. This also
> allows for a sort of seamless bridging between host and guest lan. (Technically
> a tap device is still required to be able to talk to the host, but my understanding
> is that libpcap handles this automatically.) Plus, the nic remains eth0.

No tap device is needed here. Just a plain raw socket..

> I am not sure how libpcap and libnet would work with DHCP or ARP, but since
> they work on the ethernet level I suppose a single dhcpd server on eth0 would
> work in this case, but I am not 100% sure.

It should work just fine.

Regards
Henrik

Re: [Qemu-devel] Connecting vde and LAN

[Jim C. Brown]

On Mon, Jul 11, 2005 at 04:21:31AM +0200, Henrik Nordstrom wrote:
> On Sun, 10 Jul 2005, Jim C. Brown wrote:
> 
> >The best solution is to use a vde_switch with libpcap and libnet. This 
> >allows
> >intercepting packets meant for the guests (via libnet) and forwarding them 
> >thru
> >vde.
> 
> Interesting idea. Would make full qemu networking truly a no-brainer.
> 
> Should even be possible to write just the libpcap+libnet "driver", 
> avoiding the need of a full vde_switch in such case.
> 

Yes, but if one was running multiple qemu guests, it would be cleaner to
implement this thru vde_switch instead of in qemu directly. That way eth0 is
intercepted by only a single process. (Also no worries about having permissions
to intercept packets and etc for qemu - just run vde_switch as root and qemu
as a regular user.)

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.

Re: [Qemu-devel] Connecting vde and LAN

[Henrik Nordstrom]

On Sun, 10 Jul 2005, Jim C. Brown wrote:

> Yes, but if one was running multiple qemu guests, it would be cleaner to
> implement this thru vde_switch instead of in qemu directly. That way eth0 is
> intercepted by only a single process. (Also no worries about having permissions
> to intercept packets and etc for qemu - just run vde_switch as root and qemu
> as a regular user.)

I didn't mean to implement this in qemu directly, but rather in a helper 
similar to vdeq, but the primary method should be via vde_switch. Having 
this shortcut helper is a future extension.

Inspired by this discussion I now have a prototype vde_packet glue for 
vde_switch at http://www.henriknordstrom.net/examples/vde_packet.c, 
connecting the vde_switch to the LAN via a packet socket.

Light testing looks promising, but there is some known issues which may 
make this approach unfeasible for the intended target:

a) vde_switch 1.5.8 is buggy and bounces packets back the same port they 
came from if the destination MAC is known to reside there, causing a loop 
and near meltdown of the host.  vde_switch should drop the packet in this 
case. The fix is trivial: if (target == port) return; at a suitable place 
near the top of port.c:send_dst(). I haven't looked if this may have been 
fixed in a later vde_switch release but I think not.

b) can't talk to the host itself. This is due to the packets going 
directly to the wire and never really "seen" by the host stack. Not sure 
yet if there is an easy way out, but I suppose it may be possible to set 
up a dummy tap with the same MAC and IP address as the base Ethernet 
device and duplicate broadcasts and packet directed to the host there, 
obviously assuming the administrator does not block this in firewalling..

Regards
Henrik

Re: [Qemu-devel] Connecting vde and LAN

[Jim C. Brown]

[-- Attachment #1: Type: text/plain, Size: 1545 bytes --]

On Mon, Jul 11, 2005 at 09:50:11AM +0200, Henrik Nordstrom wrote:
> On Sun, 10 Jul 2005, Jim C. Brown wrote:
> 
> I didn't mean to implement this in qemu directly, but rather in a helper 
> similar to vdeq, but the primary method should be via vde_switch. Having 
> this shortcut helper is a future extension.
> 

Ooooh! Like slirpvde.

> Inspired by this discussion I now have a prototype vde_packet glue for 
> vde_switch at http://www.henriknordstrom.net/examples/vde_packet.c, 
> connecting the vde_switch to the LAN via a packet socket.
> 

Excellent work! I've attached a modified version called vde_pcap.c, which
uses libpcap to do the dirty work instead. I haven't gotten all the linuxisms
out of it yet, but it should be easy to fix. One day, this might run on Windows
hosts.

> 
> b) can't talk to the host itself. This is due to the packets going 
> directly to the wire and never really "seen" by the host stack. Not sure 
> yet if there is an easy way out, but I suppose it may be possible to set 
> up a dummy tap with the same MAC and IP address as the base Ethernet 
> device and duplicate broadcasts and packet directed to the host there, 
> obviously assuming the administrator does not block this in firewalling..
> 
> Regards
> Henrik
> 

Alas, the accepted solution to allow pcap programs to talk to the host is to
use tuntap to create a tap device and connect the program to the tap device
instead of the real ethernet device.

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.

[-- Attachment #2: vde_pcap.c --]
[-- Type: text/plain, Size: 4958 bytes --]

/* Copyright (C) 2005 Jim Brown
 * Copyright (C) 2005 Henrik Nordstrom
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, write to the Free Software
 * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA.
 *
 * The VDE connection setup (open_vde) is loosely based on a similar
 * functions in vdeq.c by Renzo Davoli
 */

#include <sys/socket.h>
#include <netpacket/packet.h>
#include <net/ethernet.h>       /* the L2 protocols */
#include <sys/ioctl.h>
#include <fcntl.h>
#include <net/if.h>
#include <net/if_arp.h>
#include <sys/un.h>
#include <errno.h>

#include <linux/if_tun.h>

#include <stdio.h>
#include <string.h>
#include <stdint.h>
#include <stdlib.h>

#include <pcap.h>

#define SWITCH_MAGIC 0xfeedface
#define BUFSIZE 2048
#define ETH_ALEN 6
#define PCAP_READ_TIMEOUT 1

enum request_type {
    REQ_NEW_CONTROL
};

struct request_v3 {
    uint32_t magic;
    uint32_t version;
    enum request_type type;
    struct sockaddr_un sock;
};

static int
open_vde(char *name, int intno, int group)
{
    int pid = getpid();
    struct request_v3 req;
    int fdctl;
    int fddata;
    struct sockaddr_un sock;

    if ((fdctl = socket(AF_UNIX, SOCK_STREAM, 0)) < 0) {
	perror("socket");
	exit(1);
    }
    sock.sun_family = AF_UNIX;
    snprintf(sock.sun_path, sizeof(sock.sun_path), "%s", name);
    if (connect(fdctl, (struct sockaddr *) &sock, sizeof(sock))) {
	perror("connect");
	exit(1);
    }
    memset(&req, 0, sizeof(req));
    req.magic = SWITCH_MAGIC;
    req.version = 3;
    req.type = REQ_NEW_CONTROL + ((group > 0) ? ((geteuid() << 8) + group) << 8 : 0);
    req.sock.sun_family = AF_UNIX;
    sprintf(&req.sock.sun_path[1], "%5d-%2d", pid, intno);

    if ((fddata = socket(AF_UNIX, SOCK_DGRAM, 0)) < 0) {
	perror("socket");
	exit(1);
    }
    if (bind(fddata, (struct sockaddr *) &req.sock, sizeof(req.sock)) < 0) {
	perror("bind");
	exit(1);
    }
    if (send(fdctl, &req, sizeof(req), 0) < 0) {
	perror("send");
	exit(1);
    }
    if (recv(fdctl, &sock, sizeof(struct sockaddr_un), 0) < 0) {
	perror("recv");
	exit(1);
    }
    if (connect(fddata, (struct sockaddr *) &sock, sizeof(struct sockaddr_un)) < 0) {
	perror("connect");
	exit(1);
    }
    /* note: fdctl is intentionally leaked. Closed on exit by the OS. */
    return fddata;
}

void
set_nonblocking(int fd)
{
    int fl = fcntl(fd, F_GETFL);
    fcntl(fd, F_SETFL, (fl | O_NONBLOCK));
}

void
pcap_relay(u_char *args, const struct pcap_pkthdr *header, const u_char * packet)
{
    int * vde_socket = (int *)args;
    send(*vde_socket, packet, header->len, 0);
}

static void
wrap_dispatch(fd_set * rfd, pcap_t * handle, int vde_socket)
{
    int n;

    if (FD_ISSET(pcap_get_selectable_fd(handle), rfd)) {
	n = pcap_dispatch(handle, -1, pcap_relay, (u_char *)&vde_socket);
    }
}

int pcap_sendpacket(pcap_t * handle, const u_char * packet, int len)
{
   if (send(pcap_fileno(handle), packet, len, 0) == len)
   {
	return 0;
   }
   else
   {
	return -1;
   }
}

static void
relay(fd_set * rfd, int vde_socket, pcap_t * handle)
{
    int n;
    char packet[65536];
    union {
	struct sockaddr_ll ll;
	struct sockaddr_un un;
    } sa;
    socklen_t sa_len = sizeof(sa);

    if (FD_ISSET(vde_socket, rfd)) {
	n = recvfrom(vde_socket, packet, sizeof(packet), 0, (struct sockaddr *)&sa, &sa_len);
	if (n > 0)
	    pcap_sendpacket(handle, packet, n);
	else if (n == 0)
	    exit(1);
	else if (errno != EINTR) {
	    perror("recv:");
	    exit(1);
	}
    }
}

int
main(int argc, char **argv)
{
    int vde_socket;
    pcap_t * handle;
    char errbuf[PCAP_ERRBUF_SIZE];
    const u_char * packet;
    fd_set rfd;
    int fds;

    if (argc != 3) {
	printf("Usage: %s interface vde_socket\n", argv[0]);
	exit(1);
    }
    handle = pcap_open_live(argv[1], BUFSIZ, 1, PCAP_READ_TIMEOUT, errbuf);
    if (handle == NULL)
    {
    	printf("%s\n", errbuf);
	exit(1);
    }
    vde_socket = open_vde(argv[2], 0, 0);
    set_nonblocking(vde_socket);

    fds = pcap_get_selectable_fd(handle);
    if (vde_socket > fds)
	fds = vde_socket;
    fds += 1;
    FD_ZERO(&rfd);
    int i;
    for (;;) {
	int n;
	FD_SET(pcap_get_selectable_fd(handle), &rfd);
	FD_SET(vde_socket, &rfd);

	n = select(fds, &rfd, NULL, NULL, NULL);
	if (n > 0) {
	    wrap_dispatch(&rfd, handle, vde_socket);
	    relay(&rfd, vde_socket, handle);
	} else if (n < 0 && errno != EINTR) {
	    perror("select:");
	}
    }
}

Re: [Qemu-devel] Connecting vde and LAN

[Jim C. Brown]

On Mon, Jul 11, 2005 at 11:02:04AM -0400, Jim C. Brown wrote:
> > 
> > b) can't talk to the host itself. This is due to the packets going 
> > directly to the wire and never really "seen" by the host stack. Not sure 
> > yet if there is an easy way out, but I suppose it may be possible to set 
> > up a dummy tap with the same MAC and IP address as the base Ethernet 
> > device and duplicate broadcasts and packet directed to the host there, 
> > obviously assuming the administrator does not block this in firewalling..
> > 
> > Regards
> > Henrik
> > 
> 
> Alas, the accepted solution to allow pcap programs to talk to the host is to
> use tuntap to create a tap device and connect the program to the tap device
> instead of the real ethernet device.
> 

I tried using libnet 1.0 to send the packets, but that did not help. Also
tried libnet 1.1, that actually allowed the host to be pinged by the guest -
part of the time. It also caused pings on the same lan (from the guest) to fail
as well sometimes, so its not reliable enough to use.

I guess we should just do it the hard way - when grabbing packets that are
meant for the host, fake the request (fake a ping, manually do the socket call,
etc) and send the result back to the vde.

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.

Re: [Qemu-devel] Connecting vde and LAN

[Henrik Nordstrom]

On Mon, 11 Jul 2005, Jim C. Brown wrote:

> I tried using libnet 1.0 to send the packets, but that did not help. Also
> tried libnet 1.1, that actually allowed the host to be pinged by the guest -
> part of the time. It also caused pings on the same lan (from the guest) to fail
> as well sometimes, so its not reliable enough to use.

Odd.. looking into libnet 1.1.3-rc1 I see no provision in the code for 
sending link packets via anything else than a packet socket which should 
make it impossible to talk to the host..

It you forget about Ethernet and instead send the packet as a raw IP 
packet then it should reach the host, but you must then fake ARP 
responses on behalf of the host.

> I guess we should just do it the hard way - when grabbing packets that are
> meant for the host, fake the request (fake a ping, manually do the socket call,
> etc) and send the result back to the vde.

You mean diverting them to slirp? Some additional magics for ARP will be 
required.. also it doesn't solve host talking to the guest..

Regards
Henrik

Re: [Qemu-devel] Connecting vde and LAN

[Ross Kendall Axe]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Oliver Gerlich wrote:
> 
> The "problem" is that I start vde_switch and the bridging not at boot,
> but when I want to run Qemu. So then I have to restart Samba to bind to
> to br0 instead of eth0. Not so much of a problem though... Only I don't
> know what other services already rely on eth0 as my network interface :)

You've got the 'Bind interfaces only' option turned on in Samba haven't
you? Turn it off and your problem will disappear.

As to what other programs care about interfaces appearing/disappearing,
run 'netstat -ltup' to see who's listening where. Anything listening on
'*:portnumber' probably doesn't care.

Ross
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.7 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD4DBQFC1B1S9bR4xmappRARAv24AJd55+G11njzBT7ZuRht5GM6SUgZAJ9Cf6zp
3NfSFVcsGZERZjly48PcNg==
=RdA/
-----END PGP SIGNATURE-----

Re: [Qemu-devel] Connecting vde and LAN

[Oliver Gerlich]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ross Kendall Axe schrieb:
> Oliver Gerlich wrote:
> 
>>>The "problem" is that I start vde_switch and the bridging not at boot,
>>>but when I want to run Qemu. So then I have to restart Samba to bind to
>>>to br0 instead of eth0. Not so much of a problem though... Only I don't
>>>know what other services already rely on eth0 as my network interface :)
> 
> 
> You've got the 'Bind interfaces only' option turned on in Samba haven't
> you? Turn it off and your problem will disappear.
> 
> As to what other programs care about interfaces appearing/disappearing,
> run 'netstat -ltup' to see who's listening where. Anything listening on
> '*:portnumber' probably doesn't care.

Thanks for your ideas; but I try to run every service bound to specific
interfaces (lo and/or eth0), so I don't get into problems when attaching
eg. a direct internet connection or WLAN.

> 
> Ross

Oliver


_______________________________________________
Qemu-devel mailing list
Qemu-devel@nongnu.org
http://lists.nongnu.org/mailman/listinfo/qemu-devel



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFC1Ci4TFOM6DcNJ6cRAhJGAJ9yluJJa7l5ujrBCh2o2yOn+YJrPQCdF5+r
6+jDiKiDfnFD7b7oxty7ENY=
=KppC
-----END PGP SIGNATURE-----

Re: [Qemu-devel] Connecting vde and LAN

[Jim C. Brown]

On Tue, Jul 12, 2005 at 04:49:08AM +0200, Henrik Nordstrom wrote:
> It you forget about Ethernet and instead send the packet as a raw IP 
> packet then it should reach the host, but you must then fake ARP 
> responses on behalf of the host.
> 

I just tried this with libnet 1.1 (1.1.2.1 to be specific), and it doesn't
seem to work. Pings do not go through. I only handled the vde -> host case
though, do I need to do anything special for host -> vde packets?

I'll try again with libnet 1.0.2a

> >I guess we should just do it the hard way - when grabbing packets that are
> >meant for the host, fake the request (fake a ping, manually do the socket 
> >call,
> >etc) and send the result back to the vde.
> 
> You mean diverting them to slirp? Some additional magics for ARP will be 
> required.. also it doesn't solve host talking to the guest..

Yeah, I'm not sure how to handle the host -> guest case yes.

> 
> Regards
> Henrik
> 
> 
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel
> 

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.

Re: [Qemu-devel] Connecting vde and LAN

[Ross Kendall Axe]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Oliver Gerlich wrote:
> Ross Kendall Axe schrieb:
> 
>>>Oliver Gerlich wrote:
>>>
>>>
>>>>>The "problem" is that I start vde_switch and the bridging not at boot,
>>>>>but when I want to run Qemu. So then I have to restart Samba to bind to
>>>>>to br0 instead of eth0. Not so much of a problem though... Only I don't
>>>>>know what other services already rely on eth0 as my network interface :)
>>>
>>>
>>>You've got the 'Bind interfaces only' option turned on in Samba haven't
>>>you? Turn it off and your problem will disappear.
>>>
>>>As to what other programs care about interfaces appearing/disappearing,
>>>run 'netstat -ltup' to see who's listening where. Anything listening on
>>>'*:portnumber' probably doesn't care.
> 
> 
> Thanks for your ideas; but I try to run every service bound to specific
> interfaces (lo and/or eth0), so I don't get into problems when attaching
> eg. a direct internet connection or WLAN.
> 
> 
>>>Ross
> 
> 
> Oliver
> 

I personally tend to rely more on tcpwrappers and a strong firewall for
this kind of access control, and that's not even mentioning the password
protection on everything :-).

However, in the specific case of Samba, you can still do interface based
access control without using the "bind interfaces only" option.  It will
still only accept connections on the interfaces you specify, but the
problem you're having will go away.

This is all strictly IMHO, of course.

Ross


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.7 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFC1IQ59bR4xmappRARAic5AJ4sBnDMvqbnM2y+PxoH610WHFgrLgCgnEvM
w4yPf1ElMLyEbUWZXQvDCB8=
=VZmx
-----END PGP SIGNATURE-----

Re: [Qemu-devel] Connecting vde and LAN

[Henrik Nordstrom]

On Tue, 12 Jul 2005, Jim C. Brown wrote:

> I just tried this with libnet 1.1 (1.1.2.1 to be specific), and it doesn't
> seem to work. Pings do not go through. I only handled the vde -> host case
> though, do I need to do anything special for host -> vde packets?

Did you fake the ARP response to the guest? If not the guest won't get 
past ARP:ing for the host and no IP packet will be sent..

To make life simpler you should probably start testing UDP in each 
direction (netcat is your friend) with static ARP entries registered on 
both guest and host.

For host->guest packets the RAW sockets demonstrated earlier is fine if 
you accept that the guest packets is also duplicated on the local lan. I 
do not know of a method to have host->guest packets sent cleanly without 
duplication on the Ethernet without setting up a TUN/TAP or PPP interface.

Regards
Henrik

Re: [Qemu-devel] Connecting vde and LAN

[Jim C. Brown]

On Thu, Aug 04, 2005 at 12:14:53PM +0200, Henrik Nordstrom wrote:
> On Tue, 12 Jul 2005, Jim C. Brown wrote:
> 
> >I just tried this with libnet 1.1 (1.1.2.1 to be specific), and it doesn't
> >seem to work. Pings do not go through. I only handled the vde -> host case
> >though, do I need to do anything special for host -> vde packets?
> 
> Did you fake the ARP response to the guest? If not the guest won't get 
> past ARP:ing for the host and no IP packet will be sent..
> 

I attempted to, but it seems I was only partially successful. Sometimes the ARP
table would just have an "(Incomplete)" entry.

> To make life simpler you should probably start testing UDP in each 
> direction (netcat is your friend) with static ARP entries registered on 
> both guest and host.
> 

Hmm...static ARP. Never thought of that.

> For host->guest packets the RAW sockets demonstrated earlier is fine if 
> you accept that the guest packets is also duplicated on the local lan. I 
> do not know of a method to have host->guest packets sent cleanly without 
> duplication on the Ethernet without setting up a TUN/TAP or PPP interface.
> 
> Regards
> Henrik
> 

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.

Re: [Qemu-devel] Connecting vde and LAN

[Jim C. Brown]

On Thu, Aug 04, 2005 at 12:14:53PM +0200, Henrik Nordstrom wrote:
> For host->guest packets the RAW sockets demonstrated earlier is fine if 
> you accept that the guest packets is also duplicated on the local lan. I 
> do not know of a method to have host->guest packets sent cleanly without 
> duplication on the Ethernet without setting up a TUN/TAP or PPP interface.
> 
> Regards
> Henrik
> 

I was thinking that perhaps vde_packet could be modified to use a tap device
(for example tap0). Since the guests can't communicate to the address thats on
tap0, it doesn't matter what address it gets - the host address of tap0 and the
ip addresses of the guests don't even have to be on the same subnet.

So, the modified vde_packet would listen on the vde network and set packets
meant for the host (or other workstations on the LAN) to tap0. A route would be
set up so that the packets would be routed from tap0 to eth0, which would allow
the host to see them. On the other side, vde_packet would listen to eth0, and
intercept any packets meant for the guests and inject those into the vde
network.

Of course, I realize that this isn't as good as being able to get vde_pcap to
work right, as it still requires host OS support (for the tap device). However,
it sounds easier than playing with ARP and IP packets and still gives the same
benefit - we get to bridge the vde network with the LAN onto eth0 instead of
br0.

I'm still learning how to create and manipulate tuntap devices so I haven't
been able to code this up yet. Are there any obvious problems that I missed
which would prevent this from working?

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.

Re: [Qemu-devel] Connecting vde and LAN

[Henrik Nordstrom]

On Wed, 10 Aug 2005, Jim C. Brown wrote:

> I was thinking that perhaps vde_packet could be modified to use a tap device
> (for example tap0). Since the guests can't communicate to the address thats on
> tap0, it doesn't matter what address it gets - the host address of tap0 and the
> ip addresses of the guests don't even have to be on the same subnet.
>
> So, the modified vde_packet would listen on the vde network and set packets
> meant for the host (or other workstations on the LAN) to tap0. A route would be
> set up so that the packets would be routed from tap0 to eth0, which would allow
> the host to see them. On the other side, vde_packet would listen to eth0, and
> intercept any packets meant for the guests and inject those into the vde
> network.

Then we are back in the proxy-arp configuration situation which doesn't 
really require any changes except proper routing and ip-forwarding setup 
on the host. But it doesn't allow the guest Ethernet access to the LAN 
(only IP access), and for the same reasons only unicast (broadcasts are 
not routed).

> I'm still learning how to create and manipulate tuntap devices so I haven't
> been able to code this up yet. Are there any obvious problems that I missed
> which would prevent this from working?

Yes, the fact that it is a separate network device on the host, fully 
separate from eth0. Anything you send to the tap device is in fact like it 
was recevied on another Ethernet interface by the host (lets say eth1). 
Without bridgeing between the two it won't reach the real Ethernet, and 
also in many situations the host will not be overly happy about seeing 
traffic for it's eth0 address on the tap0 interface (rp-filter, 
firewalling and a number of other minor tidbits will notice).

But since raw sockets does not allow you to send packets to the host there 
is not much of a choice is there?


The following should work at the Ethernet level almost fully using the 
already available tools, but requires some host configuration:

   pcap (or raw socket): for receiving/sending to/from the LAN. This also 
grabs broadcasts sent by the host.

   tun device: for sending/receiving to/from the host.


1. Set up vde with both vde_pcap and a TUN/TAP tap device

    vde_switch -tap tap0 -daemon
    vde_pcap

2. Configure the VDE connected tap device with the same IP as the host has 
on the LAN but a netmask of 255.255.255.255 (or /32 depending on your 
taste). This makes the tap device a kind of "point-to-point Ethernet" 
device.

   ip addr add ip.of.host/32 dev tap0
   ip link set tap0 up

3. Route the guest IP address to the tap device

   ip route add ip.of.guest/32 dev tap0



This should work on Linux hosts. No idea about the viability on other host 
flavors such as *BSD or Windows.


The above should work for most situations where the host is a just a host 
on the LAN, but if the host is a LAN server for broadcast Ethernet 
protocols such as DHCP some additional configuration of each such service 
may be required to also listen on the tap0 device if the guest should be 
able to use these services. Quite likely also applies to some other 
protocols such as IPX or AppleTalk as these frames almost certainly will 
be ignored by the host on the tap0 device unless configured proper for 
each protocol.



Now, if only vendors could make the bridge setup easier to manage most of 
this would be avoided.

Regards
Henrik

Re: [Qemu-devel] Connecting vde and LAN

[Oliver Gerlich]

Henrik Nordstrom wrote:
> On Wed, 10 Aug 2005, Jim C. Brown wrote:
> 
[...]
> The above should work for most situations where the host is a just a 
> host on the LAN, but if the host is a LAN server for broadcast Ethernet 
> protocols such as DHCP some additional configuration of each such 
> service may be required to also listen on the tap0 device if the guest 
> should be able to use these services. Quite likely also applies to some 
> other protocols such as IPX or AppleTalk as these frames almost 
> certainly will be ignored by the host on the tap0 device unless 
> configured proper for each protocol.

Couldn't we avoid these incompatibilities if we would route packets only 
on the Ethernet level? If the Qemu networking setup on the host involves 
IP addresses or such things, we're already on the wrong OSI layer I think...

I'm not very experienced in networking, but IMHO the network should be 
set up like this:

                                     (eth0 on Host OS)
                                    /
( LAN ) <----> (real eth0) <-> VDE
                                    \
                                     (NIC on Guest OS)

(substitute eth0 by the LAN network interface of your computer)

So VDE acts as an Ethernet switch between LAN, Host OS and Guest OS. 
This would limit VDE action to the Ethernet layer.
To connect to the LAN, VDE would use the real (physical) eth0 of the 
host system. When packets arrive from Host OS or Guest OS, they would be 
sent to the LAN with the source MAC addresses preserved.
The connection to the Qemu guests should be easy as well (every guest 
has a MAC address anyway).
The host part is really difficult: VDE would have to provide a "faked" 
eth0 to the host OS, so that programs on the host could still use eth0 
as Ethernet interface. This faked eth0 would be connected to VDE just 
like the Guests are.
I guess this means that VDE would have to provide a kernel-layer 
component which grabs the packets from eth0 and provides the faked eth0 
for the Host OS...

Although IMHO this is the cleanest way to set up networking, it's 
probably not possible to implement this :-) But maybe we could use it as 
a starting point for the networking.

So this is a summary of what I think we should try to achieve (sorted by 
priority):
-provide Ethernet-layer access from the Guest to the LAN (so that it's 
transparent for IP traffic)
-allow using a LAN-wide DHCP server for the Guest
-the host should be able to set up his LAN interface over the LAN-wide 
DHCP server.
-the host networking still works as before (at least on the layers above 
Ethernet)
-the whole thing is easy to set up
-the whole thing can be started by any user at any time (so it's not 
started at boot time, but whenever a user starts Qemu).

If we could achieve all of these requirements, it would be fantastic :-)
But maybe we could at least reach some of them.

What are your comments on this?

Regards,
Oliver Gerlich

Re: [Qemu-devel] Connecting vde and LAN

[Jim C. Brown]

On Thu, Aug 11, 2005 at 06:24:51PM +0200, Oliver Gerlich wrote:
> Couldn't we avoid these incompatibilities if we would route packets only 
> on the Ethernet level? If the Qemu networking setup on the host involves 
> IP addresses or such things, we're already on the wrong OSI layer I think...
> 

We could, but it seems that short of kernel level bridging there is no way to
"route" ethernet packets.

> I'm not very experienced in networking, but IMHO the network should be 
> set up like this:
> 
>                                     (eth0 on Host OS)
>                                    /
> ( LAN ) <----> (real eth0) <-> VDE
>                                    \
>                                     (NIC on Guest OS)
> 
> (substitute eth0 by the LAN network interface of your computer)
> 
> So VDE acts as an Ethernet switch between LAN, Host OS and Guest OS. 
> This would limit VDE action to the Ethernet layer.

Incidently, if you have eth0 and eth1, and both are connected to the same
LAN, then it works. Just set up vde_packet/pcacp on eth1, then packed tp the
host from the VDE will be sent out over eth1 and recieved back in eth0.
This solves the problem quite nicely, and it is probably the simplest to
implement, but requires changing the hardware.

I'm trying to figure out how to achieve the same effect with eth0 and tap0
(as opposed to eth0 and eth1).

> The host part is really difficult: VDE would have to provide a "faked" 
> eth0 to the host OS, so that programs on the host could still use eth0 
> as Ethernet interface. This faked eth0 would be connected to VDE just 
> like the Guests are.
> I guess this means that VDE would have to provide a kernel-layer 
> component which grabs the packets from eth0 and provides the faked eth0 
> for the Host OS...
> 

I think this can already be done, without even needing to play with the kernel.
Just set up a tap0 and a tap1. Guests connects to tap1, while tap0 is used by
the host OS (e.g. services are set up to connect to tap0 instead of eth0).
All thats needed is some glue to connect tap0 to tap1 (this can be done
with a userland program) and then vde_pcap can connect eth0 to the tap devices.

(The reason we can glue tap0 and tap1 together, but have a much harder time with
eth0 and tap0, is because when we write to tap0, the host sees it. We can't
write to eth0 directly but (ultimately) have to use a packet socket, which
means the host won't see it.)

This isn't much better than using kernel bridging.

> Although IMHO this is the cleanest way to set up networking, it's 
> probably not possible to implement this :-) But maybe we could use it as 
> a starting point for the networking.
> 

If we're going to be poking around in the kernel anyways, it'd be a lot easier
to

a) fix packet sockets so that the host OS will see packets sent there (or
if this breaks things too badly, create a new type of socket to the same
effect)

or

b) alter the bridging code so that when we create a bridge, its named
eth0 instead of br0.

If we had a), then vde_pcap would "just work" - there'd be no need to fiddle
with faking things on tap devices or etc.

> So this is a summary of what I think we should try to achieve (sorted by 
> priority):


Most of these are my goals as well (except I don't mind having to run a bit of
glue in a startup script).

> -provide Ethernet-layer access from the Guest to the LAN (so that it's 
> transparent for IP traffic)

vde_pcap does this already.

> -allow using a LAN-wide DHCP server for the Guest

vde_pcap does this already, unless the DHCP server is running on the host.

> -the host should be able to set up his LAN interface over the LAN-wide 
> DHCP server.

vde_pcap doesn't affect this. It should work.

> -the host networking still works as before (at least on the layers above 
> Ethernet)

vde_pcap doesn't affect this. It should work, unless we're talking about
guest->host communication at the packet level. This is the main hurdle I'm
trying to tear down.

> -the whole thing is easy to set up

Define "easy". :-D Well, the whole thing can be set up in a script - all the
tricky stuff is done in the code of vde_pcap and friends, so from a users
point of view they're just running a couple of commands.

> -the whole thing can be started by any user at any time (so it's not 
> started at boot time, but whenever a user starts Qemu).

This is true for vde_pcap already. Also, tap devices can be created whenever
we want.

> 
> If we could achieve all of these requirements, it would be fantastic :-)
> But maybe we could at least reach some of them.
> 
> What are your comments on this?
> 
> Regards,
> Oliver Gerlich
> 

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.

Re: [Qemu-devel] Connecting vde and LAN

[Paul Brook]

> I guess this means that VDE would have to provide a kernel-layer
> component which grabs the packets from eth0 and provides the faked eth0
> for the Host OS...

You can do all this with the standard linux tools. Something like the 
following(untested) script. ifrename is part of the Linus Wireless Tools. If 
your distro doesn't have it you can download it here:
http://www.hpl.hp.com/personal/Jean_Tourrilhes/Linux/Tools.html

Paul

#! /bin/sh
# Take eth0 down so it can be renamed
ifdown eth0
# Rename eth0, and create a new bridge interface called eth0
# This avoids having to change host network configuration.
ifrename -i eth0 -n realeth0
brctl addbr eth0
brctl addif eth0 realeth0
# bring realeth0 up without an IP so the bridge can use it
ifconfig realeth0 0.0.0.0 up
# bring the new eth0 up
ifup eth0
# Start vde, and add it to the bridge.
vde_switch -t tap0
ifconfig tap0 0.0.0.0 up
brctl addif eth0 tap0

Re: [Qemu-devel] Connecting vde and LAN

[Herbert Poetzl]

On Thu, Aug 11, 2005 at 06:00:12PM +0100, Paul Brook wrote:
> > I guess this means that VDE would have to provide a kernel-layer
> > component which grabs the packets from eth0 and provides the faked eth0
> > for the Host OS...
> 
> You can do all this with the standard linux tools. Something like the 
> following(untested) script. ifrename is part of the Linus Wireless Tools. If 
> your distro doesn't have it you can download it here:
> http://www.hpl.hp.com/personal/Jean_Tourrilhes/Linux/Tools.html

and nameif is a lot older, part of net-tools and
basically installed on every machine which has
ifconfig ... hmm, scratch the last part, it is
installed on every machine :)

HTH,
Herbert

> Paul
> 
> #! /bin/sh
> # Take eth0 down so it can be renamed
> ifdown eth0
> # Rename eth0, and create a new bridge interface called eth0
> # This avoids having to change host network configuration.
> ifrename -i eth0 -n realeth0
> brctl addbr eth0
> brctl addif eth0 realeth0
> # bring realeth0 up without an IP so the bridge can use it
> ifconfig realeth0 0.0.0.0 up
> # bring the new eth0 up
> ifup eth0
> # Start vde, and add it to the bridge.
> vde_switch -t tap0
> ifconfig tap0 0.0.0.0 up
> brctl addif eth0 tap0
> 
> 
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel

Re: [Qemu-devel] Connecting vde and LAN

[Henrik Nordstrom]

On Thu, 11 Aug 2005, Oliver Gerlich wrote:

> Couldn't we avoid these incompatibilities if we would route packets only on 
> the Ethernet level? If the Qemu networking setup on the host involves IP 
> addresses or such things, we're already on the wrong OSI layer I think...

>From what I can tell there is no way userspace can send Ethernet frames to 
the host as if coming from the eth0 interface, short of writing a kernel 
module to make this magics happen.

> I'm not very experienced in networking, but IMHO the network should be set up 
> like this:
>
>                                    (eth0 on Host OS)
>                                   /
> ( LAN ) <----> (real eth0) <-> VDE
>                                   \
>                                    (NIC on Guest OS)

This is what you do with bridgeing.

                          (br0 on Host OS)
                          /
( LAN ) <---> eth0 <-> bridge
                          \
                          (tap0 to VDE/qemu)


Hmm.. actually the bridge setup can be simplified somewhat on Linux hosts 
by using an early boot script run before the eth0 is initialized, renaming 
eth0 to something else and creating a bridge named eth0, including the 
renamed eth0 interface..


    modprobe eth0
    ip link set eth0 name eth0real
    brctl addbr eth0
    brctl addif eth0 eth0real
    ip link set eth0real up


> So this is a summary of what I think we should try to achieve (sorted by 
> priority):
>
> -provide Ethernet-layer access from the Guest to the LAN (so that it's 
> transparent for IP traffic)

Done.

> -allow using a LAN-wide DHCP server for the Guest

Done by the above.

> -the host should be able to set up his LAN interface over the LAN-wide DHCP 
> server.

Done.

> -the host networking still works as before (at least on the layers above 
> Ethernet)

Done.

> -the whole thing is easy to set up

Done.

> -the whole thing can be started by any user at any time (so it's not started 
> at boot time, but whenever a user starts Qemu).

Requires root rights.

The restrictions to the above is that the guest can not reach the host in 
such setup.

Regards
Henrik

Re: [Qemu-devel] Connecting vde and LAN

[Henrik Nordstrom]

On Thu, 11 Aug 2005, Jim C. Brown wrote:

> Incidently, if you have eth0 and eth1, and both are connected to the same
> LAN, then it works. Just set up vde_packet/pcacp on eth1, then packed tp the
> host from the VDE will be sent out over eth1 and recieved back in eth0.

Right. Just don't set an IP on eth1 (only link up, no addressing). Frames 
sent by the guest to the host will travel eth0 -> switch -> eth1.

> This solves the problem quite nicely, and it is probably the simplest to
> implement, but requires changing the hardware.
>
> I'm trying to figure out how to achieve the same effect with eth0 and tap0
> (as opposed to eth0 and eth1).

Not easily done as you are then missing the switch component looping the 
frames back to eth0.

> If we had a), then vde_pcap would "just work" - there'd be no need to fiddle
> with faking things on tap devices or etc.

Not fully. Without having the bridge component in the mix frames sent by 
the guest to the host is likely to be echoed out on the LAN.

> Most of these are my goals as well (except I don't mind having to run a bit of
> glue in a startup script).

If you are prepared to run a bit of glue in system startup scripts then 
use bridgeing with a tap device. This provides the absolutely cleanest 
networking capabilities.

The setup I am running is working very well, with only a very small amount 
of glue required in the Linux system startup. As already shown in this 
thread such glue can be minimized to almost no changes on the host.

By using a persistent tap device the whole thing can even be run in such 
manner that all the userspace components (be this vde, or qemu directly) 
can run as the user when needed. No root rights required.

Regards
Henrik

Re: [Qemu-devel] Connecting vde and LAN

[Jim C. Brown]

On Fri, Aug 12, 2005 at 12:02:46PM +0200, Henrik Nordstrom wrote:
> >This solves the problem quite nicely, and it is probably the simplest to
> >implement, but requires changing the hardware.
> >
> >I'm trying to figure out how to achieve the same effect with eth0 and tap0
> >(as opposed to eth0 and eth1).
> 
> Not easily done as you are then missing the switch component looping the 
> frames back to eth0.
> 

Well, I'm also comparing this to, say, a user space bridge of tap0 and tap1.
For that, we are also missing the switch component, but this is easy to solve.
The user space component simply needs the file descriptors of both tap0 and
tap1, then when it receives a packet on tap0 it can simply write it to tap1 and
vice versa. Since writing a packet to the file descriptor of a tap device
causes the upper layer protocol layers to take a look at it, the issue doesn't
exist here.

>From a brief look at the kernel code, it looks like netif_rx() can be used to
achieve the same effect for a real ethernet card. Packets that are recieved
by the device driver are queued with netif_rx() to be interpreted by the
host OS protocol stack, so if vde_pcap could call netif_rx() directly then it
could inject packets at the ethernet frame level to the host OS directly.

Unfortuantly vde_pcap is a userland program and netif_rx() is a kernel function,
so unless netif_rx() is exposed thru a procfs interface or something similar
there is no way to take advantage of it.

> >If we had a), then vde_pcap would "just work" - there'd be no need to 
> >fiddle
> >with faking things on tap devices or etc.
> 
> Not fully. Without having the bridge component in the mix frames sent by 
> the guest to the host is likely to be echoed out on the LAN.
> 

Ideally, packets destined for the host will end up in the OS's protocol stack
and stay there, while packets destined elsewhere will actually go thru the
NIC and into the LAN.

Of course, vde_pcap could write packets for the host via netif_rx() and send
the rest thru the packet socket. AFAICS packets sent thru netif_rx() will not
be echoed back out the device again.

> >Most of these are my goals as well (except I don't mind having to run a 
> >bit of
> >glue in a startup script).
> 
> If you are prepared to run a bit of glue in system startup scripts then 
> use bridgeing with a tap device. This provides the absolutely cleanest 
> networking capabilities.
> 

That is very true.

> 
> Regards
> Henrik
> 

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.