bridges

bridges

[Ross Boylan]

I'm trying to understand bridging with KVM, but am still puzzled.
I think that the recommended bridging with TAP means that packets from
the VM will end up going out the host card attached to the default
gateway.  But it looks to me as if their IP address is unchanged, which
means replies will never reach me.  Is that correct?  Do I need to NAT
the packets, or is something already doing that?

Some documents indicate that I need to bring the interfaces (e.g., eth0)
down before I bring the bridge up, and that afterwards only the bridge
will have an IP address.  Is that right?

Some documents, e.g.,
http://ebtables.sourceforge.net/br_fw_ia/br_fw_ia.html, indicate
iptables should "just work" with bridging.  However, I've seen someone
with a 2.6.15 kernel ask about firewalling and be told they needed to
patch the kernel to get it work (don't have the reference handy).
Should it just work?

I'm running a 2.6.29 kernel on Debian Lenny with kvm 72+dfsg-5~lenny1.
Version 84+dfsg-2 is available in experimental.  Is there much to be
gained by going with the more recent version?

Please cc me; I'm not on the list.

Thanks.
Ross Boylan

Re: bridges

[Ross Boylan]

On Thu, 2009-05-07 at 11:13 -0600, Cam Macdonell wrote:
> 
> Ross Boylan wrote:
> > I'm trying to understand bridging with KVM, but am still puzzled.
> > I think that the recommended bridging with TAP means that packets
> from
> > the VM will end up going out the host card attached to the default
> > gateway.  But it looks to me as if their IP address is unchanged,
> which
> > means replies will never reach me.  Is that correct?  Do I need to
> NAT
> > the packets, or is something already doing that?
> 
> Hi Ross,
> 
> This is the place to start
> 
> http://www.linux-kvm.org/page/Networking.  
I saw that; it gives some recipes but I wasn't sure what their effect
was.

> You want a public bridge.
> 
> I'm not sure what "their" and "me" mean in your email.  In short,
> with 
> bridging each VM has its own IP and that VM can be accessed directly 
> from the network.
"their" = the VM.
"me" = my host machine.

So if the VM's are running on their own subnet, e.g., 10.0.2.* (I've
been assuming the subnet with TAP is like the one with the User Mode
Network stack in 3.7.3 of http://www.nongnu.org/qemu/qemu-doc.html) and
my host machine is on another net, e.g., 10.0.8.* then I think the
packet will go out with an IP of 10.0.2.2 (say).  When some other
machine tries to reply to 10.0.2.2, the packet gets lost because the
outside network thinks 10.0.2.* is not for it.  At least that's my
concern.  If the return IP address on the packet were 10.0.8.44
(supposing that's the IP of my host machine) then the packets could find
their way back.

My host machine is on an internal network with a 10.* IP.  The example
might be clearer if one supposed that the VM's were on a 192.168.*
network.

I am perhaps being influenced by the fact that I don't want to ask for
more IP's, so I don't want to configure the VM's to use an IP on our
10.0.8 network.

Does the TAP networking setup a whole subnet like the user mode network
stack (e.g., running a DHCP server), or is the idea that I would just
give the VM an IP on my subnet (10.0.8.*) in this example?

If the latter is the case (I'm now suspecting it is) then I think the
solution is clear.  I just stick the VM's on a private (to my machine)
subnet, like 192.168.*, and I do NAT on the packets as they go out.

Re: bridges

[Cam Macdonell]

Ross Boylan wrote:
> On Thu, 2009-05-07 at 11:13 -0600, Cam Macdonell wrote:
>> Ross Boylan wrote:
>>> I'm trying to understand bridging with KVM, but am still puzzled.
>>> I think that the recommended bridging with TAP means that packets
>> from
>>> the VM will end up going out the host card attached to the default
>>> gateway.  But it looks to me as if their IP address is unchanged,
>> which
>>> means replies will never reach me.  Is that correct?  Do I need to
>> NAT
>>> the packets, or is something already doing that?
>> Hi Ross,
>>
>> This is the place to start
>>
>> http://www.linux-kvm.org/page/Networking.  
> I saw that; it gives some recipes but I wasn't sure what their effect
> was.
> 
>> You want a public bridge.
>>
>> I'm not sure what "their" and "me" mean in your email.  In short,
>> with 
>> bridging each VM has its own IP and that VM can be accessed directly 
>> from the network.
> "their" = the VM.
> "me" = my host machine.
> 
> So if the VM's are running on their own subnet, 

VMs do not run on their own subnet with bridged networking.

>e.g., 10.0.2.* (I've
> been assuming the subnet with TAP is like the one with the User Mode
> Network stack in 3.7.3 of http://www.nongnu.org/qemu/qemu-doc.html) and
> my host machine is on another net, e.g., 10.0.8.* then I think the
> packet will go out with an IP of 10.0.2.2 (say).  When some other
> machine tries to reply to 10.0.2.2, the packet gets lost because the
> outside network thinks 10.0.2.* is not for it.  At least that's my
> concern.  If the return IP address on the packet were 10.0.8.44
> (supposing that's the IP of my host machine) then the packets could find
> their way back.

Using bridged networking is very different from the user stack.  The 
user stack is extremely limited and slow.

> 
> My host machine is on an internal network with a 10.* IP.  The example
> might be clearer if one supposed that the VM's were on a 192.168.*
> network.
> 
> I am perhaps being influenced by the fact that I don't want to ask for
> more IP's, so I don't want to configure the VM's to use an IP on our
> 10.0.8 network.

Then you probably want to use a NAT network.  A NAT setup puts all the 
VMs on their own network within the host machine.  iptables is necessary 
to forward the subnet packets out to the world and back.

Here is some older documentation, but not much has changed.  Look at the 
first entry under "Advanced Networking".

https://help.ubuntu.com/community/KVMFeisty

> Does the TAP networking setup a whole subnet like the user mode network
> stack (e.g., running a DHCP server), or is the idea that I would just
> give the VM an IP on my subnet (10.0.8.*) in this example?

No, bridge networking using taps (one tap per VM) and effectively sits 
all the VMs on the same network your host is on.  You would need to get 
IPs from sysadmin for each VM.

> If the latter is the case (I'm now suspecting it is) then I think the
> solution is clear.  I just stick the VM's on a private (to my machine)
> subnet, like 192.168.*, and I do NAT on the packets as they go out.

NAT is a very common solution.  Use VDE (vde.sourceforget.net) to create 
a virtual switch on your host for the VMs.  dnsmasq can serve dynamic 
IPs to the VMs on their own subnet that doesn't bother your sysadmin at 
all.  Use iptables to forward and receive packets through your host's 
NIC.

Cam

Re: bridges

[Matthew Palmer]

On Thu, May 07, 2009 at 08:57:03AM -0700, Ross Boylan wrote:
> I'm trying to understand bridging with KVM, but am still puzzled.
> I think that the recommended bridging with TAP means that packets from
> the VM will end up going out the host card attached to the default
> gateway.  But it looks to me as if their IP address is unchanged, which
> means replies will never reach me.  Is that correct?  Do I need to NAT
> the packets, or is something already doing that?
> 
> Some documents indicate that I need to bring the interfaces (e.g., eth0)
> down before I bring the bridge up, and that afterwards only the bridge
> will have an IP address.  Is that right?

Here's how I think of a Linux "soft" bridge: the bridge consists of an
Ethernet switch, and a regular interface (named after the bridge) that is
connected to that switch.  This is why you "give an IP address to the
bridge", because "the bridge" is also a NIC of it's own.

If you attach any physical interfaces (eg ethN) to the bridge, they aren't
NICs any more, they're just network cables you plug into the switch to pass
traffic to other switches.  Attaching VMs to the switch is just hooking up
more cables between the switch and the VMs.

If you want your host to do NAT for your VMs, then you do as you would for
any other firewall -- you have one switch (the bridge, in this case) with
all of your VMs and the "internal" interface of the host (in this case, the
bridge as well) all plugged in, and then a second interface to the outside
world (the physical NIC).

> Some documents, e.g.,
> http://ebtables.sourceforge.net/br_fw_ia/br_fw_ia.html, indicate
> iptables should "just work" with bridging.

Yes, iptables *does* "just work" with bridging, in the sense that iptables
can still filter IP packets passing through it's interfaces.  What you
*can't* do, though, is have some sort of magic iptables filter deep in the
bridge that plays with all traffic as it traverses.  For that, there's
ebtables, which is iptables but for Ethernet (rather than IP) traffic. 
Personally, I've never used ebtables in my life.

> However, I've seen someone
> with a 2.6.15 kernel ask about firewalling and be told they needed to
> patch the kernel to get it work (don't have the reference handy).
> Should it just work?

It should Just Work, and if you've got to patch any 2.6 (or even probably
2.4) kernel then you're doing something *very* esoteric.

- Matt