Re: [B.A.T.M.A.N.] wired only 10 gigabit batman-adv mesh

[Marek Lindner <mareklindner@neomailbox.ch>]

[-- Attachment #1: Type: text/plain, Size: 3783 bytes --]


Hi,

>                  Good Afternoon from Boston. I really love Batman-Adv ...
> brilliant layer 2 functionality.
> 
> I want to use batman-adv in a wired (gigabit and 10-gigabit) only mesh and
> wanted to know your insights.

makes me happy to hear you love our project. Typically, we communicate via our 
public mailing list allowing various sources to chime in at any point. Since I 
don't see any reason for privacy I am cc'ing the mailing list in my answer.


> The example case scenario is as follows:
> 
> 1)     4 to 6 AMD servers with 6 10-Gigabit NICs each.
> 
> 2)     2 or 3 10-Gigabit NICs used for batman-adv, which are then connected
> in ring or torus topology directly (no external switch involved)
> 
> 3)     the remaining interfaces on the server are connected to the LAN
> (switches, routers etc)
> 
> 4)     the virtual machine (qemu-kvm) tap interfaces, the physical
> non-batman-adv ethernet and bat0 interfaces are put in a bridge (brctl), so
> now we have the ability for virtual machines, wired hosts on the lan to go
> via batman-adv and talk to each other.
> 
> Is there any, down size to doing this? I see at the most 2 - 100 servers in
> one network....
> 
> From what i understand:
> 
> 1)  that the live migration of virtual machines (qemu-kvm) will be seen
> just as a migrating non-mesh client so my assumption is that live migration
> should work from that perspective. Also, what if the tap interfaces of the
> virtual machines are given to bat0 itself (if it might help in live
> migration / increasing throughput) ?
> 
> 2) The MTU if set for 1500 or 9000 or higher (eg barman-adv reads --
> "define ETHERMTU ETH_DATA_LEN") would be taken automatically by batman-adv
> and anything below 1500 would be fragmented, which gives me the idea that
> higher MTUs would not be a problem for batman-adv to handle.
> 
> 3)  There is no restriction to the number of clients in batman-adv.
> 
> am i somewhat close in understanding batman-adv? .... apologies if not...
> 
> Also would layer 2 forwarding by batman-adv would be close, same or better
> when compared to bridge (linux brctl) packet forwarding?
> 
> I have built converged-unified distributed qemu-kvm system (all metadata
> less design, with web-interface and cli, quite the opposite of vmware and
> open-stack type centralized approaches) and was in the preliminary stage of
> looking at the possibility of integrating batman-adv into the design.
> 
> Your input will be valuable for me to give server and desktop
> virtualization a mesh architecture on top of already distributed design.

I keep your description intact to allow other people to comment as well.

Before we dive into the batman-adv details I'd like to understand what 
advantage batman-adv brings to the table in your scenario. The batman-adv 
project aims to facilitate layer2 routing in primarily wireless setups with 
dynamically changing links due to link quality changes or links being modified 
in an uncontrolled fashion (community mesh network). While batman-adv also is 
able to run on wired backbones this never was the main target and bears a 
number of drawbacks compared to other technologies. A simple example to 
picture this: The standard Linux bridge (configurable via brctl) does not run 
any link layer protocol to estimate the quality of one link compared to 
another. This will give you huge advantages in terms of overhead with the cost 
of all links being treated equal. While this work fine on an all-wired setup 
it represents an unacceptable trade-off for wireless networks.

From what I can gather you are not running wireless but high throughput wired 
links. What has brought you to batman-adv ?

Cheers,
Marek

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 473 bytes --]