[B.A.T.M.A.N.] Testing example of interface bonding

[B.A.T.M.A.N.] Testing example of interface bonding

[Ray Gibson]

Hello,

Are there any documented cases (aside from the aging graph on the
wiki) on batman-adv bonding setups?

I'm testing batman-adv 2014.3.0 in an effort to experiment and test
with multi-link optimizations.  There is no wifi in the picture at the
moment, this is being done with tap interfaces right now.

My two nodes have this network config:
lan0: bridge of eth0 and bat0, 192.168.100.1 on node1, 192.168.100.2 on node2
bat0: includes tap0 and tap1 active, no IP.
eth1: "wan" link, 10.10.10.1 on node1, 10.10.10.2 on node2
tap0/1: openvpn bridge links over eth1, no IPs assigned to these interfaces.

This is a VMware environment for testing.  With the above setup, I can
ping/iperf/whatever back and forth on the 192.168.100.x network.  It
works great just extending a lan transparently on a wan link.  This
was the original idea, with redundant connections (hence the multi tap
interfaces).

Now, I'm trying to isolate the test down to bonding.  However,
enabling bonding in batctl on both nodes has no apparent effect
whatsoever.  Watching an iperf session in ifstat, I will see constant
traffic on bat0, and then either tap0 or tap1 depending on how it's
feeling at the time.  Sometimes it will shift all the traffic to the
other tap interface.  Sometimes the incoming traffic will be on one
interface and the outgoing traffic will be on the other.

Note: I'm running iperf/etc on the nodes themselves, not on separate
devices on the lan bridge.

Ultimately I am looking to try bonding several 3G or 4G devices
together with batman-adv to achieve higher throughput to a single
destination.

Thank you for any comments or suggestions.  I find batman-adv very
useful in other projects and look to extend its use into this one.

Ray

Re: [B.A.T.M.A.N.] Testing example of interface bonding

[Simon Wunderlich]

Hello Ray,

thanks a lot for your mail.

On Sunday 10 August 2014 15:00:51 Ray Gibson wrote:
> Hello,
> 
> Are there any documented cases (aside from the aging graph on the
> wiki) on batman-adv bonding setups?

Not as far as I know. The bonding feature does not seem to be the most popular 
one. :) (If there is anyone, please tell us!)

> 
> I'm testing batman-adv 2014.3.0 in an effort to experiment and test
> with multi-link optimizations.  There is no wifi in the picture at the
> moment, this is being done with tap interfaces right now.
> 
> My two nodes have this network config:
> lan0: bridge of eth0 and bat0, 192.168.100.1 on node1, 192.168.100.2 on
> node2 bat0: includes tap0 and tap1 active, no IP.
> eth1: "wan" link, 10.10.10.1 on node1, 10.10.10.2 on node2
> tap0/1: openvpn bridge links over eth1, no IPs assigned to these interfaces.
> 
> This is a VMware environment for testing.  With the above setup, I can
> ping/iperf/whatever back and forth on the 192.168.100.x network.  It
> works great just extending a lan transparently on a wan link.  This
> was the original idea, with redundant connections (hence the multi tap
> interfaces).
> 
> Now, I'm trying to isolate the test down to bonding.  However,
> enabling bonding in batctl on both nodes has no apparent effect
> whatsoever.  Watching an iperf session in ifstat, I will see constant
> traffic on bat0, and then either tap0 or tap1 depending on how it's
> feeling at the time.  Sometimes it will shift all the traffic to the
> other tap interface.  Sometimes the incoming traffic will be on one
> interface and the outgoing traffic will be on the other.
> 
> Note: I'm running iperf/etc on the nodes themselves, not on separate
> devices on the lan bridge.

Hmm. What you should see after enabling bonding would be a similar usage of 
both interfaces. I guess each tap interfaces of node1 is "directly connected" 
to  the other tap interface of node2, right? Could you please share your 
outputs of:

batctl originators
batctl originators -i tap0
batctl originators -i tap1

> 
> Ultimately I am looking to try bonding several 3G or 4G devices
> together with batman-adv to achieve higher throughput to a single
> destination.

Please note that the bonding will only benefit under some circumstances, as far 
as my experiments have shown:

 * since its round robin, you'll only see a benefit if the worst link does not 
have less than 50% throughput of the best one - otherwise it will slow the 
other links down.
 * different latencies or buffering delays in the links may lead to out of order 
packets, and not every payload traffic likes that.

I could see some improvement when having two equal wifi links though. In any 
case, please thoroughly test it before applying that to your 3g/4g 
application, there may be some pitfalls. I'd be also very interested in your 
findings. :)

Thanks,
    Simon

Re: [B.A.T.M.A.N.] Testing example of interface bonding

[Ray Gibson]

Simon,

Thank you for looking at this with me.

On Mon, Aug 11, 2014 at 2:38 PM, Simon Wunderlich <sw@simonwunderlich.de> wrote:

> Hmm. What you should see after enabling bonding would be a similar usage of
> both interfaces. I guess each tap interfaces of node1 is "directly connected"
> to  the other tap interface of node2, right? Could you please share your
> outputs of:

Yes, the tap interfaces behave as if they are directly connected together.

> batctl originators
> batctl originators -i tap0
> batctl originators -i tap1

I've put all the output in a pastebin as to not clog up everyone's inbox:

http://pastebin.com/6fqUUUYq

> Please note that the bonding will only benefit under some circumstances, as far
> as my experiments have shown:
>
>  * since its round robin, you'll only see a benefit if the worst link does not
> have less than 50% throughput of the best one - otherwise it will slow the
> other links down.
>  * different latencies or buffering delays in the links may lead to out of order
> packets, and not every payload traffic likes that.
>
> I could see some improvement when having two equal wifi links though. In any
> case, please thoroughly test it before applying that to your 3g/4g
> application, there may be some pitfalls. I'd be also very interested in your
> findings. :)
>
> Thanks,
>     Simon

Regardless of its potential impact on performance, I think it's worth
a look and test to see if there may be some potential benefit.
Currently this setup is working nice in that it's a true case of point
to point redundant links.  If additional links could be added or
removed at will to enhance throughput I would see this useful in a
variety of applications.  Currently, I'd like to just reproduce what
the function was originally designed to do and then go from there.

Thanks again for the help.

Ray

Re: [B.A.T.M.A.N.] Testing example of interface bonding

[Simon Wunderlich]

[-- Attachment #1: Type: text/plain, Size: 2045 bytes --]

Ray, 

On Tuesday 12 August 2014 11:21:18 Ray Gibson wrote:
> 
> I've put all the output in a pastebin as to not clog up everyone's inbox:
> 
> http://pastebin.com/6fqUUUYq
> 

Thanks! The output actually looks fine. I've tried a little bit myself in a few 
VMs and noticed two problems which were preventing bonding to work, at least 
for me:

 * There was a trivial (and rather stupid) logic bug in the check whether 
bonding should be considered or not. See "[PATCH] batman-adv: fix and simplify 
condition when bonding should be used"
 * Another not so trivial problem is that bonding with the multi interface 
optimization checks the originator tables for the different interfaces. However 
since you don't use WiFi and all links appear perfect, the same router is 
often chosen for the different interfaces and the bonding has not enough 
candidates to choose and do round robin. I've sent an experimental patch 
"[RFC] batman-adv: experimental sysfs variable to always apply half duplex 
penalty" which allows to turn on the interface / half duplex penalty for all 
interfaces, not just wifi interfaces. That patch will most probably not make it 
upstream, but should be good enough for testing. Enable it with:

echo 1 > /sys/devices/virtual/net/bat0/mesh/always_half_duplex_penalty

Please apply these two patches, set the new option and try again!

> [...]
> 
> Regardless of its potential impact on performance, I think it's worth
> a look and test to see if there may be some potential benefit.
> Currently this setup is working nice in that it's a true case of point
> to point redundant links.  If additional links could be added or
> removed at will to enhance throughput I would see this useful in a
> variety of applications.  Currently, I'd like to just reproduce what
> the function was originally designed to do and then go from there.

OK, sure, it can't hurt to try it out - maybe you can even find ways to 
optimize it. In any case, we would be happy to hear about your test results 
and experiences. :)

Thanks,
    Simon

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 181 bytes --]