[B.A.T.M.A.N.] Multiple radios to improve throughput

[B.A.T.M.A.N.] Multiple radios to improve throughput

[Shane Chao]

Hello,

I have a 3 node mesh network with layout as follows:

   node #1       node #2        node #3
192.168.2.1   192.168.2.5    192.168.2.9

Each node has only one radio and #1 talks to #3 via #2 and vice versa.
If I want to improve the throughput between #1 and #3, logically I would
add another radio to #2 so it can communicate to both #1 and #3
simultaneously w/o having the single radio switching back and forth
between #1 and #3.

However, I ran IPERF between #1 and #3 before and after adding the
second radio to node #2 and the bandwidth didn't really improve.
In my node #2 configuration, I did not bridge the two wifi interfaces so
I have ath0 and ath1 with their own IP addresses.  All three nodes are
configured with the same channel.  My batmand command line for node #2
is:  batmand ath0 ath1.

Can anyone advise me on what I did wrong?

Thanks,
Shane

Re: [B.A.T.M.A.N.] Multiple radios to improve throughput

[elektra]

Hi Shane -

if all radios are operating on the same channel you can't expect any 
improvement. So operate one link on channel 1 and the other on channel 
11, 13 or 14 according to the regulations in your country. Channels in 
the 5 GHz range are even better ;-)

Note also that many embedded devices have not enough CPU power to 
saturate the capacity of a single radio link operating at full speed. At 
54 Mbit a single link can have approx. 3 MByte/sec throughput, but we 
have experienced that a 200 MHz MIPS CPU doesn't provide enough power 
for more than 1.2 to 1.6 MByte/sec with a single interface depending on 
the chipset/type of your wifi card.

cu elektra

 
> Hello,
>
> I have a 3 node mesh network with layout as follows:
>
>    node #1       node #2        node #3
> 192.168.2.1   192.168.2.5    192.168.2.9
>
> Each node has only one radio and #1 talks to #3 via #2 and vice versa.
> If I want to improve the throughput between #1 and #3, logically I would
> add another radio to #2 so it can communicate to both #1 and #3
> simultaneously w/o having the single radio switching back and forth
> between #1 and #3.
>
> However, I ran IPERF between #1 and #3 before and after adding the
> second radio to node #2 and the bandwidth didn't really improve.
> In my node #2 configuration, I did not bridge the two wifi interfaces so
> I have ath0 and ath1 with their own IP addresses.  All three nodes are
> configured with the same channel.  My batmand command line for node #2
> is:  batmand ath0 ath1.
>
> Can anyone advise me on what I did wrong?
>
> Thanks,
> Shane
> _______________________________________________
> B.A.T.M.A.N mailing list
> B.A.T.M.A.N@open-mesh.net
> https://list.open-mesh.net/mm/listinfo/b.a.t.m.a.n
>
>   

Re: [B.A.T.M.A.N.] Multiple radios to improve throughput

[David Murray]

Are both of your radios using 2.4 GHz channels? How close are they and 
are you using external antennas?

This paper [1] p13 and this paper [2] p6 both talk about an inter radio 
interference issue when using multiple radios on one node.

Dave

[1] http://citeseer.ist.psu.edu/raniwala04centralized.html
[2] http://research.microsoft.com/research/pubs/view.aspx?tr_id=667

Quoting elektra <onelektra@gmx.net>:

> Hi Shane -
>
> if all radios are operating on the same channel you can't expect any 
> improvement. So operate one link on channel 1 and the other on 
> channel 11, 13 or 14 according to the regulations in your country. 
> Channels in the 5 GHz range are even better ;-)
>
> Note also that many embedded devices have not enough CPU power to 
> saturate the capacity of a single radio link operating at full speed. 
> At 54 Mbit a single link can have approx. 3 MByte/sec throughput, but 
> we have experienced that a 200 MHz MIPS CPU doesn't provide enough 
> power for more than 1.2 to 1.6 MByte/sec with a single interface 
> depending on the chipset/type of your wifi card.
>
> cu elektra
>
>
>> Hello,
>>
>> I have a 3 node mesh network with layout as follows:
>>
>>    node #1       node #2        node #3
>> 192.168.2.1   192.168.2.5    192.168.2.9
>>
>> Each node has only one radio and #1 talks to #3 via #2 and vice versa.
>> If I want to improve the throughput between #1 and #3, logically I would
>> add another radio to #2 so it can communicate to both #1 and #3
>> simultaneously w/o having the single radio switching back and forth
>> between #1 and #3.
>>
>> However, I ran IPERF between #1 and #3 before and after adding the
>> second radio to node #2 and the bandwidth didn't really improve.
>> In my node #2 configuration, I did not bridge the two wifi interfaces so
>> I have ath0 and ath1 with their own IP addresses.  All three nodes are
>> configured with the same channel.  My batmand command line for node #2
>> is:  batmand ath0 ath1.
>>
>> Can anyone advise me on what I did wrong?
>>
>> Thanks,
>> Shane
>> _______________________________________________
>> B.A.T.M.A.N mailing list
>> B.A.T.M.A.N@open-mesh.net
>> https://list.open-mesh.net/mm/listinfo/b.a.t.m.a.n
>>
>>
>
> _______________________________________________
> B.A.T.M.A.N mailing list
> B.A.T.M.A.N@open-mesh.net
> https://list.open-mesh.net/mm/listinfo/b.a.t.m.a.n
>

[B.A.T.M.A.N.] Multiple radios to improve throughput

[Shane Chao]

Elektra,

Thank you for pointing out in May 08 that most embedded boards do not 
have enough CPU power to saturate the capacity of single radio link.  I 
was experimenting Batman in a straight line, three node configuration 
with the middle node having two radios so it doesn't have to switch 
between node 1 and node 3.  My configuration is as follows:

node 1          node 2          node 3
------          ------          ------
  ath0 <--ch 1--> ath0
                  ath1 <--ch 11-> ath0

When I ran iperf between node 1 and 3, I did not see any throughput 
improvements with either one or two radios in node 2. I even stopped 
Batman and manually setup the routes and the performance remains the 
same.  However, when I upgrade node 2 from a 200MHz CPU board to a 
500MHz CPU board, the bandwidth went up 50%.  So thanks again for 
pointing out my bottleneck:)

Lastly, can someone suggest some embedded boards fast enough to push 
wifi radios to its limits?

Thanks,
Shane

RE: [B.A.T.M.A.N.] Multiple radios to improve throughput

[Predrag Balorda]

Now, is this CPU bottleneck because of batman and routing or something else?
(cannot forward packets fast enough?)
11mbits isn't really that much and I still have to see my setup hit that figure
between only two nodes, let alone multi-hop.

Now that we've played around a bit can someone please provide some benchmarks of
a 5-node network (with some multi-hop paths) with batman and with just the whole
network bridged and on the same subnet.

What sort of performance should you expect in either scenario?

As I said before, I used to get 1MB/s with non-routed bridged network of two
atheros devices but ever since I started using batman, 300KB/s is the most I was
ever able to achieve between the same two nodes. 2 hops and it goes down to
40KB/s.

> -----Original Message-----
> From: b.a.t.m.a.n-bounces@open-mesh.net [mailto:b.a.t.m.a.n-bounces@open-
> mesh.net] On Behalf Of Shane Chao
> Sent: Tuesday, June 03, 2008 1:34 AM
> To: b.a.t.m.a.n@open-mesh.net
> Subject: [B.A.T.M.A.N.] Multiple radios to improve throughput
> 
> Elektra,
> 
> Thank you for pointing out in May 08 that most embedded boards do not
> have enough CPU power to saturate the capacity of single radio link.  I
> was experimenting Batman in a straight line, three node configuration
> with the middle node having two radios so it doesn't have to switch
> between node 1 and node 3.  My configuration is as follows:
> 
> node 1          node 2          node 3
> ------          ------          ------
>   ath0 <--ch 1--> ath0
>                   ath1 <--ch 11-> ath0
> 
> When I ran iperf between node 1 and 3, I did not see any throughput
> improvements with either one or two radios in node 2. I even stopped
> Batman and manually setup the routes and the performance remains the
> same.  However, when I upgrade node 2 from a 200MHz CPU board to a
> 500MHz CPU board, the bandwidth went up 50%.  So thanks again for
> pointing out my bottleneck:)
> 
> Lastly, can someone suggest some embedded boards fast enough to push
> wifi radios to its limits?
> 
> Thanks,
> Shane
> _______________________________________________
> B.A.T.M.A.N mailing list
> B.A.T.M.A.N@open-mesh.net
> https://list.open-mesh.net/mm/listinfo/b.a.t.m.a.n

RE: [B.A.T.M.A.N.] Multiple radios to improve throughput

[Predrag Balorda]

I believe the new routerboard 433 is supposed to handle that with its 600 (or
was it 800?) MHz cpu (atheros). But then you never know...

> -----Original Message-----
> From: b.a.t.m.a.n-bounces@open-mesh.net [mailto:b.a.t.m.a.n-bounces@open-
> mesh.net] On Behalf Of Shane Chao
> Sent: Tuesday, June 03, 2008 1:34 AM
> To: b.a.t.m.a.n@open-mesh.net
> Subject: [B.A.T.M.A.N.] Multiple radios to improve throughput
> 
> Elektra,
> 
> Thank you for pointing out in May 08 that most embedded boards do not
> have enough CPU power to saturate the capacity of single radio link.  I
> was experimenting Batman in a straight line, three node configuration
> with the middle node having two radios so it doesn't have to switch
> between node 1 and node 3.  My configuration is as follows:
> 
> node 1          node 2          node 3
> ------          ------          ------
>   ath0 <--ch 1--> ath0
>                   ath1 <--ch 11-> ath0
> 
> When I ran iperf between node 1 and 3, I did not see any throughput
> improvements with either one or two radios in node 2. I even stopped
> Batman and manually setup the routes and the performance remains the
> same.  However, when I upgrade node 2 from a 200MHz CPU board to a
> 500MHz CPU board, the bandwidth went up 50%.  So thanks again for
> pointing out my bottleneck:)
> 
> Lastly, can someone suggest some embedded boards fast enough to push
> wifi radios to its limits?
> 
> Thanks,
> Shane
> _______________________________________________
> B.A.T.M.A.N mailing list
> B.A.T.M.A.N@open-mesh.net
> https://list.open-mesh.net/mm/listinfo/b.a.t.m.a.n

Re: [B.A.T.M.A.N.] Multiple radios to improve throughput

[Benjamin Henrion]

Shane Chao <schao@changind.com> [080603]:
> Elektra,
> 
> Thank you for pointing out in May 08 that most embedded boards do not 
> have enough CPU power to saturate the capacity of single radio link.  I 
> was experimenting Batman in a straight line, three node configuration 
> with the middle node having two radios so it doesn't have to switch 
> between node 1 and node 3.  My configuration is as follows:
> 
> node 1          node 2          node 3
> ------          ------          ------
>  ath0 <--ch 1--> ath0
>                  ath1 <--ch 11-> ath0
> 
> When I ran iperf between node 1 and 3, I did not see any throughput 
> improvements with either one or two radios in node 2. I even stopped 
> Batman and manually setup the routes and the performance remains the 
> same.  However, when I upgrade node 2 from a 200MHz CPU board to a 
> 500MHz CPU board, the bandwidth went up 50%.  So thanks again for 
> pointing out my bottleneck:)
> 
> Lastly, can someone suggest some embedded boards fast enough to push 
> wifi radios to its limits?

Make a test with 2 atheros minipci cards on a Alix 3C3 (500mhz) and try
to find out if the CPU is saturated.

--
Benjamin Henrion <bhenrion@ffii.org>
FFII Brussels - +32-484-566109 - +32-2-4148403

Re: [B.A.T.M.A.N.] Multiple radios to improve throughput

[cao jing]

What's the highest speed you get when you upgrade node 2 from a 200MHz
CPU board to a
500MHz CPU board.

I did the similar experiments on three soekris box. I found if you run
the iperf on the first or last node, the iperf consume a lot cpu time.

Jing

On Tue, Jun 3, 2008 at 7:30 PM, Benjamin Henrion <bh@udev.org> wrote:
> Shane Chao <schao@changind.com> [080603]:
>> Elektra,
>>
>> Thank you for pointing out in May 08 that most embedded boards do not
>> have enough CPU power to saturate the capacity of single radio link.  I
>> was experimenting Batman in a straight line, three node configuration
>> with the middle node having two radios so it doesn't have to switch
>> between node 1 and node 3.  My configuration is as follows:
>>
>> node 1          node 2          node 3
>> ------          ------          ------
>>  ath0 <--ch 1--> ath0
>>                  ath1 <--ch 11-> ath0
>>
>> When I ran iperf between node 1 and 3, I did not see any throughput
>> improvements with either one or two radios in node 2. I even stopped
>> Batman and manually setup the routes and the performance remains the
>> same.  However, when I upgrade node 2 from a 200MHz CPU board to a
>> 500MHz CPU board, the bandwidth went up 50%.  So thanks again for
>> pointing out my bottleneck:)
>>
>> Lastly, can someone suggest some embedded boards fast enough to push
>> wifi radios to its limits?
>
> Make a test with 2 atheros minipci cards on a Alix 3C3 (500mhz) and try
> to find out if the CPU is saturated.
>
> --
> Benjamin Henrion <bhenrion@ffii.org>
> FFII Brussels - +32-484-566109 - +32-2-4148403
> _______________________________________________
> B.A.T.M.A.N mailing list
> B.A.T.M.A.N@open-mesh.net
> https://list.open-mesh.net/mm/listinfo/b.a.t.m.a.n
>

RE: [B.A.T.M.A.N.] Multiple radios to improve throughput

[Mark Kelly]

[-- Attachment #1: Type: text/plain, Size: 3181 bytes --]

I too would like to see this benchmark setup.  I have a private mesh
that I've been using batman and olsr on that consists of about 8 nodes
that end up spread out of about a 2000m x 2000m area and right now I'm
only using one radio for the mesh.  I'm pushing video from about 4 of
these nodes and if I try to do more than 2 hops with the video I see
drastic drops in the video performance.

I use Avila Gateworks boards that have 533 MHz Intel Xscale processors
and I've never seen the cpu even close to taxed by batman, olsr, or the
radios, as a point of reference for the original poster.  They also have
a newer board with a 633 Mhz cpu.  

On Tue, 2008-06-03 at 10:24 +0200, Predrag Balorda wrote:

> Now, is this CPU bottleneck because of batman and routing or something else?
> (cannot forward packets fast enough?)
> 11mbits isn't really that much and I still have to see my setup hit that figure
> between only two nodes, let alone multi-hop.
> 
> Now that we've played around a bit can someone please provide some benchmarks of
> a 5-node network (with some multi-hop paths) with batman and with just the whole
> network bridged and on the same subnet.
> 
> What sort of performance should you expect in either scenario?
> 
> As I said before, I used to get 1MB/s with non-routed bridged network of two
> atheros devices but ever since I started using batman, 300KB/s is the most I was
> ever able to achieve between the same two nodes. 2 hops and it goes down to
> 40KB/s.
> 
> > -----Original Message-----
> > From: b.a.t.m.a.n-bounces@open-mesh.net [mailto:b.a.t.m.a.n-bounces@open-
> > mesh.net] On Behalf Of Shane Chao
> > Sent: Tuesday, June 03, 2008 1:34 AM
> > To: b.a.t.m.a.n@open-mesh.net
> > Subject: [B.A.T.M.A.N.] Multiple radios to improve throughput
> > 
> > Elektra,
> > 
> > Thank you for pointing out in May 08 that most embedded boards do not
> > have enough CPU power to saturate the capacity of single radio link.  I
> > was experimenting Batman in a straight line, three node configuration
> > with the middle node having two radios so it doesn't have to switch
> > between node 1 and node 3.  My configuration is as follows:
> > 
> > node 1          node 2          node 3
> > ------          ------          ------
> >   ath0 <--ch 1--> ath0
> >                   ath1 <--ch 11-> ath0
> > 
> > When I ran iperf between node 1 and 3, I did not see any throughput
> > improvements with either one or two radios in node 2. I even stopped
> > Batman and manually setup the routes and the performance remains the
> > same.  However, when I upgrade node 2 from a 200MHz CPU board to a
> > 500MHz CPU board, the bandwidth went up 50%.  So thanks again for
> > pointing out my bottleneck:)
> > 
> > Lastly, can someone suggest some embedded boards fast enough to push
> > wifi radios to its limits?
> > 
> > Thanks,
> > Shane
> > _______________________________________________
> > B.A.T.M.A.N mailing list
> > B.A.T.M.A.N@open-mesh.net
> > https://list.open-mesh.net/mm/listinfo/b.a.t.m.a.n
> 
> _______________________________________________
> B.A.T.M.A.N mailing list
> B.A.T.M.A.N@open-mesh.net
> https://list.open-mesh.net/mm/listinfo/b.a.t.m.a.n
> 

[-- Attachment #2: Type: text/html, Size: 3871 bytes --]

Re: [B.A.T.M.A.N.] Multiple radios to improve throughput

[Benjamin Henrion]

On Thu, Jun 5, 2008 at 2:39 PM, Mark Kelly <mark.kelly@cadre5.com> wrote:
> I too would like to see this benchmark setup.  I have a private mesh that
> I've been using batman and olsr on that consists of about 8 nodes that end
> up spread out of about a 2000m x 2000m area and right now I'm only using one
> radio for the mesh.  I'm pushing video from about 4 of these nodes and if I
> try to do more than 2 hops with the video I see drastic drops in the video
> performance.
>
> I use Avila Gateworks boards that have 533 MHz Intel Xscale processors and
> I've never seen the cpu even close to taxed by batman, olsr, or the radios,
> as a point of reference for the original poster.  They also have a newer
> board with a 633 Mhz cpu.

I would like to see some test of performance with mesh with 2 radios.

Hopping on the same radio as most freifunk network does is a disaster
in terms of performance, because the radio is not full-duplex.

I have bought 12 foneras to make a test with 6 nodes (2 foneras per
node), to see if OLSR chooses the fastest route in terms of
throughput, and it does not. I have to try to play with
LinkQualityMult on the ethernet link.

Does someone has an firmware image with batman for lafonera I can use with ap51?

--
Benjamin Henrion <bhenrion at ffii.org>
FFII Brussels - +32-484-566109 - +32-2-4148403

Re: [B.A.T.M.A.N.] Multiple radios to improve throughput

[Aaron Kaplan]

On Jun 5, 2008, at 3:17 PM, Benjamin Henrion wrote:

>
> I have bought 12 foneras to make a test with 6 nodes (2 foneras per
> node), to see if OLSR chooses the fastest route in terms of
> throughput, and it does not. I have to try to play with
> LinkQualityMult on the ethernet link.

Not *yet* :)

With ETT metric this will be the case.

With MIC metrics we will even be able to avoid collisions much more.

Stay tuned or help in making this possible :)

a.


---
there's no place like 127.0.0.1

[B.A.T.M.A.N.] Multiple radios to improve throughput

[Shane Chao]

Hi all,

I ran another iperf test with the following scenario below:

node 1          node 2          node 3
------          ------          ------
  ath0 <--ch 1--> ath0
                  ath1 <--ch 11-> ath0

Both nodes 1 and 3 are still using WRAP board w/ 266MHz cpu with single
radio inside but this time I replaced node 2 with a desktop PC w/ 1GHz cpu
and 2 radios.  The PC is running Ubuntu Linux and Madwifi drivers.  The
result of iperf from node 1 to 3 isn't any better than when node 2 was
using WRAP w/ 2 radios.  I was expecting higher throughput from node 1 to
3 due to a faster cpu on node 2 to forward packets between the 2 radios
inside but I am now confused.

I am currently not running Batman and instead manually creating the route
table in trying to isolate the throughput problem.

Maybe there is some configuration parameters that need to be set on node
2?  Any advice would be appreciated.

Thanks,
Shane

Re: [B.A.T.M.A.N.] Multiple radios to improve throughput

[Benjamin Henrion]

On Thu, Jun 12, 2008 at 5:53 AM, Shane Chao <schao@changind.com> wrote:
> Hi all,
>
> I ran another iperf test with the following scenario below:
>
> node 1          node 2          node 3
> ------          ------          ------
>  ath0 <--ch 1--> ath0
>                  ath1 <--ch 11-> ath0
>
> Both nodes 1 and 3 are still using WRAP board w/ 266MHz cpu with single
> radio inside but this time I replaced node 2 with a desktop PC w/ 1GHz cpu
> and 2 radios.  The PC is running Ubuntu Linux and Madwifi drivers.  The
> result of iperf from node 1 to 3 isn't any better than when node 2 was
> using WRAP w/ 2 radios.  I was expecting higher throughput from node 1 to
> 3 due to a faster cpu on node 2 to forward packets between the 2 radios
> inside but I am now confused.
>
> I am currently not running Batman and instead manually creating the route
> table in trying to isolate the throughput problem.
>
> Maybe there is some configuration parameters that need to be set on node
> 2?  Any advice would be appreciated.

Try to scan if you don't have any ap on the same channel or
overlapping channels in the neighborhood. Then, put the ch1 on 1 and
the ch2 on 14. That would maximise the separation.

And try this outdoor in order to avoid reflexions indoor.

--
Benjamin Henrion <bhenrion at ffii.org>
FFII Brussels - +32-484-566109 - +32-2-4148403