[B.A.T.M.A.N.] video streaming over batman-adv

[B.A.T.M.A.N.] video streaming over batman-adv

[Santiago Álvarez Álvarez]

Hi everybody,
I’m an engineer working in a R&D Project using batman-adv protocol. We’re
trying to develop a mesh network using linux devices with the objective of
transmitting video in streaming. Video is being generated with gstreamer
using UDP.
The application works relatively fine with 2 devices, but efficiency heavily
decreases when introducing a new node in the mesh network (information need
two jumps to arrive to destination). Introducing more jumps in the network
is exponentially worst. We have been expecting a better behavior of this
protocol, since level 2 routing should be transparent  (just increasing
delay when increasing the number of nodes).
We are using batman version 2015.0,  over wifi interfaces, and our streaming
applications is working only point-to-point (not multicast or broadcast).
The nodes in between server and client are working just as wireless
repeaters using batman-adv.
We also discovered that worst case happens when changes in batman routing
tables are happening (sometimes, client is able to reach server in just one
jump, but with low quality, and chooses that option instead of jumping
through the repeater node, with better quality). When that happens, we're
droping lots of packets.
Any of you haver tried batman-adv for video streaming? Which was the maximum
number of jumps between batman nodes that you can manage maintaining a good
video quality? 
Can you recommend any specific configuration for improving batman behavior?

Re: [B.A.T.M.A.N.] video streaming over batman-adv

[Simon Wunderlich]

[-- Attachment #1: Type: text/plain, Size: 3004 bytes --]

Hi Santiago,

On Friday 16 October 2015 12:26:48 Santiago Álvarez Álvarez wrote:
> Hi everybody,
> I’m an engineer working in a R&D Project using batman-adv protocol. We’re
> trying to develop a mesh network using linux devices with the objective of
> transmitting video in streaming. Video is being generated with gstreamer
> using UDP.
> The application works relatively fine with 2 devices, but efficiency heavily
> decreases when introducing a new node in the mesh network (information need
> two jumps to arrive to destination). Introducing more jumps in the network
> is exponentially worst. We have been expecting a better behavior of this
> protocol, since level 2 routing should be transparent  (just increasing
> delay when increasing the number of nodes).

Did you consider the effects of the half-duplex nature? Your intemediate hop 
will have to listen to the sender and and send each packet also to the 
receiver. This means it has to send and receive each packet twice, so needs 
double amount of airtime and therefore the throughput will be halved. Adding 
more repeaters is even worse. This is a well known effect in mesh networks and 
is not limited to Layer2/3. Usually your first hop drops to 50%, second to 33% 
and so on.

The only way to resolve that would be to use multiradio access points on 
different frequencies.

> We are using batman version 2015.0,  over wifi interfaces, and our streaming
> applications is working only point-to-point (not multicast or broadcast).
> The nodes in between server and client are working just as wireless
> repeaters using batman-adv.
> We also discovered that worst case happens when changes in batman routing
> tables are happening (sometimes, client is able to reach server in just one
> jump, but with low quality, and chooses that option instead of jumping
> through the repeater node, with better quality). When that happens, we're
> droping lots of packets.

Another thing you should consider is rates: your wifi card has a rate control 
algorithm implemented which will choose the best rate, which can be anything 
between 1 Mbit/s and 450 MBit/s (depending on  your hardware of course). 
Choosing a low throughput rate also results into losses if your video stream 
needs more throughput than the channel can give you.

> Any of you haver tried batman-adv for video streaming? Which was the maximum
> number of jumps between batman nodes that you can manage maintaining a good
> video quality?

I've seen for some cases. It can work over multiple hops, but it heavily 
depends on your video stream. There are also WiFi parameters you might tune.

> Can you recommend any specific configuration for improving batman behavior?

First, I would suggest to use a higher multicast rate (WiFi configuration). 
Another thing to consider, if this is a setup which should only send video 
streams, is to limit the WiFi rates to some high rates like 18 Mbit/s and up.

Cheers,
     Simon

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [B.A.T.M.A.N.] video streaming over batman-adv

[Linus Lüssing]

Hi Santiago,

Simon already mentioned a few good points, for instance increasing
the multicast rate to improve the route selection and that you'd
need to take the half-duplex nature into account.

Another thing to consider is, that most video codecs are
very susceptible to packet loss. Unfortunately, especially the
2.4GHz "crap band" with various devices with no
CSMA-CA ("listen-before-talk") can have quite a lot of that, depending
on your environment. Even with the retry-mechanism for unicast, I
wouldn't be surprised if you had 1-3% packet loss. That of course
multiplies per hop.

Even if the retry-mechanism could cope with your medium, then
retries will introduce a good deal of jitter. Unfortunately, RTP
doesn't like jitter and your receiver might consider packets too
late and might drop them on the application layer.


You should probably have a look what is creating your "decrease in
efficency". Whether your medium is saturated (I can recommend
H.O.R.S.T. (1) for that) or something else causing packet loss. Or
whether actually all packets arrive and it's just jitter (check
with Wireshark for instance).

If it isn't saturation but just general packet loss or jitter
then you can easily send UDP packets redundantly with gstreamer
(e.g. with the "tee" element). You should be able to tweak the
buffering time and jitter resistance with gstreamer, too.

Also, try simply configuring static routes over your adhoc
interfaces and compare. If you have the same issues there then you
know it isn't batman-adv's fault :).

Regards, Linus

(1): http://br1.einfach.org/tech/horst/


PS: Would you mind sharing your gstreamer pipeline? Just to check
some basic things like having an "gstrtpjitterbuffer" element for
instance. Also, are you using 2.4 or 5GHz wifi? Are these 802.11n
devices?

PPS: "We have been expecting a better behavior of this protocol,
since level 2 routing should be transparent (just increasing delay
when increasing the number of nodes)."
=> I fail to see why for this three node setup it should matter
whether it's a layer 2 or 3 routing protocol - a layer 3 routing
protocol should be transparent for the application layer, too ;).
It's probably actually a layer 1 problem you are observing, which
your application currently can't cope with :).


On Fri, Oct 16, 2015 at 12:26:48PM +0200, Santiago Álvarez Álvarez wrote:
> Hi everybody,
> I’m an engineer working in a R&D Project using batman-adv protocol. We’re
> trying to develop a mesh network using linux devices with the objective of
> transmitting video in streaming. Video is being generated with gstreamer
> using UDP.
> The application works relatively fine with 2 devices, but efficiency heavily
> decreases when introducing a new node in the mesh network (information need
> two jumps to arrive to destination). Introducing more jumps in the network
> is exponentially worst. We have been expecting a better behavior of this
> protocol, since level 2 routing should be transparent  (just increasing
> delay when increasing the number of nodes).
> We are using batman version 2015.0,  over wifi interfaces, and our streaming
> applications is working only point-to-point (not multicast or broadcast).
> The nodes in between server and client are working just as wireless
> repeaters using batman-adv.
> We also discovered that worst case happens when changes in batman routing
> tables are happening (sometimes, client is able to reach server in just one
> jump, but with low quality, and chooses that option instead of jumping
> through the repeater node, with better quality). When that happens, we're
> droping lots of packets.
> Any of you haver tried batman-adv for video streaming? Which was the maximum
> number of jumps between batman nodes that you can manage maintaining a good
> video quality? 
> Can you recommend any specific configuration for improving batman behavior?
> 

Re: [B.A.T.M.A.N.] video streaming over batman-adv

[Santiago Álvarez Álvarez]

Thank you Simon and Linus for your answer,
I'll try right now your suggestions and see if I get an improvement.
In the meantime, for your information, I can tell you that I'm using 2.4GHz wifi Edimax 7811UN and TP-Link TL-WN722N.  They are 802.11n capable, but I'm not using them like that. I'll also share the gstreamer instructions that I'm using for video streaming:

gst-launch-1.0 v4l2src device="/dev/video0" ! 'video/x-raw,width=1280,height=720' !  x264enc bitrate=1024 key-int-max=15 tune=zerolatency speed-preset=superfast ! mpegtsmux ! queue ! udpsink host="192.168.2.99" port=8500

gst-launch-1.0 udpsrc port=8500 ! tsdemux ! decodebin ! queue ! autovideosink



-----Mensaje original-----
De: B.A.T.M.A.N [mailto:b.a.t.m.a.n-bounces@lists.open-mesh.org] En nombre de Linus Lüssing
Enviado el: viernes, 16 de octubre de 2015 15:22
Para: The list for a Better Approach To Mobile Ad-hoc Networking <b.a.t.m.a.n@lists.open-mesh.org>
Asunto: Re: [B.A.T.M.A.N.] video streaming over batman-adv

Hi Santiago,

Simon already mentioned a few good points, for instance increasing the multicast rate to improve the route selection and that you'd need to take the half-duplex nature into account.

Another thing to consider is, that most video codecs are very susceptible to packet loss. Unfortunately, especially the 2.4GHz "crap band" with various devices with no CSMA-CA ("listen-before-talk") can have quite a lot of that, depending on your environment. Even with the retry-mechanism for unicast, I wouldn't be surprised if you had 1-3% packet loss. That of course multiplies per hop.

Even if the retry-mechanism could cope with your medium, then retries will introduce a good deal of jitter. Unfortunately, RTP doesn't like jitter and your receiver might consider packets too late and might drop them on the application layer.


You should probably have a look what is creating your "decrease in efficency". Whether your medium is saturated (I can recommend H.O.R.S.T. (1) for that) or something else causing packet loss. Or whether actually all packets arrive and it's just jitter (check with Wireshark for instance).

If it isn't saturation but just general packet loss or jitter then you can easily send UDP packets redundantly with gstreamer (e.g. with the "tee" element). You should be able to tweak the buffering time and jitter resistance with gstreamer, too.

Also, try simply configuring static routes over your adhoc interfaces and compare. If you have the same issues there then you know it isn't batman-adv's fault :).

Regards, Linus

(1): http://br1.einfach.org/tech/horst/


PS: Would you mind sharing your gstreamer pipeline? Just to check some basic things like having an "gstrtpjitterbuffer" element for instance. Also, are you using 2.4 or 5GHz wifi? Are these 802.11n devices?

PPS: "We have been expecting a better behavior of this protocol, since level 2 routing should be transparent (just increasing delay when increasing the number of nodes)."
=> I fail to see why for this three node setup it should matter whether it's a layer 2 or 3 routing protocol - a layer 3 routing protocol should be transparent for the application layer, too ;).
It's probably actually a layer 1 problem you are observing, which your application currently can't cope with :).


On Fri, Oct 16, 2015 at 12:26:48PM +0200, Santiago Álvarez Álvarez wrote:
> Hi everybody,
> I’m an engineer working in a R&D Project using batman-adv protocol. 
> We’re trying to develop a mesh network using linux devices with the 
> objective of transmitting video in streaming. Video is being generated 
> with gstreamer using UDP.
> The application works relatively fine with 2 devices, but efficiency 
> heavily decreases when introducing a new node in the mesh network 
> (information need two jumps to arrive to destination). Introducing 
> more jumps in the network is exponentially worst. We have been 
> expecting a better behavior of this protocol, since level 2 routing 
> should be transparent  (just increasing delay when increasing the number of nodes).
> We are using batman version 2015.0,  over wifi interfaces, and our 
> streaming applications is working only point-to-point (not multicast or broadcast).
> The nodes in between server and client are working just as wireless 
> repeaters using batman-adv.
> We also discovered that worst case happens when changes in batman 
> routing tables are happening (sometimes, client is able to reach 
> server in just one jump, but with low quality, and chooses that option 
> instead of jumping through the repeater node, with better quality). 
> When that happens, we're droping lots of packets.
> Any of you haver tried batman-adv for video streaming? Which was the 
> maximum number of jumps between batman nodes that you can manage 
> maintaining a good video quality?
> Can you recommend any specific configuration for improving batman behavior?
>