Problems with Multiple Interfaces

Problems with Multiple Interfaces

[joe]

Hi Everybody,

I'm running BATMAN v 2019.2 with ath10k on OpenWRT.  The particular router I'm using has two interfaces, 2.4 GHz and 5GHz.  With each router using single interface (e.g. 2.4 GHz), it works fine.  When I add a second interface (e.g. 5GHz), it sometimes hangs.  

I know that BATMAN is expected to alternate between the two interfaces, but that doesn't seem to be working.  As you probably know, using two interfaces should dramatically improve throughput if you need to make a hop through another mesh node.

To verify this is the problem, I did a test where I ran three routers,  2.4G <> Dual (2.4G & 5G) <> 5G.  I would try to ping between the two routers that were using just a single interface.  Obviously, that must hop through the dual radio router.  The dual radio router has both interfaces on the bat0 master, and batctl reports they're both active.  

The system would sometimes lose link, although batctl would report getting neighbor messages.  Even using the batctl ping function doesn't get through.  I can see traffic flowing on both the 2.4 GHz and 5GHz interfaces. 

I'm happy to provide lots of configuration detail, but I thought I'd start with a high level description in case this is a known problem.

Any assistance is appreciated.  Thank you,

--Joe

Re: Problems with Multiple Interfaces

[Simon Wunderlich]

[-- Attachment #1: Type: text/plain, Size: 1859 bytes --]

On Thursday, April 8, 2021 7:56:29 AM CEST joe@careyhome.org wrote:
> Hi Everybody,
> 
> I'm running BATMAN v 2019.2 with ath10k on OpenWRT.  The particular router
> I'm using has two interfaces, 2.4 GHz and 5GHz.  With each router using
> single interface (e.g. 2.4 GHz), it works fine.  When I add a second
> interface (e.g. 5GHz), it sometimes hangs.
> 
> I know that BATMAN is expected to alternate between the two interfaces, but
> that doesn't seem to be working.  As you probably know, using two
> interfaces should dramatically improve throughput if you need to make a hop
> through another mesh node.
> 
> To verify this is the problem, I did a test where I ran three routers,  2.4G
> <> Dual (2.4G & 5G) <> 5G.  I would try to ping between the two routers
> that were using just a single interface.  Obviously, that must hop through
> the dual radio router.  The dual radio router has both interfaces on the
> bat0 master, and batctl reports they're both active.
> 
> The system would sometimes lose link, although batctl would report getting
> neighbor messages.  Even using the batctl ping function doesn't get
> through.  I can see traffic flowing on both the 2.4 GHz and 5GHz
> interfaces.
> 
> I'm happy to provide lots of configuration detail, but I thought I'd start
> with a high level description in case this is a known problem.
> 
> Any assistance is appreciated.  Thank you,
> 
> --Joe

Hi Joe,

I'd expect your setup to work. Could you try to run "batctl ping" between the 
nodes, maybe also on the intermediate links? If you see that it's failing too, 
could you set up some IPv4 addresses (or use IPv6 link local addresses) on 
your ad-hoc/mesh interfaces, and try to ping to the next hop? I suspect that 
something is off with the wifi driver, i.e. broadcasts are still working but 
unicast pings are dropped. 

Cheers,
        Simon

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Problems with Multiple Interfaces

[Linus Lüssing]

On Fri, Apr 09, 2021 at 12:17:36PM +0200, Simon Wunderlich wrote:
> On Thursday, April 8, 2021 7:56:29 AM CEST joe@careyhome.org wrote:
> > Hi Everybody,
> > 
> > I'm running BATMAN v 2019.2 with ath10k on OpenWRT.  The particular router
> > I'm using has two interfaces, 2.4 GHz and 5GHz.  With each router using
> > single interface (e.g. 2.4 GHz), it works fine.  When I add a second
> > interface (e.g. 5GHz), it sometimes hangs.

One more note: ath10k firmware refuses to provide an expected
throughput for the wifi link.

The fallback to an estimate which uses the raw tx bitrate was
added not that long ago in batman-adv v2020.0:

https://www.open-mesh.org/news/95

I'm wondering if you might have very low throughput values in
batman-adv and if that might lead to issues with interface
alternating.

Also, would be interesting to know if you have the same issues
when using BATMAN IV, which mainly uses a packetloss based metric.

Regards, Linus

Re: Problems with Multiple Interfaces

[joe]

Linus and Simon,

Thank you for your responses.

I did try "batctl ping" and the adjacent device was unreachable.  I could see it with "batctl n". I did not try a simple IPv6 link local ping.  However, I could see it using "iw dev mesh_5g station dump"

I agree that it feels like a problem with ath10k reporting the data rate incorrectly (or not at all).

I am going to try upgrading to the latest release, but it will be a few days.  I'll let you know how it goes.

Thanks again,

--Joe

Re: Problems with Multiple Interfaces

[Simon Wunderlich]

[-- Attachment #1: Type: text/plain, Size: 1065 bytes --]

On Saturday, April 10, 2021 2:39:53 AM CEST joe@careyhome.org wrote:
> Linus and Simon,
> 
> Thank you for your responses.
> 
> I did try "batctl ping" and the adjacent device was unreachable.  I could
> see it with "batctl n". I did not try a simple IPv6 link local ping. 
> However, I could see it using "iw dev mesh_5g station dump"
> 
> I agree that it feels like a problem with ath10k reporting the data rate
> incorrectly (or not at all).
> 
> I am going to try upgrading to the latest release, but it will be a few
> days.  I'll let you know how it goes.
> 
> Thanks again,
> 
> --Joe

Sounds great! You may want to take some more measures to find out if the 
packets are actually getting transferred, e.g. doing tcpdump on both the 
sending and receiving side. If you see packets sent but not received on the 
other side, you know that it's a driver issue. Also "tcpdump" helps. For 
ath10k, you may want to try 11s/mesh mode with TTL=1, while with ath10k-ct you 
may prefer IBSS mode (I think that wasn't mentioned in your first mail).

Cheers,
      Simon

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Problems with Multiple Interfaces

[Sven Eckelmann]

[-- Attachment #1: Type: text/plain, Size: 400 bytes --]

On Tuesday, 13 April 2021 13:53:16 CEST Simon Wunderlich wrote:
[...]
> Also "tcpdump" helps. For 
> ath10k, you may want to try 11s/mesh mode with TTL=1, while with ath10k-ct you 
> may prefer IBSS mode (I think that wasn't mentioned in your first mail).

Please avoid the ath10k-ct firmware. And for ath10k, please make sure to set 
mesh_fwding=0 on the mesh(point) interface.

Kind regards,
	Sven

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Problems with Multiple Interfaces

[Andi Depressivum]

Am Di., 13. Apr. 2021 um 14:03 Uhr schrieb Sven Eckelmann <sven@narfation.org>:

> Please avoid the ath10k-ct firmware. And for ath10k, please make sure to set
> mesh_fwding=0 on the mesh(point) interface.

This is true for Wave-1 chipsets (QCA98xx) but Wave-2 Chipsets
(QCA99xx) needs the ath10-ct firmware to get 802.11s running.

Haven't got stable results with BATMAN V in a home mesh environment.
Nodes could ping each other via batctl ping but IP ping often failed.
Maybe some issues of mesh routing when many routes to the same target
had nearly the same link speeds? I'm using a dense setup of many
routers, so every router is seeing at least 2-3 neighbours. The
preferred routes (marked with a [*] in batctl o ?) switched quite
often. How is the BATMAN V metric affected by link quality & hop
count?

Regards,
Andi