How to mesh over ethernet VLAN?

How to mesh over ethernet VLAN?

[Andi Depressivum]

Hello everybody,

What's the correct setup to mesh also over an existing ethernet VLAN link?

My core network with all routers & servers is running on VLAN 3
(eth0.3 in the routers). How do I have to setup batman-adv to a) mesh
also over this existing link and b) propagate this VLAN also over
wireless links. Bridging eth0.3 with bat0.3 and adding eth0.3 also as
a hardif doesn't work, as it will remove eth0.3 from the bridge.

If I add eth0 instead as hardif the routers are only seeing each other
over wireless link. Or do I have to use a separate VLAN for ethernet
meshing which isn't bridged to a bat0.X interface?

Thanks a lot!

Re: How to mesh over ethernet VLAN?

[Linus Lüssing]

On Sun, Apr 04, 2021 at 07:52:35PM +0200, Andi Depressivum wrote:
> Hello everybody,
> 
> What's the correct setup to mesh also over an existing ethernet VLAN link?
> 
> My core network with all routers & servers is running on VLAN 3
> (eth0.3 in the routers). How do I have to setup batman-adv to a) mesh
> also over this existing link and b) propagate this VLAN also over
> wireless links. Bridging eth0.3 with bat0.3 and adding eth0.3 also as
> a hardif doesn't work, as it will remove eth0.3 from the bridge.

Right, that is not supported. That goes in the direction of
creating an encapsulating loop. It's best to avoid sharing a link for both
the payload frames and the mesh layer.

One option could be to add an extra VLAN, like VLAN 4, dedicated
for the mesh layer with the batman-adv frames, parallel to the
VLAN 3.

Or if most/all hosts are running batman-adv, so if architecturally
batman-adv is forming the backbone of your network, you could
run batman-adv over this VLAN 3 only. And then either run
batman-adv on the servers themselves or bridge the servers into the
batman-adv mesh on their adjacent mesh router(s).

Regards, Linus

Re: How to mesh over ethernet VLAN?

[Andi Depressivum]

> Right, that is not supported. That goes in the direction of
> creating an encapsulating loop. It's best to avoid sharing a link for both
> the payload frames and the mesh layer.

You're right, I've missed the encapsulation loop when thinking about it

> One option could be to add an extra VLAN, like VLAN 4, dedicated
> for the mesh layer with the batman-adv frames, parallel to the
> VLAN 3.

That's the way I've configured it before thinking about VLAN-sharing.
Works as expected except the virtual interface for the mesh link needs
a different MAC address to the payload interfaces.

> Or if most/all hosts are running batman-adv, so if architecturally
> batman-adv is forming the backbone of your network, you could
> run batman-adv over this VLAN 3 only. And then either run
> batman-adv on the servers themselves or bridge the servers into the
> batman-adv mesh on their adjacent mesh router(s).

That was my very first configuration approach but it's rather slow
(about 200mbit/s over a gigabit link) compared to native VLANs. I've
tried to set the MTU size to 1536 for the mesh interface but for some
reason the MTU of the interface stays at 1500?!

config interface 'mesh_lan'
        option mtu '1536'
        option proto 'batadv_hardif'
        option master 'bat0'
        option ifname 'eth0.2'
        option macaddr 'f4:f1:6d:37:b6:85'
        option hop_penalty '5'

Kind regards, Andi

Re: How to mesh over ethernet VLAN?

[Linus Lüssing]

On Tue, Apr 06, 2021 at 11:33:26PM +0200, Andi Depressivum wrote:
> That was my very first configuration approach but it's rather slow
> (about 200mbit/s over a gigabit link) compared to native VLANs. I've
> tried to set the MTU size to 1536 for the mesh interface but for some
> reason the MTU of the interface stays at 1500?!

The MTU of eth0.2 or bat0?

One of the performance penalties might occur when batman-adv needs
to use fragmentation. batman-adv is going to add its own header
on top of the 1500 byte frames received on bat0.

1536 on eth0.2 looks good though. Then 1500 bytes should fit
without fragmentation on bat0. You can check with "batctl
td" or Wireshark if you see batman-adv fragments on eth0.2.

If 1536 is somehow not applied to eth0.2, might be a driver issue
then. You can check wether you can manually alter an interface MTU
with "ip link dev eth0.2 set mtu 1536", for instance.

Regards, Linus

Re: How to mesh over ethernet VLAN?

[Andi Depressivum]

> The MTU of eth0.2 or bat0?

It doesn't matter, I cannot set the MTU of eth0.2, eth0 or eth1 to
something higher than 1500

> One of the performance penalties might occur when batman-adv needs
> to use fragmentation. batman-adv is going to add its own header
> on top of the 1500 byte frames received on bat0.

That's why I'm trying to increase the MTU

> 1536 on eth0.2 looks good though. Then 1500 bytes should fit
> without fragmentation on bat0. You can check with "batctl
> td" or Wireshark if you see batman-adv fragments on eth0.2.

That's only the config. IP LINK still showing 1500.

> If 1536 is somehow not applied to eth0.2, might be a driver issue
> then. You can check wether you can manually alter an interface MTU
> with "ip link dev eth0.2 set mtu 1536", for instance.

Actual OpenWRT trunk on TP-Link C7 / C2600 devices. Can decrease to
anything lower than 1500 but cannot raise above 1500.

Thanks,
Andi

Re: How to mesh over ethernet VLAN?

[Linus Lüssing]

On Fri, Apr 09, 2021 at 03:38:15PM +0200, Andi Depressivum wrote:
> Actual OpenWRT trunk on TP-Link C7 / C2600 devices. Can decrease to
> anything lower than 1500 but cannot raise above 1500.

Hm, yes, many/most ethernet devices don't support jumbo frames.

You could test with iperf3 and its --set-mss option if smaller,
unfragmented frames have a better performance.

A few more options to solve this are then:

* using the DHCP MTU option on the DHCP server (though many
  DHCP clients do not properly support this)
* use iptables MSS clamping (only works for TCP and routed
  traffic)
* on a gateway router, set a route to the internet with a lower
  MTU, which then results in "ICMP packet too big" messages back
  to the client with containing the desired MTU and the initial
  host should retry with a smaller packet size
  (only works for routed traffic)

Or if you have full control of all hosts in the network you can
also reduce the MTU on the hosts.

Also note batman-adv v2021.0 had some performance
improvements for frames fragmented by batman-adv:

https://www.open-mesh.org/news/100 ->
* https://git.open-mesh.org/batman-adv.git/commit/92064deda9b063ca2d5a53b307c6127a9453357c
* https://git.open-mesh.org/batman-adv.git/commit/0966d5424bb87e863037301488519ccdd69e4d26
* https://git.open-mesh.org/batman-adv.git/commit/3e3ff987876d3be70d928561acbefe5a48ab1654

Regards, Linus

Re: How to mesh over ethernet VLAN?

[Andi Depressivum]

What's the advantage in running BATMAN also on a separate VLAN when
the ethernet nodes are already connected to all switched VLANs? In
case of using BATMAN V it seems to brake rerouting when the cable is
removed as the link is already shown with 1000MBit/s.

Regards,
Andi