Using NFT for dynamic per-packet traffic load balancing

Using NFT for dynamic per-packet traffic load balancing

[Tomas Mudrunka]

Hello
I was experimenting with using NFT to balance traffic between several 
links, this is what i came up with:

table bridge balancer {
     chain egress_forward {
         type filter hook forward priority filter; policy accept;
         meta ibrname br0 drop;
     }
}

table netdev balancer {
     map dispatch_map {
         typeof numgen random mod 100 : oif
         flags interval
         elements = {
             0-9   : "eth0",
             10-29 : "eth1",
             30-99 : "eth2",
         }
     }

     chain br_egress {
         type filter hook egress device br0 priority filter; policy drop;
         fwd to numgen random mod 100 map @dispatch_map
     }
}


You create bridge br0 with eth0-eth2 interfaces in it.
There is a map that sets weights of how much traffic should be directed 
to individual links, so you can fully use multiple links even when the 
bandwidth is not equal.
For wireless links you can even get signal quality feedback from radio 
hardware and live tune the map based on it. Tried up to 10 times a 
second without any issues using following command:

echo 'flush map netdev balancer dispatch_map; add element netdev 
balancer dispatch_map { 0-32 : "eth0", 33-66 : "eth1", 67-99 : "eth2" }' 
| sudo nft -f -

Being able to dynamicaly fine tune this map from userspace in realtime 
is killer-feature of this approach and cannot be achieved by regular 
linux interface bonding (link aggregation). While it is very important 
for me, i agree this is quite niche requirement.

The nft ruleset prevents bridge from actualy forwarding between ports, 
so each frame is sent only via single randomly selected interface. 
Received packets are not forwarded anywhere except for local ip stack.

I know this setup is still quite rough, but i wonder if there is 
something that can be improved. It seems to be really interresting 
topic.

Maybe i don't like using "fwd to" and should just drop frames on all 
interfaces except for one. because right now i cannot see egress frames 
on br0 using wireshark, only ingress.

Maybe i can ditch bridge completely and use dummy interface instead? I 
am not sure what is cleanest way. i want to be able to use wireshark to 
see all frames going through the load balancing on single interface, 
which is not really happening right now. but i can observe individual 
interfaces, which is not as useful, but allowed me to verify the 
ballancing works properly and follows weight ratio defined in the map.

Do you have any experience with such approach? or suggestions?

-- 
Best regards
      Tomáš Mudruňka