Re: [Discuss] IPv4 packets lost with macvlan over bond alb

[Jay Vosburgh <jay.vosburgh@canonical.com>]

Hangbin Liu <liuhangbin@gmail.com> wrote:

>Hi Jay, any thoughts?

	Just an update that I've done some poking with your reproducer;
as described, the ARP reply for the IP address associated with the
macvlan interface is coming back with the bond's MAC, not the MAC of the
macvlan.  If I manually create a neighbour entry on the client with the
corrent IP and MAC for the macvlan, then connectivity works as expected.

	I'll have to look a bit further into the ARP MAC selection logic
here (i.e., where does that MAC come from when the ARP reply is
generated).  It also makes me wonder what's special about macvlan, e.g.,
why doesn't regular VLAN (or other stacked devices) fail in the same way
(or maybe it does and nobody has noticed).

	-J

>Thanks
>Hangbin
>On Fri, Jun 02, 2023 at 04:09:00PM +0800, Hangbin Liu wrote:
>> Hi Jay,
>> 
>> It looks there is an regression for commit 14af9963ba1e ("bonding: Support
>> macvlans on top of tlb/rlb mode bonds"). The author export modified ARP to
>> remote when there is macvlan over bond, which make remote add neighbor
>> with macvlan's IP and bond's mac. The author expect RLB will replace all
>> inner packets to correct mac address if target is macvlan, but RLB only
>> handle ARP packets. This make all none arp packets macvlan received have
>> incorrect mac address, and dropped directly.
>> 
>> In short, remote client learned macvlan's ip with bond's mac. So the macvlan
>> will receive packets with incorrect macs and dropped.
>> 
>> To fix this, one way is to revert the patch and only send learning packets for
>> both tlb and alb mode for macvlan. This would make all macvlan rx packets go
>> through bond's active slave.
>> 
>> Another way is to replace the bond's mac address to correct macvlan's address
>> based on the rx_hashtbl . But this may has impact to the receive performance
>> since we need to check all the upper devices and deal the mac address for
>> each packets in bond_handle_frame().
>> 
>> So which way do you prefer?
>> 
>> Reproducer:
>> ```
>> #!/bin/bash
>> 
>> # Source the topo in bond selftest
>> source bond_topo_3d1c.sh
>> 
>> trap cleanup EXIT
>> 
>> setup_prepare
>> bond_reset "mode balance-alb"
>> ip -n ${s_ns} addr flush dev bond0
>> 
>> ip -n ${s_ns} link add link bond0 name macv0 type macvlan mode bridge
>> ip -n ${s_ns} link set macv0 up
>> 
>> # I just add macvlan on the server netns, you can also move it to another netns for testing
>> ip -n ${s_ns} addr add ${s_ip4}/24 dev macv0
>> ip -n ${s_ns} addr add ${s_ip6}/24 dev macv0
>> ip netns exec ${c_ns} ping ${s_ip4} -c 4
>> sleep 5
>> ip netns exec ${c_ns} ping ${s_ip4} -c 4
>> ```
>> 
>> Thanks
>> Hangbin

---
	-Jay Vosburgh, jay.vosburgh@canonical.com