Re: PROBLEM: Issue with setting veth MAC address being unreliable.

[Ido Schimmel <idosch@idosch.org>]

On Fri, Jul 12, 2024 at 05:02:13PM -0400, Jason Zhou wrote:
> Hi all,
> 
> We have performed our own testing with the MacAddressPolicy set to
> none rather than persistent on CentOS 9, and it fixed the problem we
> were seeing with the MAC address mismatches before and after us trying
> to manually set it.
> So we're pretty confident that the cause is what Ido stated, and that
> we were racing against udev as we did not set a MAC address when
> creating our veth device pair, making udev think it should give out a
> new MAC address.
> We will release a patch on Apache Mesos to mitigate this potential
> race condition on systems with systemd version > 242.
> Thank you so much for the help!
> 
> For documenting this issue, I believe that this race condition would
> also be present for the peer veth interface?

Yes, but when creating the veth pair you can set the addresses of both
devices:

# ip link add name bla1 address 00:11:22:33:44:55 type veth peer name bla2 address 00:aa:bb:cc:dd:ee
# ip link show dev bla1
11: bla1@bla2: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:11:22:33:44:55 brd ff:ff:ff:ff:ff:ff
# ip link show dev bla2
10: bla2@bla1: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:aa:bb:cc:dd:ee brd ff:ff:ff:ff:ff:ff

> We create the peer along with veth and move the peer to another
> namespace, but udev would be notified of its creation, so it will try
> to also overwrite the peer's MAC address.

The peer can be created in the desired namespace with the desired
address:

# ip netns add ns1
# ip link add name bla1 address 00:11:22:33:44:55 type veth peer name bla2 address 00:aa:bb:cc:dd:ee netns ns1
# ip link show dev bla1
10: bla1@if8: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:11:22:33:44:55 brd ff:ff:ff:ff:ff:ff link-netns ns1
bash-5.2# ip -n ns1 link show dev bla2
8: bla2@if10: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:aa:bb:cc:dd:ee brd ff:ff:ff:ff:ff:ff link-netnsid 0

> However this is not an issue for the loopback interface that comes
> with every namespace creation, as they will not be affected by
> NetworkManager and hence udev will not try to modify them.
> Please correct me if I'm wrong!

AFAICT udev ignores the loopback devices because they have a type of
ARPHRD_LOOPBACK, unlike the veth devices that have a type of
ARPHRD_ETHER.