* VXLAN multicast receive not working
@ 2012-11-21 23:27 Bernhard Schmidt
0 siblings, 0 replies; 5+ messages in thread
From: Bernhard Schmidt @ 2012-11-21 23:27 UTC (permalink / raw)
To: netdev
Hello,
I'm just trying to play with VXLAN a bit and wanted to build a Linux
gateway routing into seperate VXLAN segments.
Debian Wheezy, running 3.7-rc6, with current git HEAD of iproute2.
It's a VMware VM but that should not matter much.
Two vmxnet3 NICs, one with management and one with my VXLAN transport
network.
4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
state UNKNOWN qlen 1000
link/ether 00:50:56:8e:0d:c8 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.250/24 scope global eth1
inet6 fe80::250:56ff:fe8e:dc8/64 scope link
valid_lft forever preferred_lft forever
In the same network segment are two VMware ESXi 5.0 hosts with Nexus
1000V for VLAN termination (10.0.0.1 and 10.0.0.2)
On top of that there is a VXLAN interface defined, with ID 12340 and
group 239.0.0.42.
6: vxlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue
state UNKNOWN mode DEFAULT
link/ether f6:59:e7:db:82:92 brd ff:ff:ff:ff:ff:ff
vxlan id 12340 group 239.0.0.42 dev eth1 port 32768 61000 ageing 300
That interface has an address as well
6: vxlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue
state UNKNOWN
link/ether f6:59:e7:db:82:92 brd ff:ff:ff:ff:ff:ff
inet 10.1.1.1/24 scope global vxlan0
inet6 fe80::f459:e7ff:fedb:8292/64 scope link
valid_lft forever preferred_lft forever
The same VXLAN domain is defined on the Nexus 1000V and a VM is attached
to it. When I send some broadcast traffic down vxlan0 (i.e. ping
10.1.1.2 which generates an ARP request) the VM sees the packet just
fine.
When I do it the other way around (the VM sends a broadcast ARP for
10.1.1.3) I see a packet coming into eth1 on the multicast group, but
vxlan0 stays silent.
I have captured one of those packets, wireshark does not support
disecting it yet but in my eyes the packet is correct. I've put it
online at http://users.birkenwald.de/~berni/temp/vxlan.pcap
Weirdly enough, as soon as I populate the ARP and VXLAN forwarding table
by pinging back from the destination to the source (so the source can
learn both MAC->Nexthop for VXLAN and IP->MAC from the ARP request) it
starts working.
To summarize, Multicast/Broadcast from N1k to Linux seems to be broken,
the encapsulated packet is seen on the Ethernet but the decapsulated
packet is not seen on vxlan0. Broadcast/Multicast in the other direction
works just fine as well as Unicast in both directions.
Thanks for any pointers,
Bernhard
^ permalink raw reply [flat|nested] 5+ messages in thread
* VXLAN multicast receive not working
@ 2012-11-22 0:05 Bernhard Schmidt
2012-11-22 0:09 ` Bernhard Schmidt
0 siblings, 1 reply; 5+ messages in thread
From: Bernhard Schmidt @ 2012-11-22 0:05 UTC (permalink / raw)
To: netdev
[Apologies if you receive this twice, my original mail seems to be lost]
Hello,
I'm just trying to play with VXLAN a bit and wanted to build a Linux
gateway routing into seperate VXLAN segments.
Debian Wheezy, running 3.7-rc6, with current git HEAD of iproute2.
It's a VMware VM but that should not matter much.
Two vmxnet3 NICs, one with management and one with my VXLAN transport
network.
4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
state UNKNOWN qlen 1000
link/ether 00:50:56:8e:0d:c8 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.250/24 scope global eth1
inet6 fe80::250:56ff:fe8e:dc8/64 scope link
valid_lft forever preferred_lft forever
In the same network segment are two VMware ESXi 5.0 hosts with Nexus
1000V for VLAN termination (10.0.0.1 and 10.0.0.2)
On top of that there is a VXLAN interface defined, with ID 12340 and
group 239.0.0.42.
6: vxlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue
state UNKNOWN mode DEFAULT
link/ether f6:59:e7:db:82:92 brd ff:ff:ff:ff:ff:ff
vxlan id 12340 group 239.0.0.42 dev eth1 port 32768 61000 ageing 300
That interface has an address as well
6: vxlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue
state UNKNOWN
link/ether f6:59:e7:db:82:92 brd ff:ff:ff:ff:ff:ff
inet 10.1.1.1/24 scope global vxlan0
inet6 fe80::f459:e7ff:fedb:8292/64 scope link
valid_lft forever preferred_lft forever
The same VXLAN domain is defined on the Nexus 1000V and a VM is attached
to it. When I send some broadcast traffic down vxlan0 (i.e. ping
10.1.1.2 which generates an ARP request) the VM sees the packet just
fine.
When I do it the other way around (the VM sends a broadcast ARP for
10.1.1.3) I see a packet coming into eth1 on the multicast group, but
vxlan0 stays silent.
I have captured one of those packets, wireshark does not support
disecting it yet but in my eyes the packet is correct. I've put it
online at http://users.birkenwald.de/~berni/temp/vxlan.pcap
Weirdly enough, as soon as I populate the ARP and VXLAN forwarding table
by pinging back from the destination to the source (so the source can
learn both MAC->Nexthop for VXLAN and IP->MAC from the ARP request) it
starts working.
To summarize, Multicast/Broadcast from N1k to Linux seems to be broken,
the encapsulated packet is seen on the Ethernet but the decapsulated
packet is not seen on vxlan0. Broadcast/Multicast in the other direction
works just fine as well as Unicast in both directions.
Thanks for any pointers,
Bernhard
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: VXLAN multicast receive not working
2012-11-22 0:05 Bernhard Schmidt
@ 2012-11-22 0:09 ` Bernhard Schmidt
2012-11-24 0:08 ` Bernhard Schmidt
0 siblings, 1 reply; 5+ messages in thread
From: Bernhard Schmidt @ 2012-11-22 0:09 UTC (permalink / raw)
To: netdev
On Thu, Nov 22, 2012 at 01:05:25AM +0100, Bernhard Schmidt wrote:
> The same VXLAN domain is defined on the Nexus 1000V and a VM is attached
> to it. When I send some broadcast traffic down vxlan0 (i.e. ping
> 10.1.1.2 which generates an ARP request) the VM sees the packet just
> fine.
>
> When I do it the other way around (the VM sends a broadcast ARP for
> 10.1.1.3) I see a packet coming into eth1 on the multicast group, but
> vxlan0 stays silent.
I think I found a possible reason, my vxlan interface is on top of eth1
7: vxlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue
state UNKNOWN mode DEFAULT
link/ether 96:06:c6:cf:a0:2e brd ff:ff:ff:ff:ff:ff
vxlan id 12340 group 239.0.0.42 dev eth1 port 32768 61000 ageing 300
but the multicast group is joined only on eth0
root@lxbscDA-VXLAN:~/iproute2# ip maddr
1: lo
inet 224.0.0.1
inet6 ff02::1
2: eth0
link 33:33:00:00:00:01
link 01:00:5e:00:00:01
link 33:33:ff:8e:0d:c7
link 01:00:5e:00:00:2a
inet 239.0.0.42
inet 224.0.0.1
inet6 ff02::1:ff8e:dc7 users 2
inet6 ff02::1
4: eth1
link 33:33:00:00:00:01
link 01:00:5e:00:00:01
link 33:33:ff:8e:0d:c8
inet 224.0.0.1
inet6 ff02::1:ff8e:dc8
inet6 ff02::1
7: vxlan0
link 33:33:00:00:00:01
link 01:00:5e:00:00:01
link 33:33:ff:cf:a0:2e
inet 224.0.0.1
inet6 ff02::1:ffcf:a02e
inet6 ff02::1
At first glance I did not spot the issue with my two weeks of C, so
someone else has to hunt this down.
Regards,
Bernhard
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: VXLAN multicast receive not working
2012-11-22 0:09 ` Bernhard Schmidt
@ 2012-11-24 0:08 ` Bernhard Schmidt
2012-11-24 1:24 ` Bernhard Schmidt
0 siblings, 1 reply; 5+ messages in thread
From: Bernhard Schmidt @ 2012-11-24 0:08 UTC (permalink / raw)
To: netdev
Bernhard Schmidt <berni@birkenwald.de> wrote:
> On Thu, Nov 22, 2012 at 01:05:25AM +0100, Bernhard Schmidt wrote:
>
>> The same VXLAN domain is defined on the Nexus 1000V and a VM is attached
>> to it. When I send some broadcast traffic down vxlan0 (i.e. ping
>> 10.1.1.2 which generates an ARP request) the VM sees the packet just
>> fine.
>>
>> When I do it the other way around (the VM sends a broadcast ARP for
>> 10.1.1.3) I see a packet coming into eth1 on the multicast group, but
>> vxlan0 stays silent.
>
> I think I found a possible reason, my vxlan interface is on top of eth1
>
> 7: vxlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue
> state UNKNOWN mode DEFAULT
> link/ether 96:06:c6:cf:a0:2e brd ff:ff:ff:ff:ff:ff
> vxlan id 12340 group 239.0.0.42 dev eth1 port 32768 61000 ageing 300
>
> but the multicast group is joined only on eth0
Confirmed working as soon as eth0 can receive the group multicast
address (connected to the same VLAN), even if the vxlan0 interface is
still configured to eth1.
Bernhard
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: VXLAN multicast receive not working
2012-11-24 0:08 ` Bernhard Schmidt
@ 2012-11-24 1:24 ` Bernhard Schmidt
0 siblings, 0 replies; 5+ messages in thread
From: Bernhard Schmidt @ 2012-11-24 1:24 UTC (permalink / raw)
To: netdev
Bernhard Schmidt <berni@birkenwald.de> wrote:
>>> The same VXLAN domain is defined on the Nexus 1000V and a VM is attached
>>> to it. When I send some broadcast traffic down vxlan0 (i.e. ping
>>> 10.1.1.2 which generates an ARP request) the VM sees the packet just
>>> fine.
>>>
>>> When I do it the other way around (the VM sends a broadcast ARP for
>>> 10.1.1.3) I see a packet coming into eth1 on the multicast group, but
>>> vxlan0 stays silent.
>>
>> I think I found a possible reason, my vxlan interface is on top of eth1
>>
>> 7: vxlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue
>> state UNKNOWN mode DEFAULT
>> link/ether 96:06:c6:cf:a0:2e brd ff:ff:ff:ff:ff:ff
>> vxlan id 12340 group 239.0.0.42 dev eth1 port 32768 61000 ageing 300
>>
>> but the multicast group is joined only on eth0
>
> Confirmed working as soon as eth0 can receive the group multicast
> address (connected to the same VLAN), even if the vxlan0 interface is
> still configured to eth1.
When swapping the configuration around using eth0 for the VXLAN
transport and eth1 for Management, the group is joined on eth1. It is
always joined where the default route points, so it looks like the
interface is not set at all.
# ip route add <group>/32 dev <vxlanintf>
works around the problem.
Bernhard
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2012-11-24 1:24 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-11-21 23:27 VXLAN multicast receive not working Bernhard Schmidt
-- strict thread matches above, loose matches on Subject: below --
2012-11-22 0:05 Bernhard Schmidt
2012-11-22 0:09 ` Bernhard Schmidt
2012-11-24 0:08 ` Bernhard Schmidt
2012-11-24 1:24 ` Bernhard Schmidt
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).