* Bridge connectivity interruptions while devices join or leave the bridge
@ 2018-09-19 9:10 Johannes Wienke
2018-09-19 10:07 ` Ido Schimmel
0 siblings, 1 reply; 6+ messages in thread
From: Johannes Wienke @ 2018-09-19 9:10 UTC (permalink / raw)
To: netdev
I am sorry for probably misusing this list, but I couldn't find any
other mailing list suitable for asking in-detail Linux networking
questions. As I am not subscribed, please CC me in a potential reply.
I am currently tracking down a connectivity issues of docker containers
on a custom bridge network, which I could reduce to the Linux network
stack without docker being involved.
The situation that I am observing is the following: I have a bridge
device, which is connected to the outer world using forwarding and
masquerading (so the bridge does not contain the outgoing network
interface of the host). This bridge is used to perform network
operations by a long-running process, which is restricted to this bridge
using network namespaces and veth devices (exactly what docker does
internally). What I see is that every time a (virtual) network device is
added to or removed from the bridge, the communication of the
long-running process is interrupted.
I have created two scripts that can be used to replicate the situation.
They are available at:
https://gist.github.com/languitar/9ac8dc5c8db7cf4a89e1546f6e32ca7b
setup.bash sets up the bridge, veth devices, network namespace and the
iptables rules to replicate the network setup and simulates the
long-running process by periodically performing (volatile) UDP DNS
requests in a while loop.
When launching this script, all DNS requests should succeed and you
should see success messages at a regular pace.
To simulate devices joining and leaving the bridge, you can start
interruptor.bash.
As soon as this script is running, you can observe that DNS requests
will be delayed frequently and some of them even fail. In a parallel
pcap you would see that sometimes the UDP packages from the DNS lookup
are not routed to the outside world, but instead end up at the bridge
device without ever leaving the host system.
Can someone explain what is happening here and why adding and removing
devices to a bridge results in the connectivity issues? How to avoid
this behavior? I'd be glad for any hint on that.
Kind regards
Johannes
--
Johannes Wienke, Researcher at CoR-Lab / CITEC, Bielefeld University
Address: Inspiration 1, D-33619 Bielefeld, Germany (Room 1.307)
Phone: +49 521 106-67277
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Bridge connectivity interruptions while devices join or leave the bridge
2018-09-19 9:10 Bridge connectivity interruptions while devices join or leave the bridge Johannes Wienke
@ 2018-09-19 10:07 ` Ido Schimmel
2018-09-19 11:00 ` Johannes Wienke
0 siblings, 1 reply; 6+ messages in thread
From: Ido Schimmel @ 2018-09-19 10:07 UTC (permalink / raw)
To: Johannes Wienke; +Cc: netdev
On Wed, Sep 19, 2018 at 11:10:22AM +0200, Johannes Wienke wrote:
> Can someone explain what is happening here and why adding and removing
> devices to a bridge results in the connectivity issues? How to avoid
> this behavior? I'd be glad for any hint on that.
The MAC address of the bridge ('brtest' in your example) is inherited
from the bridge port with the "smallest" MAC address. Thus, when you
generate veth devices with random MACs and enslave them to the bridge,
you sometimes change the bridge's MAC address as well. And since the
bridge is the default gateway sometimes packets are sent to the wrong
MAC address.
You can avoid randomly changing the bridge's MAC by setting it
explicitly:
# ip link set dev brtest address <some_mac>
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Bridge connectivity interruptions while devices join or leave the bridge
2018-09-19 10:07 ` Ido Schimmel
@ 2018-09-19 11:00 ` Johannes Wienke
2018-09-19 16:45 ` Ido Schimmel
0 siblings, 1 reply; 6+ messages in thread
From: Johannes Wienke @ 2018-09-19 11:00 UTC (permalink / raw)
To: Ido Schimmel; +Cc: netdev
[-- Attachment #1.1: Type: text/plain, Size: 1081 bytes --]
Dear Ido,
On 9/19/18 12:07 PM, Ido Schimmel wrote:
> On Wed, Sep 19, 2018 at 11:10:22AM +0200, Johannes Wienke wrote:
>> Can someone explain what is happening here and why adding and removing
>> devices to a bridge results in the connectivity issues? How to avoid
>> this behavior? I'd be glad for any hint on that.
>
> The MAC address of the bridge ('brtest' in your example) is inherited
> from the bridge port with the "smallest" MAC address. Thus, when you
> generate veth devices with random MACs and enslave them to the bridge,
> you sometimes change the bridge's MAC address as well. And since the
> bridge is the default gateway sometimes packets are sent to the wrong
> MAC address.
Thank you very much! This was the important hint and solves the issues.
This behavior of inheriting the mac address is really unexpected to us.
Is it documented somewhere?
Kind regards
Johannes
--
Johannes Wienke, Researcher at CoR-Lab / CITEC, Bielefeld University
Address: Inspiration 1, D-33619 Bielefeld, Germany (Room 1.307)
Phone: +49 521 106-67277
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Bridge connectivity interruptions while devices join or leave the bridge
2018-09-19 11:00 ` Johannes Wienke
@ 2018-09-19 16:45 ` Ido Schimmel
2018-09-19 17:03 ` Stephen Hemminger
0 siblings, 1 reply; 6+ messages in thread
From: Ido Schimmel @ 2018-09-19 16:45 UTC (permalink / raw)
To: Johannes Wienke; +Cc: netdev
On Wed, Sep 19, 2018 at 01:00:15PM +0200, Johannes Wienke wrote:
> This behavior of inheriting the mac address is really unexpected to us.
> Is it documented somewhere?
Not that I'm aware, but it's a well established behavior.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Bridge connectivity interruptions while devices join or leave the bridge
2018-09-19 16:45 ` Ido Schimmel
@ 2018-09-19 17:03 ` Stephen Hemminger
2018-09-19 20:57 ` Johannes Wienke
0 siblings, 1 reply; 6+ messages in thread
From: Stephen Hemminger @ 2018-09-19 17:03 UTC (permalink / raw)
To: Ido Schimmel; +Cc: Johannes Wienke, netdev
On Wed, 19 Sep 2018 19:45:08 +0300
Ido Schimmel <idosch@idosch.org> wrote:
> On Wed, Sep 19, 2018 at 01:00:15PM +0200, Johannes Wienke wrote:
> > This behavior of inheriting the mac address is really unexpected to us.
> > Is it documented somewhere?
>
> Not that I'm aware, but it's a well established behavior.
Not documented, has always been that way. It seems to be part of 802 standard maybe?
Anyway, if you set a MAC address of the bridge device it makes it sticky;
i.e it won't change if ports of bridge change.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Bridge connectivity interruptions while devices join or leave the bridge
2018-09-19 17:03 ` Stephen Hemminger
@ 2018-09-19 20:57 ` Johannes Wienke
0 siblings, 0 replies; 6+ messages in thread
From: Johannes Wienke @ 2018-09-19 20:57 UTC (permalink / raw)
To: Stephen Hemminger, Ido Schimmel; +Cc: netdev
On 19.09.18 19:03, Stephen Hemminger wrote:
> On Wed, 19 Sep 2018 19:45:08 +0300
> Ido Schimmel <idosch@idosch.org> wrote:
>
>> On Wed, Sep 19, 2018 at 01:00:15PM +0200, Johannes Wienke wrote:
>>> This behavior of inheriting the mac address is really unexpected to us.
>>> Is it documented somewhere?
>>
>> Not that I'm aware, but it's a well established behavior.
>
> Not documented, has always been that way. It seems to be part of 802 standard maybe?
> Anyway, if you set a MAC address of the bridge device it makes it sticky;
> i.e it won't change if ports of bridge change.
Yes sure. It was just unexpected to meet this and we ha no clue that
this could be a reasonable default. I just wonder what the motivation
for hat is and of course some more obvious documentation would have
helped a lot in debugging/avoid this.
Cheers,
Johannes
--
Johannes Wienke, Researcher at CoR-Lab / CITEC, Bielefeld University
Address: Inspiration 1, D-33619 Bielefeld, Germany (Room 1.307)
Phone: +49 521 106-67277
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-09-20 2:38 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-09-19 9:10 Bridge connectivity interruptions while devices join or leave the bridge Johannes Wienke
2018-09-19 10:07 ` Ido Schimmel
2018-09-19 11:00 ` Johannes Wienke
2018-09-19 16:45 ` Ido Schimmel
2018-09-19 17:03 ` Stephen Hemminger
2018-09-19 20:57 ` Johannes Wienke
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).