* Re: ARP flux vs. weak/strong ES model
2019-02-15 21:23 ARP flux vs. weak/strong ES model Lewis G Rosenthal
@ 2019-02-17 5:10 ` Grant Taylor
2019-02-18 0:37 ` Erik Auerswald
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Grant Taylor @ 2019-02-17 5:10 UTC (permalink / raw)
To: lartc
[-- Attachment #1: Type: text/plain, Size: 5972 bytes --]
On 2/15/19 1:11 PM, Erik Auerswald wrote:
> The first time I read about this, the issue was framed in the weak
> end-system model vs. strong end-system model terminology. Thus I have
> used that terminology in the past whenever a related problem occurred.
The earliest reference I could find to "ARP flux" was in the Guide to IP
Layer Network Administration document, with references going back to 2003.
I get the impression that the "ARP flux" nomenclature may have
originated from the Linux community and grown out from there.
Many, but not all, of the references to "weak vs strong end system" are
older than that, or by people that learned about things (likely) pre-2000.
I think that "weak vs strong end system" and "ARP flux" are functionally
the same thing. Or at least the same concept told from opposing ends.
It seems to me that Linux's (default) "weak end system" model is the
cause of "ARP flux" (in Linux).
> I have seen it just recently, and do not remember having read it before.
> It turned up while searching the web for ARP related Linux sysctl settings.
Seeing as how the "ARP flux" phrase seems (to me) to be Linux centric.
It makes sense that it would turn up when searching for articles about
ARP issues on Linux.
> To turn this around, searching the web for "ARP flux" seems to turn up
> better results regarding the original poster's issue than searching for
> "weak end-system model". YMMV.
I've frequently noticed that older concepts are frequently harder to
find than newer concepts. I think this is likely somewhat natural based
on how recent a concept is.
There is probably a health portion of things being better documented and
/ or communicated post-2000, particularly with availability of web /
Internet things.
> TL;DR I prefer precision when talking about technical problems. :-)
I tend to agree.
But I don't think "ARP flux" is a precise reason. I think the "weak vs
strong end system" is the reason behind the "ARP flux" symptom.
> I'd like to take a closer look at the problem at hand to better
> understand it and the terminology used to describe it.
>
> I'd say the distinction between weak and strong end-system (ES) model is
> related to the problem at hand, but "ARP flux" is a distinct issue not
> necessarily observed with a host implementing the weak (ES) model.
Do you know of any examples where the "ARP flux" /symptom/ was observed
on a "strong end system"?
> RFC 1122 continues to state that "[The weak ES] model [...] is necessary
> for hosts that have embedded gateway functionality."
I disagree with that assertion. (Not your recounting of it.)
> A gateway (or router) needs to accept IP packets not addressed to the
> receiving interface in order to forward them. But most commercial
> routers will not answer an ARP request ingressing on the wrong
> interface, unless Proxy-ARP is activated. By default, Linux does answer
> such ARP requests.
I believe the MUST vs MUST NOT text is primarily about traffic to the
end system, not traffic passing through the system. Particularly how
the system's IP stack responds in relation to IPs bound locally.
I think that routing / forwarding is decidedly different. Yes, a router
must allow traffic in that is not destined to the router. (Assuming
that we're talking about a forwarding interface and not a management
interface.) That's the router's job.
But the router does not need to respond to ARP requests from one
interface for IPs on a different interface.
> This answer to ARP requests arriving at the wrong interface is the root
> cause for most problems commonly subsumed under "Linux implements the
> weak ES model". But those problems do not occur with other weak ES model
> implementing systems, e.g. Extreme Networks EXOS switches.
Each IP stack is different. I wouldn't take the difference in behavior
of the Linux IP stack and EXOS's IP stack as definitive behavior.
> Additionally, changing Linux behaviour (via sysctl) to not answer those
> ARP requests does not change the weak ES model applicability to Linux
> (i.e. IP packets delivered inside an Ethernet frame addressed to an
> interface MAC of a Linux system are accepted even if they are addressed
> to another IP address of the Linux host; packet forwarding is not
> affected either).
>
> Linux's cavalier default behaviour in answering ARP requests might be
> motivated by the weak ES model in helping other systems on the LAN reach
> the Linux server, even if they use an IP address assigned to another LAN
> the Linux server is connected to. Thus the problem of "ARP flux" is
> probably closely related to how Linux implements the weak ES model, but
> not necessarily to the weak ES model itself as described in RFC 1122.
I feel like "ARP flux" is a /symptom/ of "weak ES model".
Does a fever cause a cold? Or can a cold cause a fever? You can buy
both cold medicine and fever reducers. ;-)
(I know it's not a perfect analogy. But I suspect you get my point.)
> Thus I argue that using "ARP flux" to describe the ARP problem observed
> with Linux is preferable to attributing the problems to Linux's
> implementation of the weak ES model.
You logic makes sense. I choose to think something different.
> Please note that I have not searched for the origin or an authoritative
> source on "ARP flux", and cannot guarantee that it is indeed
> consistently used to describe the aforementioned problem. But it did
> turn up on related web searches, and seems to directly refer to the ARP
> problem at hand.
See my comments above.
> Sorry for being pedantic.
Apology returned as unnecessary.
I often find that I need to be pedantic when making sure I understand
something, and / or communicating minutia details about it with someone.
--
Grant. . . .
unix || die
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4008 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: ARP flux vs. weak/strong ES model
2019-02-15 21:23 ARP flux vs. weak/strong ES model Lewis G Rosenthal
2019-02-17 5:10 ` Grant Taylor
@ 2019-02-18 0:37 ` Erik Auerswald
2019-02-18 2:18 ` Grant Taylor
2019-02-18 12:10 ` Erik Auerswald
3 siblings, 0 replies; 5+ messages in thread
From: Erik Auerswald @ 2019-02-18 0:37 UTC (permalink / raw)
To: lartc
Ji,
On 2/17/19 06:10, Grant Taylor wrote:
> On 2/15/19 1:11 PM, Erik Auerswald wrote:
> [...]
> The earliest reference I could find to "ARP flux" was in the Guide to IP
> Layer Network Administration document, with references going back to 2003.
>
> I get the impression that the "ARP flux" nomenclature may have
> originated from the Linux community and grown out from there.
Perhaps because it is a Linux only thing? That is just a guess.
>> [...]
>> I'd say the distinction between weak and strong end-system (ES) model
>> is related to the problem at hand, but "ARP flux" is a distinct issue
>> not necessarily observed with a host implementing the weak (ES) model.
>
> Do you know of any examples where the "ARP flux" /symptom/ was observed
> on a "strong end system"?
I have only ever seen it on Linux.
>> RFC 1122 continues to state that "[The weak ES] model [...] is
>> necessary for hosts that have embedded gateway functionality."
>
> I disagree with that assertion. (Not your recounting of it.)
I'd say that assertion is often correct, but need not be.
I think that this assertion motivates looking at non-Linux systems,
especially traditional routers, and if they act as weak or strong
end-systems. And then look at their ARP handling, excluding Proxy ARP.
>> A gateway (or router) needs to accept IP packets not addressed to the
>> receiving interface in order to forward them. But most commercial
>> routers will not answer an ARP request ingressing on the wrong
>> interface, unless Proxy-ARP is activated. By default, Linux does
>> answer such ARP requests.
>
> I believe the MUST vs MUST NOT text is primarily about traffic to the
> end system, not traffic passing through the system. Particularly how
> the system's IP stack responds in relation to IPs bound locally.
>
> I think that routing / forwarding is decidedly different. Yes, a router
> must allow traffic in that is not destined to the router. (Assuming
> that we're talking about a forwarding interface and not a management
> interface.) That's the router's job.
Yes, a system dedicated to packet forwarding is different. Often it is
an end-system as well for management purposes, sometimes for additional
services (inlcuding BGP). RFC 1122 looks at IP hosts, not necessarily
dedicated gateways, but includes hosts that act as gateways, too. Thus I
do not say that RFC 1122 is necessarily pertinent to dedicated routers,
but often even dedicated routers act like end-systems with embedded
gateway functionality.
> But the router does not need to respond to ARP requests from one
> interface for IPs on a different interface.
Exactly. Unless Proxy ARP is active, any "traditional" router I know
answers ARP requests only if they arrive at the interface configured
with the IP address in the ARP request.
Again, any "traditional" router accepts IP packets directed to any of
its interface IPs irrespective of the ingress interface. That is the
basis for using a loopback address for router management or BGP
sessions. In that case a router acts as an end-system as well.
The above can often be changed via configuration, to separate management
interfaces from forwarding interfaces.
>> This answer to ARP requests arriving at the wrong interface is the
>> root cause for most problems commonly subsumed under "Linux implements
>> the weak ES model". But those problems do not occur with other weak ES
>> model implementing systems, e.g. Extreme Networks EXOS switches.
>
> Each IP stack is different. I wouldn't take the difference in behavior
> of the Linux IP stack and EXOS's IP stack as definitive behavior.
But the weak vs. strong ES model is independent of IP stacks. It is a
description of behaviour, and can be used as guidance when implementing
an IP stack.
The two examples are just that, examples, nothing more. Cisco IOS (with
deactivated Proxy ARP) is another example for weak ES model without ARP
flux. But just one example (e.g. of Extreme or Cisco) suffices to show
that weak ES model does not imply ARP flux. The Linux example shows that
weak ES model and ARP flux can occur together.
>> Additionally, changing Linux behaviour (via sysctl) to not answer
>> those ARP requests does not change the weak ES model applicability to
>> Linux (i.e. IP packets delivered inside an Ethernet frame addressed to
>> an interface MAC of a Linux system are accepted even if they are
>> addressed to another IP address of the Linux host; packet forwarding
>> is not affected either).
>>
>> Linux's cavalier default behaviour in answering ARP requests might be
>> motivated by the weak ES model in helping other systems on the LAN
>> reach the Linux server, even if they use an IP address assigned to
>> another LAN the Linux server is connected to. Thus the problem of "ARP
>> flux" is probably closely related to how Linux implements the weak ES
>> model, but not necessarily to the weak ES model itself as described in
>> RFC 1122.
>
> I feel like "ARP flux" is a /symptom/ of "weak ES model".
I'd say it is somewhat independent of the weak ES model. It is a symptom
of the Linux IP stack. That IP stack may be built around weak ES model
ideas. Other IP stacks adhere to the weak ES model as well without
exhibiting ARP flux. Thus I do not accept that "ARP flux" is a symptom
(or necessary result) of "weak ES model".
But it seems to me that combining strong ES model with ARP flux does not
make sense. As such I do see some relation between ARP flux and weak ES
model.
> [...]
>> Thus I argue that using "ARP flux" to describe the ARP problem
>> observed with Linux is preferable to attributing the problems to
>> Linux's implementation of the weak ES model.
>
> You logic makes sense. I choose to think something different.
Then we have to agree to disagree.
Thanks,
Erik
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: ARP flux vs. weak/strong ES model
2019-02-15 21:23 ARP flux vs. weak/strong ES model Lewis G Rosenthal
2019-02-17 5:10 ` Grant Taylor
2019-02-18 0:37 ` Erik Auerswald
@ 2019-02-18 2:18 ` Grant Taylor
2019-02-18 12:10 ` Erik Auerswald
3 siblings, 0 replies; 5+ messages in thread
From: Grant Taylor @ 2019-02-18 2:18 UTC (permalink / raw)
To: lartc
[-- Attachment #1: Type: text/plain, Size: 6046 bytes --]
On 2/17/19 5:37 PM, Erik Auerswald wrote:
> I have only ever seen it on Linux.
Likewise.
> I think that this assertion motivates looking at non-Linux systems,
> especially traditional routers, and if they act as weak or strong
> end-systems. And then look at their ARP handling, excluding Proxy ARP.
Before looking at other systems, I'd want to step back and think how
weak vs strong end-systems /should/ behave regarding ARP flux.
Aside: I think of Proxy ARP as a form of routing. After all, the Proxy
ARP router, is doing exactly what it would do to route a packet if it
had naturally come to it / it's MAC address. All Proxy ARP is doing is
responding to ARP requests for things on the other side of the router
such that the packet does come to the router.
> Yes, a system dedicated to packet forwarding is different. Often it is
> an end-system as well for management purposes, sometimes for additional
> services (inlcuding BGP). RFC 1122 looks at IP hosts, not necessarily
> dedicated gateways, but includes hosts that act as gateways, too. Thus I
> do not say that RFC 1122 is necessarily pertinent to dedicated routers,
> but often even dedicated routers act like end-systems with embedded
> gateway functionality.
Fair.
> Exactly. Unless Proxy ARP is active, any "traditional" router I know
> answers ARP requests only if they arrive at the interface configured
> with the IP address in the ARP request.
ACK
> Again, any "traditional" router accepts IP packets directed to any of
> its interface IPs irrespective of the ingress interface. That is the
> basis for using a loopback address for router management or BGP
> sessions. In that case a router acts as an end-system as well.
That's not entirely true. Especially when filtering / firewalling is in
place to only allow traffic from specific interfaces.
I've also viewed that as the traffic would be routed through the device
to the proper interface which would then process it accordingly. In,
over / through and then up the IP stack instead of in and directly up
the IP stack.
> The above can often be changed via configuration, to separate management
> interfaces from forwarding interfaces.
I view the fact that the router is also an end system for management
purposes as independent of it's routing function.
> But the weak vs. strong ES model is independent of IP stacks. It is a
> description of behaviour, and can be used as guidance when implementing
> an IP stack.
Okay.
> The two examples are just that, examples, nothing more. Cisco IOS (with
> deactivated Proxy ARP) is another example for weak ES model without ARP
> flux. But just one example (e.g. of Extreme or Cisco) suffices to show
> that weak ES model does not imply ARP flux. The Linux example shows that
> weak ES model and ARP flux can occur together.
Let's back up and discuss what is actually allowing ARP flux to happen.
As I understand it, the /flux/ comes from the fact that the MAC address
that ARPing hosts get replies from changes and fluctuates. Hence the name.
It's my understanding that this happens because Linux does not filter
(in any meaning of the word) incoming ARP requests (or outgoing replies)
based on the physical interface. This is especially true when you have
multiple interfaces in the same broadcast domain.
Aside: The last time I tried to put two interfaces in the same subnet
and connect them to the same broadcast domain on a Cisco, it would not
allow me to do so.
> I'd say it is somewhat independent of the weak ES model. It is a symptom
> of the Linux IP stack. That IP stack may be built around weak ES model
> ideas. Other IP stacks adhere to the weak ES model as well without
> exhibiting ARP flux.
Sorry for being pedantic, but I think we need to clearly define the
configuration and behavior that we're discussing.
I say this because I think that "ARP flux" is a symptom of having a
Linux box with two interfaces in the same broadcast domain, thus able to
hear the same ARP request and that the flux comes from the ensuing race
conditions as to which interface will be processed -and- reply first.
I feel like this same scenario is seldom played out in traditional
network gear. And if we want to have the discussion about this, we
should configure said gear comparably and test how it behaves.
I will also state that Linux may likely respond to ARP requests from an
inside interface for IPs on the outside interface. But in such a
scenario, there is only one interface connected to the broadcast domain,
thus there is nothing to flux over as it will always be the single
possible interface.
So, let's define what the connections are, and how things are configured.
I'm stating two interfaces connected to the same broadcast domain, each
with IPs in different subnets. (Thus the broadcast domain is overloaded
and has multiple subnets on it.) I think there is a reasonable chance
that the ARP flux symptoms can occur in this configuration.
I'm thinking Linux /kernel/ default (no distro sysctl modifications or
kernel compilation tweaks). I'm also thinking Proxy ARP is disabled.
Do you agree? Or do you want to alter the configuration?
> Thus I do not accept that "ARP flux" is a symptom (or necessary result)
> of "weak ES model".
Okay.
> But it seems to me that combining strong ES model with ARP flux does not
> make sense. As such I do see some relation between ARP flux and weak ES
> model.
Okay.
> Then we have to agree to disagree.
I don't see a problem with that. It's less than ideal, but it is a
perfectly valid outcome.
That being said, you do have me questioning things. At the moment, I'm
sticking with what I've thought for years. But I am interested in
continuing the conversation and learning, what ever the lesson may be.
> Thanks,
You're welcome.
Thank you.
--
Grant. . . .
unix || die
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4008 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: ARP flux vs. weak/strong ES model
2019-02-15 21:23 ARP flux vs. weak/strong ES model Lewis G Rosenthal
` (2 preceding siblings ...)
2019-02-18 2:18 ` Grant Taylor
@ 2019-02-18 12:10 ` Erik Auerswald
3 siblings, 0 replies; 5+ messages in thread
From: Erik Auerswald @ 2019-02-18 12:10 UTC (permalink / raw)
To: lartc
Hi,
On 2/18/19 03:18, Grant Taylor wrote:
> On 2/17/19 5:37 PM, Erik Auerswald wrote:
>> I have only ever seen it on Linux.
>
> Likewise.
>
>> I think that this assertion motivates looking at non-Linux systems,
>> especially traditional routers, and if they act as weak or strong
>> end-systems. And then look at their ARP handling, excluding Proxy ARP.
>
> Before looking at other systems, I'd want to step back and think how
> weak vs strong end-systems /should/ behave regarding ARP flux.
As I see it, strong ES model should preclude ARP flux, because an ES
first telling "use this MAC to send to my IP X", but then discarding the
packet sent based on that information, seems nonsensical.
In the weak ES model, telling another ES to send to MAC Y to deliver to
IP X, although the interface with MAC Y uses IP Z≠X seems OK, but not
mandatory. To me this feels much like the use of Proxy ARP to work
around misconfigured end-systems. Ignoring ARP requests on all
interfaces but the one with the asked for IP seems OK, too.
> Aside: I think of Proxy ARP as a form of routing.
Freely combining MAC and IP addresses in ARP replies looks quite similar
to Proxy ARP to me.
> After all, the Proxy
> ARP router, is doing exactly what it would do to route a packet if it
> had naturally come to it / it's MAC address. All Proxy ARP is doing is
> responding to ARP requests for things on the other side of the router
> such that the packet does come to the router.
That is correct, but Proxy ARP will result in a router answering an ARP
request that was not for the receiving interface, thus it could be
confused with ARP flux in a test.
>> [...]
>> Again, any "traditional" router accepts IP packets directed to any of
>> its interface IPs irrespective of the ingress interface. That is the
>> basis for using a loopback address for router management or BGP
>> sessions. In that case a router acts as an end-system as well.
>
> That's not entirely true. Especially when filtering / firewalling is in
> place to only allow traffic from specific interfaces.
Filtering / firewalling can be in effect in Linux as well, including for
Ethernet (ebtables). That would most likely affect ARP flux and weak ES
model behaviour as well, depending on rule set.
> I've also viewed that as the traffic would be routed through the device
> to the proper interface which would then process it accordingly. In,
> over / through and then up the IP stack instead of in and directly up
> the IP stack.
It is not observably "routed" in that all routing actions (L2 rewrite,
TTL decrement) usually happen when sending the packet _out_ the egress
interface.
>> [...]
> Let's back up and discuss what is actually allowing ARP flux to happen.
>
> As I understand it, the /flux/ comes from the fact that the MAC address
> that ARPing hosts get replies from changes and fluctuates. Hence the name.
>
> It's my understanding that this happens because Linux does not filter
> (in any meaning of the word) incoming ARP requests (or outgoing replies)
> based on the physical interface. This is especially true when you have
> multiple interfaces in the same broadcast domain.
That is a logical explanation of the name "ARP flux".
> Aside: The last time I tried to put two interfaces in the same subnet
> and connect them to the same broadcast domain on a Cisco, it would not
> allow me to do so.
Correct. Huawei VRP allows a special case (a loopback interface with a
/32 IP inside an IP subnet with shorter prefix active on another
interface, including Ethernet interfaces), but I have not yet found time
to thoroughly test the behaviour of that. Other networking equipment I
used did not allow two interfaces in the same (or overlapping) subnet(s).
>> I'd say it is somewhat independent of the weak ES model. It is a
>> symptom of the Linux IP stack. That IP stack may be built around weak
>> ES model ideas. Other IP stacks adhere to the weak ES model as well
>> without exhibiting ARP flux.
>
> Sorry for being pedantic, but I think we need to clearly define the
> configuration and behavior that we're discussing.
>
> I say this because I think that "ARP flux" is a symptom of having a
> Linux box with two interfaces in the same broadcast domain, thus able to
> hear the same ARP request and that the flux comes from the ensuing race
> conditions as to which interface will be processed -and- reply first.
>
> I feel like this same scenario is seldom played out in traditional
> network gear. And if we want to have the discussion about this, we
> should configure said gear comparably and test how it behaves.
>
> I will also state that Linux may likely respond to ARP requests from an
> inside interface for IPs on the outside interface. But in such a
> scenario, there is only one interface connected to the broadcast domain,
> thus there is nothing to flux over as it will always be the single
> possible interface.
>
> So, let's define what the connections are, and how things are configured.
>
> I'm stating two interfaces connected to the same broadcast domain, each
> with IPs in different subnets. (Thus the broadcast domain is overloaded
> and has multiple subnets on it.) I think there is a reasonable chance
> that the ARP flux symptoms can occur in this configuration.
>
> I'm thinking Linux /kernel/ default (no distro sysctl modifications or
> kernel compilation tweaks). I'm also thinking Proxy ARP is disabled.
>
> Do you agree? Or do you want to alter the configuration?
I want to extend that scenario to include two interfaces A and B with
different IP addresses connected to two separate broadcast domains.
While that does not result in fluctuating ARP replies (ARP flux), it
does result in ARP replies combining MAC of interface A with IP of
interface B.
Both scenarios (two interfaces connected to one broadcast domain, two
interfaces connected to separate broadcast domains) show symptoms of the
same underlying cause. The name ARP _flux_ is more fitting to the first
scenario, the second could be better described as "ARP confusion" (I
made that name up just now).
> [...]
> That being said, you do have me questioning things. At the moment, I'm
> sticking with what I've thought for years. But I am interested in
> continuing the conversation and learning, what ever the lesson may be.
Likewise.
Best regards,
Erik
^ permalink raw reply [flat|nested] 5+ messages in thread