Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[Andi Kleen]

dean gaudet <dean-list-linux-kernel@arctic.org> writes:

> On Sat, 13 Jul 2002, David S. Miller wrote:
> 
> >
> > You have to use specific source-routing settings in conjuntion with
> > enabling arp_filter in order for arp_filter to have any effect.
> >
> > This is a FAQ.
> 
> a couple google queries yielded no answer to this faq... is there a posted
> example somewhere?

arpfilter normally needs no special routing entries, unless you want
to do weird things (like filtering ARP based on source). The main use
of arp filter is to prevent multiple arp answers on multiple devices
when the host has more than one interface to the same network. The other
use is to allow load balancing for incoming connections together 
with multi path routing.

It can be abused for more complex filtering scenaries:

The arpfilter routing decision takes the reversed address tuple in account.
When the routing decision yields the device that the ARP arrived on
then the ARP is answered otherwise not.
You can construct policy routing rules that match the ARP requests you
want to prevent with some tricks, but do not match outgoing packets.
Easy? It's not easy, but nobody said it was.

The main use of this seem to be certain HA failover setups.
Some people use a patch that allows to disable ARP per interface for it
("hidden") but for some reasons it was not integrated. 

> 
> is the default behaviour of use to anyone?  this question comes up like
> every other month.

It would be likely easier/more straightforward if there was a special
ARP routing table that is only consulted by ARP filter (as an extension 
to the current multi table routing). Then you could just put reject routes
there to filter ARP Unfortunately nobody has stepped forward to implement it
yet, so it remained a dream so far.

Another thing that was implemented is a netfilter chain for ARP, but
afaik there are no filtering modules for it yet, so Joe User cannot
use it. It likely just exists somewhere as a proprietary module in
someone's firewall appliance and all they did was to contribute the
hook. It probably would not be hard to rewrite a filter module for it,
but again nobody did it yet.

Hope this helps,

-Andi

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[David S. Miller]

   From: Andi Kleen <ak@suse.de>
   Date: 14 Jul 2002 20:40:23 +0200
   
   Another thing that was implemented is a netfilter chain for ARP, but
   afaik there are no filtering modules for it yet, so Joe User cannot
   use it. It likely just exists somewhere as a proprietary module in
   someone's firewall appliance and all they did was to contribute the
   hook. It probably would not be hard to rewrite a filter module for it,
   but again nobody did it yet.

I wrote the kernel bits but never got around to coding up the
netfilter user tools.  Assuming, just because of lacking userland
tools, it was for some "proprietary firewall appliance" is pretty
damn rude.  You could have at least looked at who the author was
and asked him (ie. me). :(

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[Julian Anastasov]

	Hello,

Bill Davidsen wrote:

> Um, many people who are being probed from the Internet would like very
> much to NOT have it there, certainly by default. And would like to send an
> arp request and in return get a valid mac address.

	It seems you don't like the way ARP is replied in Linux. But
note that ARP follows IP, i.e. Linux replies on each device where it
is willing to accept IP for the same path. Such behaviour is caused by
the way IP is set by default in RFC1812: Source Address Validation
is disabled (rp_filter=0).

> I totally miss why anyone would consider this behaviouracceptable, much
> less desirable. Perhaps you or someone could explain why either accepting
> source routing or sending out invalid arp responses are desirable at all,
> much less as default behaviour. One is a security hole, the other brings
> the network group to the door with arpwatch output.

	What security, you are running without rp_filter set. Set it
and your Linux box will reply only through one interface without
using arp_filter.

> After some hours of reading the google results I see lots of questions, a
> few dubious workarounds, and zero people claiming that it's a good thing
> to work that way.

	Why? Do += 1 for me. What is wrong? In the URL I posted
you can find many solutions that additionally allow tuning of the
ARP behaviour because such fine grained control is used for
different setups. What was last discussed about the ARP and
rp_filter can stick in the following list of recommendations:

- use arp_filter=0 and rp_filter=0 where you don't care what
devices are used for the ARP/IP traffic. The DEFAULT behaviour!

- use arp_filter=1 and rp_filter=0 where you prefer each subnet
to use one interface but the IP traffic still can use many
devices (very useful for cross-subnet talks)

- use arp_filter=1 and rp_filter=1 where you want to restrict
both ARP and IP to same interface. This can break cross-subnet
talks if used. Here rp_filter=1 overrides the arp_filter
value.

- use arp_filter=1, rp_filter=1 and tune the rp_filter to
know about where each device is attached (you can see my
unofficial patch named "rp_filter_mask"). Such setting gives
you the ability to attach multiple interfaces to one hub and
to treat these interfaces as attached to "same security zone".
It is extension to rp_filter, we are not restricted to allow
traffic from one device. By this way you can use safely
rp_filter=1 for all devices attached to this hub and still to
allow cross-subnet talks when using these devices. This is the
difference from the previous example, i.e. arp_filter RECOMMENDS
each subnet to use one device but still to allow secure talks
for IP between hosts from different subnets.

Hope that answers much of your questions regarding Linux ARP.
If you still want to learn more about Linux ARP please switch
to linux-net or linux-netdev mlist.

Regards

--
Julian Anastasov <ja@ssi.bg>

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[Daniel Gryniewicz]

On Tue, 2002-07-16 at 18:32, Julian Anastasov wrote:
> 
> 	Hello,
> 
> 	It seems you don't like the way ARP is replied in Linux. But
> note that ARP follows IP, i.e. Linux replies on each device where it
> is willing to accept IP for the same path. Such behaviour is caused by
> the way IP is set by default in RFC1812: Source Address Validation
> is disabled (rp_filter=0).
> 

<heavily snipped>

Okay, I have no problem with this. What I have a problem with is Linux
sending an ARP request out an interface and using the address of
*another interface* as the tell address.  This is just broken.

-- 
Recursion n.:
        See Recursion.
                        -- Random Shack Data Processing Dictionary

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[Julian Anastasov]

	Hello,

On 16 Jul 2002, Daniel Gryniewicz wrote:

> Okay, I have no problem with this. What I have a problem with is Linux
> sending an ARP request out an interface and using the address of

	Please apply rp_filter_mask and be happy :) By this way
you can safely use rp_filter with hubs if that is your problem.

> *another interface* as the tell address.  This is just broken.

	From routing perspective it is not broken: ARP request is
built from data approved from routing and extracted from the
IP packet. If you provide different sender IP in the ARP request
you risk this probe to be answered on another target host's interface,
i.e. the remote host can answer it on eth0 and then our IP packet (with
different src IP) will be dropped on this device if it is allowed
on eth1 and the rp_filter_mask I mentioned in previous mail is not
supported/set (and it is usually not). This happens if the remote host
has 2 devices attached to the same hub we are connected and in such
case it usually accept one ARP probe on one interface (eth*/rp_filter=1).
This is the reason Linux ARP to use addresses from the IP packets.
You must not break this rule or you risk problems. Do you see the 
symmetry? If you lie in your ARP probe then you can receive wrong MAC for 
that target. Note that we resolve the path (SRCIP->DSTIP), not only DSTIP.
We ask "where should I send traffic from SRCIP to DSTIP?". The
question "where is DSTIP" is too ambiguous, you risk to receive
wrong answer (as usually happens). This is not mentioned in RFC826. 
Nor the word "hub" :)

Again, you are welcome on linux-net where we can find solution for
every setup which involves Linux ARP :)

Regards

--
Julian Anastasov <ja@ssi.bg>

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[Julian Anastasov]

	Hello All, David,

Andi Kleen wrote:

> The main use of this seem to be certain HA failover setups.
> Some people use a patch that allows to disable ARP per interface for it
> ("hidden") but for some reasons it was not integrated.

	Yes, "hidden" has its own purpose which is limitted
usually to HA setups. Then Alexey proposed (from long time)
such ARP tricks to be controlled with hook(s)/rules. My ideas in
such direction resulted in "iparp" which is completed:

http://www.linuxvirtualserver.org/~julian/#iparp

	Its purpose is "tuning" of ARP (the kernel's use
only), not for security. For example, it can't deal with remote
replies.

	But at the same time I see David implements arptables.
May be now it is good time to ask:

	David,

	can you explain what kind of control will be
possible with arptables. For now, I see only filtering of
remote probes. As we know the clusters have different needs.
Can you draw a list of goals that will be achieved with
arptables and whether the 2 ARP requirements for clusters ("drop
probes to VIP" and the most needed "don't announce VIP in our
probes") will exist in arptables? The 3th requirement (used
from "hidden" to avoid problems with autoselecting VIP as src
IP can be easily solved in routing).

> It would be likely easier/more straightforward if there was a special
> ARP routing table that is only consulted by ARP filter (as an extension
> to the current multi table routing). Then you could just put reject routes
> there to filter ARP Unfortunately nobody has stepped forward to implement it
> yet, so it remained a dream so far.
>
> Another thing that was implemented is a netfilter chain for ARP, but
> afaik there are no filtering modules for it yet, so Joe User cannot
> use it. It likely just exists somewhere as a proprietary module in
> someone's firewall appliance and all they did was to contribute the
> hook. It probably would not be hard to rewrite a filter module for it,
> but again nobody did it yet.

Regards

--
Julian Anastasov <ja@ssi.bg>

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[David S. Miller]

   From: Julian Anastasov <ja@ssi.bg>
   Date: Sun, 14 Jul 2002 22:35:04 +0000 (GMT)
   
   	can you explain what kind of control will be
   possible with arptables.

arptables can filter both incoming and outgoing ARP packets.
There are no limitations, it is perfectly generic.

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[Julian Anastasov]

	Hello,

Bill Davidsen wrote:

> 1, setting default, and setting 'all." The current settings, just the NICs
> in question, are producing two arp-replies with settings:

	As Alan already said, you need only arp_filter and I see
that you have the right settings in Node_B.

> eth0/arp_filter
> 1
> eth1/arp_filter
> 1

	Start with fixing your picture (all hubs and wires, please).
Linux ARP never sends 1 broadcasts through 2 devices, so it seems there is 
a hub near Node_A (or Node_A is running bridging), I can't believe 230.1 
and 230.4 are directly connected with cross cable. Make sure you have
the needed host/subnet routes for each interface. Also, make
sure tcpdump really shows the ARP replies, make the tests with
arp -d IP ; ping -c 1 IP

Regards

--
Julian Anastasov <ja@ssi.bg>

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[Bill Davidsen]

On Sun, 14 Jul 2002, Julian Anastasov wrote:

> 	Start with fixing your picture (all hubs and wires, please).
> Linux ARP never sends 1 broadcasts through 2 devices, so it seems there is 
> a hub near Node_A (or Node_A is running bridging), I can't believe 230.1 
> and 230.4 are directly connected with cross cable. Make sure you have
> the needed host/subnet routes for each interface. Also, make
> sure tcpdump really shows the ARP replies, make the tests with
> arp -d IP ; ping -c 1 IP

The tests were made with a copy of tcpdump on every NIC of every machine
involved, with a hardware sniffer on each of the NICs. It was analyzed by
ISP network gurus, and it appears that it really does happen, and both
D-Link and Cisco switches don't like it when it does.

The problem was solved by substituting a more capable node.

Since I'm told by network people that current behaviour is the desired
behaviour, by definition there is no bug. By a mixture of diddling flags
in /proc/sys and hand routing every address through the desired NIC you
can make it work, so "no problem."

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

[BUG?] unwanted proxy arp in 2.4.19-pre10

[Bill Davidsen]

I think there's a bug, or at least unexpected behaviour in 2.4.19-pre10
regarding ARP replies. I am getting multiple arp-replies to a who-has
request, and there seems to be no check in arp.c to verify that the
interface used for an arp reply actually has that address or has proxy
set.

Details:


     node_A 192.168.230.1              router
                |       \               192.168.231.127
		|	 \		192.168.0.0/16 access
                |      bridge            /
                |           \           /
                |            \         /
          192.168.230.4      192.168.231.4
                       \      /
                        node_B


When node_A does an arp request, who-has 192.168.230.4, it gets a
correct answer from the NIC with that IP. It also gets a reply from the
NIC on the 192.168.231 IP, because the ARP broadcast was bridged to that
NIC and there's no check to see if that NIC actually has the IP in
question. Since the networks are bridged for the moment, the 2nd reply
also arrives, later, and winds up in the arp table on node_A, where it
results in all traffic going through the bridge to the wrong NIC.

In the absense of the proxy_arp flag, I would not expect that reply,
the IP is not on that NIC. Before I "fix" that, is this intended
behaviour for some reason? Will I break something if I add check logic?
Is there something in /proc/sys/net/ipv4 I missed which will avoid this
response?

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[Alan Cox]

On Sat, 2002-07-13 at 17:21, Bill Davidsen wrote:
> In the absense of the proxy_arp flag, I would not expect that reply,
> the IP is not on that NIC. Before I "fix" that, is this intended
> behaviour for some reason? Will I break something if I add check logic?
> Is there something in /proc/sys/net/ipv4 I missed which will avoid this
> response?

Your suspicion and the reality don't match. The RFC's leave the
situation unclear and some OS's do either. Newer 2.4 has arpfilter which
can be used to control what actually occurs

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[Gerhard Mack]

On 13 Jul 2002, Alan Cox wrote:

> On Sat, 2002-07-13 at 17:21, Bill Davidsen wrote:
> > In the absense of the proxy_arp flag, I would not expect that reply,
> > the IP is not on that NIC. Before I "fix" that, is this intended
> > behaviour for some reason? Will I break something if I add check logic?
> > Is there something in /proc/sys/net/ipv4 I missed which will avoid this
> > response?
>
> Your suspicion and the reality don't match. The RFC's leave the
> situation unclear and some OS's do either. Newer 2.4 has arpfilter which
> can be used to control what actually occurs

Can we at least have matching defaults for ipv4 and ipv6 ??  Having ipv6
behave the opposite just isn't intuitive.

	Gerhard



--
Gerhard Mack

gmack@innerfire.net

<>< As a computer I find your faith in technology amusing.

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[Alan Cox]

On Sat, 2002-07-13 at 20:19, Gerhard Mack wrote:
> > Your suspicion and the reality don't match. The RFC's leave the
> > situation unclear and some OS's do either. Newer 2.4 has arpfilter which
> > can be used to control what actually occurs
> 
> Can we at least have matching defaults for ipv4 and ipv6 ??  Having ipv6
> behave the opposite just isn't intuitive.

IPv6 doesn't have ARP it has neighbour discovery. The two are very
different in quite a few ways. 

Alan

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[Bill Davidsen]

On 13 Jul 2002, Alan Cox wrote:

> On Sat, 2002-07-13 at 17:21, Bill Davidsen wrote:
> > In the absense of the proxy_arp flag, I would not expect that reply,
> > the IP is not on that NIC. Before I "fix" that, is this intended
> > behaviour for some reason? Will I break something if I add check logic?
> > Is there something in /proc/sys/net/ipv4 I missed which will avoid this
> > response?
> 
> Your suspicion and the reality don't match. The RFC's leave the
> situation unclear and some OS's do either. Newer 2.4 has arpfilter which
> can be used to control what actually occurs

I tried setting conf/arp_filter, proxy_arp, and looked at rp_filter but
didn't try anything with it. I'm using tcpdump on the machine sending
who-has and getting two packets back. I tried the obvious setting eth0 and
1, setting default, and setting 'all." The current settings, just the NICs
in question, are producing two arp-replies with settings:

newsmst01:conf# for n in */arp_filter;do echo $n; cat $n; done
all/arp_filter
0
default/arp_filter
0
eth0/arp_filter
1
eth1/arp_filter
1
lo/arp_filter
0
newsmst01:conf# 

This was with 2.4.19-pre10ac2+one smp locking patch.

Oh well, thanks anyway, if it's intended to work that way I'll look at
making it so.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[David S. Miller]

You have to use specific source-routing settings in conjuntion with
enabling arp_filter in order for arp_filter to have any effect.

This is a FAQ.

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[dean gaudet]

On Sat, 13 Jul 2002, David S. Miller wrote:

>
> You have to use specific source-routing settings in conjuntion with
> enabling arp_filter in order for arp_filter to have any effect.
>
> This is a FAQ.

a couple google queries yielded no answer to this faq... is there a posted
example somewhere?

is the default behaviour of use to anyone?  this question comes up like
every other month.

-dean

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[Bill Davidsen]

On Sun, 14 Jul 2002, dean gaudet wrote:

> On Sat, 13 Jul 2002, David S. Miller wrote:
> 
> >
> > You have to use specific source-routing settings in conjuntion with
> > enabling arp_filter in order for arp_filter to have any effect.
> >
> > This is a FAQ.
> 
> a couple google queries yielded no answer to this faq... is there a posted
> example somewhere?

Clearly FAQ means frequently asked, not answered. I can't find the
appropriate patch, clearly some people regard allowing source routing to
be a benefit.
 
> is the default behaviour of use to anyone?  this question comes up like
> every other month.

Yes, it's useful to hackers to send a packet to your external interface
with the address of your internal internal interface, if the packet is
accepted it might bypass some of your firewall protections. I assume the
source routing settings David mentions are the normal "if the destination
IP is not this machine log and drop the packet." With a few possible
exceptions for packets other than normal tcp/udp. That's why the bogus arp
is harmeful, the requestor then tries to use the mac address for the wrong
IP and the packets just get dropped by the firewall code.

Like you, I can't see why it would be anything but a bug to do what is
essentially proxy arp by default.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[David S. Miller]

   From: Bill Davidsen <davidsen@tmr.com>
   Date: Sun, 14 Jul 2002 21:39:12 -0400 (EDT)

   Clearly FAQ means frequently asked, not answered. I can't find the
   appropriate patch, clearly some people regard allowing source routing to
   be a benefit.
    
Source routing exists in every single 2.4.x kernel every released.
What are you talking about?  There is no patch to speak of, it's
already there.

Franks a lot,
David S. Miller
davem@redhat.com

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[Bill Davidsen]

On Sun, 14 Jul 2002, David S. Miller wrote:

>    From: Bill Davidsen <davidsen@tmr.com>
>    Date: Sun, 14 Jul 2002 21:39:12 -0400 (EDT)
> 
>    Clearly FAQ means frequently asked, not answered. I can't find the
>    appropriate patch, clearly some people regard allowing source routing to
>    be a benefit.
>     
> Source routing exists in every single 2.4.x kernel every released.
> What are you talking about?  There is no patch to speak of, it's
> already there.

Um, many people who are being probed from the Internet would like very
much to NOT have it there, certainly by default. And would like to send an
arp request and in return get a valid mac address.

I totally miss why anyone would consider this behaviouracceptable, much
less desirable. Perhaps you or someone could explain why either accepting
source routing or sending out invalid arp responses are desirable at all,
much less as default behaviour. One is a security hole, the other brings
the network group to the door with arpwatch output.

After some hours of reading the google results I see lots of questions, a
few dubious workarounds, and zero people claiming that it's a good thing
to work that way. 

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: [BUG?] unwanted proxy arp in 2.4.19-pre10

[Bill Davidsen]

On Sat, 13 Jul 2002, David S. Miller wrote:

> 
> You have to use specific source-routing settings in conjuntion with
> enabling arp_filter in order for arp_filter to have any effect.
> 
> This is a FAQ.

Frequently asked, but all I find is complex ways to work around the bug
rather than any patches. I do have the source routing settings in place,
virtually all packets sent to an IP not on the NIC are loggged and
droppped, so I won't have a problem with spoofing. I did turn off the
firewall on a machine to check the problem, in practice all the packets
with incorrect MAC addresses would be dropped.

I fear someone with less draconian firewalls might accept an internal IP
address on an external NIC, however. I get about 800 log entries a month
on some machines, and they're behind a boundary router.

I thought I was missing something, clearly this is a known problem.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.