How do we arp for NAT? Secondary IPs, proxy arp? something else?

How do we arp for NAT? Secondary IPs, proxy arp? something else?

[Jesse Molina]

Hello

I've googled all over and I don't really see an obvious answer to the 
question that I have.

Here is my situation:  I have a GNU/Linux host performing very typical 
firewall duties;  two interfaces, one with an Internet public IP and 
another interface on an RFC1918 net.  Hosts on the RFC1918 net have 
iptables SNATs to public IPs and then I filter to allow some services in 
and others not, with stateful inspection in forwarding.

Normally, in order to get the multiple public IPs for these SNAT'ed 
hosts to respond to arp requests from the firewall, I simply add them as 
secondary IPs on the public interface of the firewall (eth0:1, eth0:2,...).

The problem with this is that the firewall itself runs some services and 
they have the potential to use these secondary IPs as their ephemeral 
source addresses when they reach out to something on the Internet! 
That's bad, as those IPs should be exclusively used by only the hosts 
for which they were designed for.  Assume I have no control over the 
applications which bind to a local interface to use for their outbound 
session traffic.

It seems like using these secondary addresses is not the right thing to do.

Is there a better way to make the firewall arp for these public IPs that 
are SNAT mapped to the internal RFC1918 IPs?  This is a little like 
proxy arp, but that involves the same layer 3 IP network physically 
split by interfaces (think dialup NAS), and since these are different 
networks, I don't imagine this is applicable.

What else is there?  Loop interfaces with proxy arping?  I've been 
reading about some functionality for NAT in the ip tool (ip route add 
nat ...) but it looks depreciated.  There also seems to be something 
like "ip rule add nat ..." but I've not figured that out yet.  I had 
read somewhere that "ip route add nat ..." specifically would arp for 
the translated address, but again, the man pages says that's depreciated 
in the 2.6 kernel.

How do others handle this?



-- 
# Jesse Molina
# Mail = jesse@opendreams.net
# Page = page-jesse@opendreams.net
# Cell = 1.602.323.7608
# Web  = http://www.opendreams.net/jesse/

Re: How do we arp for NAT? Secondary IPs, proxy arp? something else?

[Tore Anderson]

Hi Jesse,

* Jesse Molina

> What else is there?  Loop interfaces with proxy arping?  I've been
> reading about some functionality for NAT in the ip tool (ip route add
> nat ...) but it looks depreciated.  There also seems to be something
> like "ip rule add nat ..." but I've not figured that out yet.  I had
> read somewhere that "ip route add nat ..." specifically would arp for
> the translated address, but again, the man pages says that's depreciated
> in the 2.6 kernel.

I'd simply route the IP adresses used for NAT to your Linux-based
firewall, if I were you.  That way you'll only need a /30 link network
to be used on the public interface, while the addresses used for NAT do
not have to be local to the firewall in any way.  As an added bonus
you'll get less ARP traffic on the public interface, as the upstream
router only needs to learn the L2-address of the next-hop router (your
firewall, that is).

BR,
-- 
Tore Anderson
Redpill Linpro AS - http://www.redpill-linpro.com/

Re: How do we arp for NAT? Secondary IPs, proxy arp? something else?

[Jesse Molina]

That's a pretty good suggestion, but it's more of a workaround than 
something that actually addresses the issue at hand.  I'm looking for a 
solution on the GNU/Linux host, not in the world around it.

To restate my question:  What alternative ways are there to make the 
GNU/Linux system reply to ARP requests for an IP, without that IP being 
an actual interface on the host, or that interface must not be used by 
local services *in any way*, for the reasons of using it via SNAT/DNAT?

Here is an example where the solution you suggested would not work:  I 
have a Qwest ADSL line with a /29 network.  That's what we have, and 
it's not going to change.  Qwest will not issue you a /30 for the 
point-to-point between the ADSL router device and your GNU/Linux 
firewall.  The ADSL router's filtering and firewall capabilities suck or 
just don't exist.  A bridge firewall would work here, but we could not 
use NAT and RFC1918 addresses.  We have 100 actual hosts on that RFC1918 
network, but only four of them need a public resource, and they are all 
tcp/80 web servers.

All commercial firewall products that I know of can do this.  You don't 
give your Cisco ASA/PIX a secondary IP -- the nat or static statement 
induces the host to ARP for the IP that you have assigned for the 
translation.  Same thing with Checkpoint, same thing with NetScreen.

Thanks for the suggestion though -- that's certainly a good one, but it 
still seems like there is functionality missing from the Linux kernel to 
handle this, or it's somewhere that I don't know of.



Tore Anderson wrote:
> Hi Jesse,
> 
> * Jesse Molina
> 
>> What else is there?  Loop interfaces with proxy arping?  I've been
>> reading about some functionality for NAT in the ip tool (ip route add
>> nat ...) but it looks depreciated.  There also seems to be something
>> like "ip rule add nat ..." but I've not figured that out yet.  I had
>> read somewhere that "ip route add nat ..." specifically would arp for
>> the translated address, but again, the man pages says that's depreciated
>> in the 2.6 kernel.
> 
> I'd simply route the IP adresses used for NAT to your Linux-based
> firewall, if I were you.  That way you'll only need a /30 link network
> to be used on the public interface, while the addresses used for NAT do
> not have to be local to the firewall in any way.  As an added bonus
> you'll get less ARP traffic on the public interface, as the upstream
> router only needs to learn the L2-address of the next-hop router (your
> firewall, that is).
> 
> BR,

-- 
# Jesse Molina
# Mail = jesse@opendreams.net
# Page = page-jesse@opendreams.net
# Cell = 1.602.323.7608
# Web  = http://www.opendreams.net/jesse/

Re: How do we arp for NAT? Secondary IPs, proxy arp? something else?

[Tore Anderson]

* Jesse Molina

> That's a pretty good suggestion, but it's more of a workaround than
> something that actually addresses the issue at hand.  I'm looking for a
> solution on the GNU/Linux host, not in the world around it.

Why is that?  Isn't making things work the most important thing to you?

Accurately setting up the routing tables (on all involved devices) isn't
a workaround, it's the most logical solution.  I'm sorry that you have
an uncooperative provider - that's the problem that requires a
"workaround", in my opinion.

> To restate my question:  What alternative ways are there to make the
> GNU/Linux system reply to ARP requests for an IP, without that IP being
> an actual interface on the host, or that interface must not be used by
> local services *in any way*, for the reasons of using it via SNAT/DNAT?

I'm not sure you can make Linux respond to ARP solicitations without
having a local IP address.  Perhaps you use arptables to mangle the
addresses used for NAT to/from the firewall's local address (in both the
inbound and the outbound directions) - no guarantees it will work
though, I never tried it.

You can also simply add the addresses to a local interface on the
firewall, so that ARP requests will be answered.  Preventing access to
local services running on the firewall through those addresses is easy,
just add rules to iptables' INPUT chain that discard any traffic
destined for them.

> Here is an example where the solution you suggested would not work:  I
> have a Qwest ADSL line with a /29 network.  That's what we have, and
> it's not going to change.  Qwest will not issue you a /30 for the
> point-to-point between the ADSL router device and your GNU/Linux
> firewall.

If you have admin access to the Qwest router, you could still use my
initial suggestion by adding static routes to the NAT-ed addresses using
the primary address of your firewall as the next-hop.  The more specific
routes will take precedence over the link-local /29, and things will
work just fine.  It feels a bit more like a hack this way, though.

Another option is to add static entries in the ARP table of the ADSL
router for the addresses used for NAT, that way you don't have to
persuade the firewall to reply to ARP for non-local adresses.

BR,
-- 
Tore Anderson
Redpill Linpro AS - http://www.redpill-linpro.com/

Re: How do we arp for NAT? Secondary IPs, proxy arp? something else?

[Mike Wright]

Jesse Molina wrote:
> 
> That's a pretty good suggestion, but it's more of a workaround than 
> something that actually addresses the issue at hand.  I'm looking for a 
> solution on the GNU/Linux host, not in the world around it.
> 
> To restate my question:  What alternative ways are there to make the 
> GNU/Linux system reply to ARP requests for an IP, without that IP being 
> an actual interface on the host, or that interface must not be used by 
> local services *in any way*, for the reasons of using it via SNAT/DNAT?

Hi Jesse,

Looking through the ebtables man pages this is found:

    arpreply
        The arpreply target can be used in the PREROUTING chain of the 
nat table.  If this target sees an ARP request it will automatically 
reply with an ARP reply. The used MAC address for the reply can be 
specified.

This is also available as a qualifier:
     --ip-destination   The destination IP address.

You'll need to catch the specified macs when the traffic comes in. 
Those targets are also available in ebtables.

Seems like all the pieces you want are available.

Hope that is useful in some way.

:m)

Re: How do we arp for NAT? Secondary IPs, proxy arp? something else?

[Pascal Hambourg]

Hello,

Jesse Molina a écrit :
> 
> To restate my question:  What alternative ways are there to make the 
> GNU/Linux system reply to ARP requests for an IP, without that IP being 
> an actual interface on the host, or that interface must not be used by 
> local services *in any way*, for the reasons of using it via SNAT/DNAT?

ip route add local <address>/<mask> table local dev <interface>

This way <address>/<mask> will be considered local by the system which 
will reply to ARP requests for it, actually usable by any local process, 
but won't appear assigned to <interface> so chances are that no local 
process will use it unless told explicitly.

Re: How do we arp for NAT? Secondary IPs, proxy arp? something else?

[Jesse Molina]

BAMM!

That's exactly what I was looking for.  I'll have to give that a try and 
see if the behavior is as expected, but the way you describe it, that's 
exactly what I was looking for.

I'm really surprised that this isn't an issue that has come up before, 
especially since security is involved.

I am very grateful for your reply.



Pascal Hambourg wrote:
> Hello,
> 
> Jesse Molina a écrit :
>>
>> To restate my question:  What alternative ways are there to make the 
>> GNU/Linux system reply to ARP requests for an IP, without that IP 
>> being an actual interface on the host, or that interface must not be 
>> used by local services *in any way*, for the reasons of using it via 
>> SNAT/DNAT?
> 
> ip route add local <address>/<mask> table local dev <interface>
> 
> This way <address>/<mask> will be considered local by the system which 
> will reply to ARP requests for it, actually usable by any local process, 
> but won't appear assigned to <interface> so chances are that no local 
> process will use it unless told explicitly.
> -- 
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
# Jesse Molina
# Mail = jesse@opendreams.net
# Page = page-jesse@opendreams.net
# Cell = 1.602.323.7608
# Web  = http://www.opendreams.net/jesse/

Re: How do we arp for NAT? Secondary IPs, proxy arp? something else?

[Jesse Molina]

Just wanted to reply again for documentation purposes regarding this old 
thread.  This was EXACTLY what I was looking for.

Here are some sample commands to replicate this;

# Add the IP 1.2.3.4 to eth1
# sudo ip route add local 1.2.3.4 table local dev eth1
#
# Show the local table
# sudo ip route show table local
#
# Delete the IP 1.2.3.4 from eth1
# sudo ip route delete local 1.2.3.4 table local dev eth1



I have not observed any daemon or service using the IPs that I 
configured with this method for ephemeral (outbound) services.  My 
upstream packet accounting confirms this.

Yes, the system replies to ARP requests for these IPs.

Unfortunately, local daemons that glob onto all addresses are still 
answering to services on these IPs if you don't filter or re-direct the 
traffic via iptables DNAT/SNAT, but that's not a major issue since it 
can be controlled as mentioned here.



I have been using this for a couple of weeks, works great.  I recommend 
this procedure over using secondary IPs on regular interfaces.

I hope this is useful to someone in the future.



Pascal Hambourg wrote:
> Hello,
> 
> Jesse Molina a écrit :
>>
>> To restate my question:  What alternative ways are there to make the 
>> GNU/Linux system reply to ARP requests for an IP, without that IP 
>> being an actual interface on the host, or that interface must not be 
>> used by local services *in any way*, for the reasons of using it via 
>> SNAT/DNAT?
> 
> ip route add local <address>/<mask> table local dev <interface>
> 
> This way <address>/<mask> will be considered local by the system which 
> will reply to ARP requests for it, actually usable by any local process, 
> but won't appear assigned to <interface> so chances are that no local 
> process will use it unless told explicitly.
> -- 
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
# Jesse Molina
# Mail = jesse@opendreams.net
# Page = page-jesse@opendreams.net
# Cell = 1.602.323.7608
# Web  = http://www.opendreams.net/jesse/

Re: How do we arp for NAT? Secondary IPs, proxy arp? something else?

[Jesse Molina]

Assume we have no control over the services -- they pick any old IP (and 
I assure you, they do, this isn't just academic) from that public-facing 
IP physical interface and start using ephemeral ports, which then can 
conflict with RPC or whatever for the SNAT/DNAT host that the IP is 
supposed to exclusively support.

Furthermore, let's say that an upstream firewall counts packets from 
source/destination IPs for the purposes of auditing traffic.  By a local 
service on the GNU/Linux firewall using these IPs that they are not 
supposed to, it creates the false illusion that the host for which the 
IP is supposed to be exclusively used for is sending/receiving traffic. 
  If we had such upstream filtering going on, this could even 
permit/deny traffic unintended from policy.

There might be some other unintended consequences that I've not thought 
of.  Either way, it just seems wrong that there isn't a way to do this 
if GNU/Linux is going intended to be used as a modern fully featureless 
firewall that can perform NAT/PAT.



M**** K**** wrote:
> Hello,
> 
>> The problem with this is that the firewall itself runs some services
>> and they have the potential to use these secondary IPs as their
>> ...
>> It seems like using these secondary addresses is not the right thing to
>> do.
> 
> One non-invasive approach is to launch local services on firewall box
> specifying bind address (0.0.0.0 is default and binds to all
> addressess).
> 
> Method mentioned by Tore Anderson is the cleanest way out, but you'd
> have to persuade your upstream provider to route a subnet of public IPs
> throu your firewall box. This way you wouldn't have to assing secondary
> addresses for SNAT/DNAT to work.
> 
> Cheers,

-- 
# Jesse Molina
# Mail = jesse@opendreams.net
# Page = page-jesse@opendreams.net
# Cell = 1.602.323.7608
# Web  = http://www.opendreams.net/jesse/

Re: How do we arp for NAT? Secondary IPs, proxy arp? something else?

[Robert Nichols]

Jesse Molina wrote:
> 
> Hello
> 
> I've googled all over and I don't really see an obvious answer to the 
> question that I have.
> 
> Here is my situation:  I have a GNU/Linux host performing very typical 
> firewall duties;  two interfaces, one with an Internet public IP and 
> another interface on an RFC1918 net.  Hosts on the RFC1918 net have 
> iptables SNATs to public IPs and then I filter to allow some services in 
> and others not, with stateful inspection in forwarding.
> 
> Normally, in order to get the multiple public IPs for these SNAT'ed 
> hosts to respond to arp requests from the firewall, I simply add them as 
> secondary IPs on the public interface of the firewall (eth0:1, eth0:2,...).
> 
> The problem with this is that the firewall itself runs some services and 
> they have the potential to use these secondary IPs as their ephemeral 
> source addresses when they reach out to something on the Internet! 
> That's bad, as those IPs should be exclusively used by only the hosts 
> for which they were designed for.  Assume I have no control over the 
> applications which bind to a local interface to use for their outbound 
> session traffic.

Packets that originate on the firewall machine itself go through the
OUTPUT chain.  Forwarded packets from the RFC1918 net do not.  Block
the packets in the OUTPUT chain of the filter table.

-- 
Bob Nichols     "NOSPAM" is really part of my email address.
                 Do NOT delete it.

Re: How do we arp for NAT? Secondary IPs, proxy arp? something else?

[Покотиленко Костик]

В Вск, 24/05/2009 в 03:37 -0700, Jesse Molina пишет:
> Hello
> 
> I've googled all over and I don't really see an obvious answer to the 
> question that I have.
> 
> Here is my situation:  I have a GNU/Linux host performing very typical 
> firewall duties;  two interfaces, one with an Internet public IP and 
> another interface on an RFC1918 net.  Hosts on the RFC1918 net have 
> iptables SNATs to public IPs and then I filter to allow some services in 
> and others not, with stateful inspection in forwarding.
> 
> Normally, in order to get the multiple public IPs for these SNAT'ed 
> hosts to respond to arp requests from the firewall, I simply add them as 
> secondary IPs on the public interface of the firewall (eth0:1, eth0:2,...).
> 
> The problem with this is that the firewall itself runs some services and 
> they have the potential to use these secondary IPs as their ephemeral 
> source addresses when they reach out to something on the Internet! 
> That's bad, as those IPs should be exclusively used by only the hosts 
> for which they were designed for.  Assume I have no control over the 
> applications which bind to a local interface to use for their outbound 
> session traffic.

Did you try to explicitely SNAT localy generated traffic to the right
address? It seem a solution for your task. You can do this for all or
selectively by -m owner.

-- 
Покотиленко Костик <casper@meteor.dp.ua>