[LARTC] routing to two interfaces

[LARTC] routing to two interfaces

[Paco Brufal]

Hello,

	Sorry if this question is very common, but I searched the
maillist archive and didn't found an answer...

	I have a linux box with 3 interfaces, 2 of them have public IPs
(eth1 and eth2), and the third is a private IP (our LAN). I want to do the
following: if a packet is coming from eth1, it must be forwarded to eth0,
and when it comes back, it must be routed to eth1. In case of packet comes
from eth2, it must be forwarded to eth0, and the respose must be routed to
eth2. In other words, a packet must leave our network by the interface it
come.

	I tried several combinations of iptables, 'ip rule' and 'ip
route', but it didn't work...

	I appreciate any help, thanks :)

-- 

...Raggabum (Dj Isaac Remix). Triplet. 1996
--- Debian + Mutt + Postfix
 * Origin: Web Page: http://pbrufal.kleenux.org (Fido 2:346/7.68)
 
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] routing to two interfaces

[Martin A. Brown]

Paco,

IP routing decisions are stateless.

You'll need to define your problem a bit more clearly (for yourself).  
Once you understand exactly what you are trying to do (why you are 
splitting the traffic this way) you'll be able to answer the question you 
pose.

If you wish to look at some simple examples of split access, see my 
chapter on advanced routing:

  http://plorf.net/linux-ip/html/adv-routing.htm

For outbound split access (or inbound), see these sections in particular.

  http://plorf.net/linux-ip/html/adv-routing.htm#ADV-MULTI-INTERNET-OUTBOUND
  http://plorf.net/linux-ip/html/adv-routing.htm#ADV-MULTI-INTERNET-INBOUND

It doesn't sound like you wish to use a multipath route, but if you do, 
you should probably read the LARTC docs on load sharing split access:

  http://lartc.org/howto/lartc.rpdb.multiple-links.html

Good luck,

-Martin

 : Hello,
 : 
 : 	Sorry if this question is very common, but I searched the
 : maillist archive and didn't found an answer...
 : 
 : 	I have a linux box with 3 interfaces, 2 of them have public IPs
 : (eth1 and eth2), and the third is a private IP (our LAN). I want to do the
 : following: if a packet is coming from eth1, it must be forwarded to eth0,
 : and when it comes back, it must be routed to eth1. In case of packet comes
 : from eth2, it must be forwarded to eth0, and the respose must be routed to
 : eth2. In other words, a packet must leave our network by the interface it
 : come.
 : 
 : 	I tried several combinations of iptables, 'ip rule' and 'ip
 : route', but it didn't work...
 : 
 : 	I appreciate any help, thanks :)
 : 
 : 

-- 
Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] routing to two interfaces

[Robert Felber]

On Wed, Nov 20, 2002 at 11:55:37PM +0100, Paco Brufal wrote:
> 	I have a linux box with 3 interfaces, 2 of them have public IPs
> (eth1 and eth2), and the third is a private IP (our LAN). I want to do the
> following: if a packet is coming from eth1, it must be forwarded to eth0,
> and when it comes back, it must be routed to eth1. In case of packet comes
> from eth2, it must be forwarded to eth0, and the respose must be routed to
> eth2. In other words, a packet must leave our network by the interface it
> come.

try out following:
it's just an idea and i didn't try it out

# delete tables 2 and 3 to avoid duplicates
# add for each table the gateway to use
$IP route del table 2
$IP route add table 2 via $ETH1_GATEWAY_IP dev eth1
$IP route del table 3
$IP route add table 3 via $ETH2_GATEWAY_IP dev eth2 

# mark each packet regarding to its incomming device
$IPTABLES -t mangle -A INPUT -i eth1 -j MARK --set-mark 2
$IPTABLES -t mangle -A INPUT -i eth2 -j MARK --set-mark 3

# delete to avoid duplicates
# lookup the right table regarding to packets fwmark setting
# flush the routing cache
$IP rule del fwmark 2 table 2
$IP rule del fwmark 3 table 3
$IP rule add fwmark 2 table 2
$IP rule add fwmark 3 table 3
$IP route flush cache


--
rob
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] routing to two interfaces

[Robert Felber]

i missed the nat rule though:

$IPTABLES -t nat -A POSTROUTING -s 192.168.0.0/24 -o $ETH1 -j SNAT \
	--to-source $ETH1_GATEWAY_IP
$IPTABLES -t nat -A POSTROUTING -s 192.168.0.0/24 -o $ETH2 -j SNAT \
	--to-source $ETH2_GATEWAY_IP 

> try out following:
> it's just an idea and i didn't try it out
> 
> # delete tables 2 and 3 to avoid duplicates
> # add for each table the gateway to use
> $IP route del table 2
> $IP route add table 2 via $ETH1_GATEWAY_IP dev eth1
> $IP route del table 3
> $IP route add table 3 via $ETH2_GATEWAY_IP dev eth2 
> 
> # mark each packet regarding to its incomming device
> $IPTABLES -t mangle -A INPUT -i eth1 -j MARK --set-mark 2
> $IPTABLES -t mangle -A INPUT -i eth2 -j MARK --set-mark 3
> 
> # delete to avoid duplicates
> # lookup the right table regarding to packets fwmark setting
> # flush the routing cache
> $IP rule del fwmark 2 table 2
> $IP rule del fwmark 3 table 3
> $IP rule add fwmark 2 table 2
> $IP rule add fwmark 3 table 3
> $IP route flush cache
> 
> 
> --
> rob
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
> 
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] routing to two interfaces

[Martin A. Brown]

Rob,

There's a problem with your solution!

fwmark; transient
- - - - - - - - - - - 
The structure of the packet as it passes through the firewall/router 
contains the fwmark.  As soon as the packet leaves the box, it no longer 
has the fwmark.

Your solution handles the packets inbound from the outside world, but 
neglects to handle the outbound packets from the internal network.

SNAT; sets the correct source IP (for outbound connections)
- - - - - - - - - - - - - - - -
Even if using SNATs as you suggest, there is still has no way to tell if a
packet belongs to a session inbound over eth1 or eth2.  This is the
statelessness of IP routing!

scenarios
- - - - - - - - - - -
For example, maybe Paco has an RFC1918 addressed server which s/he wants
to make available on two different public IPs.  This requires one
solution.  (inbound problem; SNAT can't help; DNAT maybe)

Perhaps Paco wants to send some traffic out one link, and some out the
other link--yet another solution.  (generic policy routing)

Finally, a link load sharing split access solution is different yet. 
(multipath route)


In order to make any recommendation, we would need to know what the IP 
address ranges are and specifically why/how Paco envisions using these 
two links.

-Martin

 : i missed the nat rule though:
 :
 : $IPTABLES -t nat -A POSTROUTING -s 192.168.0.0/24 -o $ETH1 -j SNAT \
 :       --to-source $ETH1_GATEWAY_IP
 : $IPTABLES -t nat -A POSTROUTING -s 192.168.0.0/24 -o $ETH2 -j SNAT \
 :       --to-source $ETH2_GATEWAY_IP 
 :
 : try out following:
 : it's just an idea and i didn't try it out
 : 
 : # delete tables 2 and 3 to avoid duplicates
 : # add for each table the gateway to use
 : $IP route del table 2
 : $IP route add table 2 via $ETH1_GATEWAY_IP dev eth1
 : $IP route del table 3
 : $IP route add table 3 via $ETH2_GATEWAY_IP dev eth2 
 : 
 : # mark each packet regarding to its incomming device
 : $IPTABLES -t mangle -A INPUT -i eth1 -j MARK --set-mark 2
 : $IPTABLES -t mangle -A INPUT -i eth2 -j MARK --set-mark 3
 : 
 : # delete to avoid duplicates
 : # lookup the right table regarding to packets fwmark setting
 : # flush the routing cache
 : $IP rule del fwmark 2 table 2
 : $IP rule del fwmark 3 table 3
 : $IP rule add fwmark 2 table 2
 : $IP rule add fwmark 3 table 3
 : $IP route flush cache
 : 
 : 
 : --
 : rob
 : _______________________________________________
 : LARTC mailing list / LARTC@mailman.ds9a.nl
 : http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
 : 

-- 
Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] routing to two interfaces

[Robert Felber]

On Wed, Nov 20, 2002 at 05:45:29PM -0600, Martin A. Brown wrote:
> There's a problem with your solution!
> 
> fwmark; transient
> - - - - - - - - - - - 
> The structure of the packet as it passes through the firewall/router 
> contains the fwmark.  As soon as the packet leaves the box, it no longer 
> has the fwmark.
> 
> Your solution handles the packets inbound from the outside world, but 
> neglects to handle the outbound packets from the internal network.
>
> SNAT; sets the correct source IP (for outbound connections)
> - - - - - - - - - - - - - - - -
> Even if using SNATs as you suggest, there is still has no way to tell if a
> packet belongs to a session inbound over eth1 or eth2.  This is the
> statelessness of IP routing!
> 
> In order to make any recommendation, we would need to know what the IP 
> address ranges are and specifically why/how Paco envisions using these 
> two links.

Yes, true. I admit i didn't think long enough about it.
Well actually, i think he just wants the packets coming in eth1
will go out eth1 again, and the same for eth2. Nothing more nothing
less. I had kind of the same problem but with the restriction that
i had one extranet device with a limited set of subnets and one 
internet device and one lan device so it was easy because i could set
proper routes for the affected intranet subnets. Well, anyway.
I suggest to setup a virtual eth0:1 device. Packets from eth1 leave then
at eth0:0 and packets from eth2 leave at eth0:1. Then he should be able
to set proper gateways and nats for eth0:x device.

--
rob

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] routing to two interfaces

[Robert Felber]

On Thu, Nov 21, 2002 at 01:04:51AM +0100, Robert Felber wrote:
> I suggest to setup a virtual eth0:1 device. Packets from eth1 leave then
> at eth0:0 and packets from eth2 leave at eth0:1. Then he should be able
> to set proper gateways and nats for eth0:x device.
ok, another wrong though. the packets back from LAN will arrive at the
client's default gateway. So this is no solution either.

--
rob
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] routing to two interfaces

[Paco Brufal]

On nov/20/2002, Martin A. Brown wrote:

> packet belongs to a session inbound over eth1 or eth2.  This is the
> statelessness of IP routing!

	I'm thinking in one possibility, but I'm really new with iproute and
I don't know if it is possible...

	eth0 has the IP 10.10.10.1, and I create an alias eth0:1 with IP
10.10.10.2. With 'ip', I route packets from eth1 to eth0, and packets from
eth2 to eth0:1. When these packets returns from the LAN, they go to an
especific IP (10.10.10.1 or 10.10.10.2). Then I mark these packets with
iptables (maybe in PREROUTING?), i.e., packets to eth0 with mark X, and
packets to eth0:1 with mark Y, and route these packets with 'ip route'
looking the mark of the packet (mark X -> eth1, mark Y -> eth2). The last
action is to MASQUERADE the packets for each interface...

> In order to make any recommendation, we would need to know what the IP 
> address ranges are and specifically why/how Paco envisions using these 
> two links.

	The two public interfaces aren't in the same range (80.37... and
80.59...). The purpose of this Linux box is to provide high availability to
several servers, but the two public interfaces of this box may work at the
same time. eth1 handles DNS traffic, and eth2 handles SMTP and HTTP
traffic. When one of the link goes down, the other may take all traffic (we
detect the link-down and change the DNS to point to the working interface).

-- 

...Bonkers (Stunned Guys Mix). The Riders. 1996
--- Debian + Mutt + Postfix
 * Origin: Web Page: http://pbrufal.kleenux.org (Fido 2:346/7.68)
 
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] routing to two interfaces

[Paco Brufal]

On nov/21/2002, Robert Felber wrote:

> ok, another wrong though. the packets back from LAN will arrive at the
> client's default gateway. So this is no solution either.

	But the client (10.10.10.3) is in the same network of the eth0 (or
eth0:1) interface, then it doesn't uses default gateway, it simply replies
to the source IP (10.10.10.1 or 10.10.10.2), or I'm wrong?

-- 

...Integator. Dj Dave Forbes. 1996
--- Debian + Mutt + Postfix
 * Origin: Web Page: http://pbrufal.kleenux.org (Fido 2:346/7.68)
 
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] routing to two interfaces

[Martin A. Brown]

Paco,

Let me summarize what I think you want to do.

 : 	The two public interfaces aren't in the same range (80.37... and
 : 80.59...). The purpose of this Linux box is to provide high availability to
 : several servers, but the two public interfaces of this box may work at the
 : same time. 

 - linux-box has public IPs 80.37.x.x and 80.59.x.x


 : eth1 handles DNS traffic, and eth2 handles SMTP and HTTP traffic.

 - clarification needed.  you are selecting eth1 for outbound DNS?
   you are selecting eth2 for outbound SMTP and HTTP?


 : When one of the link goes down, the other may take all traffic (we
 : detect the link-down and change the DNS to point to the working
 : interface).

So, you have server(s) in your internal network which need to be 
accessible from the outside world on either IP-A or IP-B.  Perhaps the 
same set of services on each public IP, correct?

If I interpret correctly, what you are describing is not high
availability, that's just putting services on two different public IPs.  
Sure, you buy yourself some insurance by having the service available on
two different networks, but it's still not HA.  For reference, check out
the linux high availability project (link below).


 : 	eth0 has the IP 10.10.10.1, and I create an alias eth0:1 with IP
 : 10.10.10.2. With 'ip', I route packets from eth1 to eth0, and packets from
 : eth2 to eth0:1. When these packets returns from the LAN, they go to an
 : especific IP (10.10.10.1 or 10.10.10.2). Then I mark these packets with
 : iptables (maybe in PREROUTING?), i.e., packets to eth0 with mark X, and
 : packets to eth0:1 with mark Y, and route these packets with 'ip route'
 : looking the mark of the packet (mark X -> eth1, mark Y -> eth2). The last
 : action is to MASQUERADE the packets for each interface...

I don't think this will work.  First, it doesn't matter how many IPs you 
configure on your eth0 for transmitting the packets into the internal 
network, nor what you use for default gateways on the internal hosts.  

Imagine:

  - server SOURCE has packet for outside address (DEST), looks up in 
    routing table; selects 10.10.10.1 (or 10.10.10.2) as a default gateway
  - server looks up 10.10.10.1 (or .2) in ARP cache or with ARP request
  - server transmits ethernet frame with IP payload and addresses SOURCE 
    and DEST
  - linux-box gets packet with SOURCE and DEST
  - linux-box now needs to make routing decision

I don't see how multiple IPs bound to an ethernet interface solves any 
problem.  I think it unnecessarily complicates your solution.

So, I'd suggest (again) reading the following:

  http://plorf.net/linux-ip/html/adv-routing.htm#ADV-MULTI-INTERNET-INBOUND

If you don't like the way I wrote it, I'd love to hear what you think is 
missing, but I'll refer you to this:

  http://lists.netfilter.org/pipermail/netfilter/2001-May/011697.html

Good luck, Paco.  I'd suggest returning here if you have specific troubles 
after you have digested these, and understand how they can help solve your 
current problem.

-Martin

     linux high availability:  http://linux-ha.org/ 

-- 
Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] routing to two interfaces

[Arthur van Leeuwen]

On Wed, 20 Nov 2002, Paco Brufal wrote:

> 	Sorry if this question is very common, but I searched the
> maillist archive and didn't found an answer...

Did you read the HOWTO?

> 	I have a linux box with 3 interfaces, 2 of them have public IPs
> (eth1 and eth2), and the third is a private IP (our LAN). I want to do the
> following: if a packet is coming from eth1, it must be forwarded to eth0,
> and when it comes back, it must be routed to eth1. In case of packet comes
> from eth2, it must be forwarded to eth0, and the respose must be routed to
> eth2. In other words, a packet must leave our network by the interface it
> come.

Okay, given the fact that you are using private IP space in your LAN this
should be very doable using NAT and connection tracking. Read up on the
`Using multiple uplinks' sections in the HOWTO.

What I don't really understand though is that you seem to want to allow
connections from *outside* to either eth1 or eth2, routing those through to
the internal network, and then be able to route the stuff back. The only
simple solution to that problem I can see is to use application level proxies
on the firewall. Thereby you can suddenly let the application proxy handle the
correct forwarding and keep the necessary state on the firewall.

Doei, Arthur.

-- 
  /\    / |      arthurvl@sci.kun.nl      | Work like you don't need the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there's nobody watching

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] routing to two interfaces

[Paco Brufal]

On nov/20/2002, Martin A. Brown wrote:

> Good luck, Paco.  I'd suggest returning here if you have specific troubles 
> after you have digested these, and understand how they can help solve your 
> current problem.

	OK, I read what you suggested me, and the documents didn't answer my
question, but they gave me some ideas, and now all is working like I want :)

	I have the luck that each service in the internal network has it own
IP, then I set up some iptables rules to redirect (via DNAT) the external
packets to internal servers. Then, in the router box, I set up routing
tables based on source hosts (the internal servers). When a packet arrives
from 10.10.10.2 (DNS), route it via eth1, if packet comes from 10.10.10.3
(HTTP) or 10.10.10.4 (SMTP), route it via eth2.

	If somebody is interested in the exact commands, I can post the
script I used...

	Thanks to everybody :)

-- 

...Slamma Jamma. Dj Omar Santana. 1996
--- Debian + Mutt + Postfix
 * Origin: Web Page: http://pbrufal.kleenux.org (Fido 2:346/7.68)
 
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/