* RE: [LARTC] Masq/route based on port
2001-12-06 9:58 [LARTC] Masq/route based on port Miron
@ 2001-12-06 15:18 ` Greg Scott
2001-12-06 19:03 ` Miron
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Greg Scott @ 2001-12-06 15:18 UTC (permalink / raw)
To: lartc
This doesn't seem right:
> My firewall configuration:
> iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK
> --set-mark 2
> iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128
> iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128
I'm not the expert, but I don't think you want to SNAT.
You're marking inbound packets with destination port 80 and then putting
rules in the POSTROUTING table to fudge in a different IP address for
outbound packets. That doesn't seem right. It doesn't redirect inbound
packets the way you want.
I think you want to DNAT instead of SNAT and forget about marking packets.
Set up some PREROUTING rules and DNAT all incoming port 80 stuff over to
the interface you want. That should be all you need to do because the
connection tracking should take care of getting the reply packets from your
internal web server back to where they belong.
Verify this with the experts before you do it, but I think I'm right on
this one.
- Greg Scott
-----Original Message-----
From: Miron [mailto:miron@hyper.to]
Sent: Thursday, December 06, 2001 3:58 AM
To: lartc@mailman.ds9a.nl
Subject: [LARTC] Masq/route based on port
I have following setup:
- eth0 is an internal network
- eth1 is an Internet connection (IP = 1.1.1.128, GW=1.1.1.1)
- eth2 is another Internet connection (IP = 2.2.2.128, GW=2.2.2.1)
I would like to masquerade port 80 through eth2, but all other traffic
should be masq'ed through eth1.
My routing configuration:
(default route in main table is 1.1.1.1)
ip rule add fwmark 2 pref 1002 table 666
ip route flush table 666
ip route add default via 2.2.2.1 dev eth3 proto static table 666
ip route flush cache
My firewall configuration:
iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK
--set-mark 2
iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128
iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128
Unfortunately, this does not work. Outgoing packets are fine. Incoming
packets on port 80 are not de-masqueraded and do not reach the internal
hosts.
Also, if I change the ip rule above to be based on the source address
(instead of a mark), connections start working fine.
Here is the output of 'ip rule ls', to prove that I do have fwmark compiled:
0: from all lookup local
1002: from all fwmark 2 lookup http
32766: from all lookup main
32767: from all lookup 253
I am wondering if there is some kind of bug related to the interaction
between fwmark and NAT. Any ideas?
Thanks,
Miron Cuperman
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO:
http://ds9a.nl/2.4Routing/
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [LARTC] Masq/route based on port
2001-12-06 9:58 [LARTC] Masq/route based on port Miron
2001-12-06 15:18 ` Greg Scott
@ 2001-12-06 19:03 ` Miron
2001-12-06 23:18 ` Greg Scott
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Miron @ 2001-12-06 19:03 UTC (permalink / raw)
To: lartc
This is a home setup, not a server setup. We have no servers on our
network. The reason we want port 80 on eth2 is because eth2 has more
download bandwidth. For other protocols we want eth1, because it has
more symmetric bandwidth.
Greg Scott wrote:
>This doesn't seem right:
>
>>My firewall configuration:
>> iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK
>>--set-mark 2
>> iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128
>> iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128
>>
>
>I'm not the expert, but I don't think you want to SNAT.
>
>You're marking inbound packets with destination port 80 and then putting
>rules in the POSTROUTING table to fudge in a different IP address for
>outbound packets. That doesn't seem right. It doesn't redirect inbound
>packets the way you want.
>
>I think you want to DNAT instead of SNAT and forget about marking packets.
>Set up some PREROUTING rules and DNAT all incoming port 80 stuff over to
>the interface you want. That should be all you need to do because the
>connection tracking should take care of getting the reply packets from your
>internal web server back to where they belong.
>
>Verify this with the experts before you do it, but I think I'm right on
>this one.
>
>- Greg Scott
>
>
>
>-----Original Message-----
>From: Miron [mailto:miron@hyper.to]
>Sent: Thursday, December 06, 2001 3:58 AM
>To: lartc@mailman.ds9a.nl
>Subject: [LARTC] Masq/route based on port
>
>
>I have following setup:
>
>- eth0 is an internal network
>- eth1 is an Internet connection (IP = 1.1.1.128, GW=1.1.1.1)
>- eth2 is another Internet connection (IP = 2.2.2.128, GW=2.2.2.1)
>
>I would like to masquerade port 80 through eth2, but all other traffic
>should be masq'ed through eth1.
>
>My routing configuration:
>
> (default route in main table is 1.1.1.1)
>
> ip rule add fwmark 2 pref 1002 table 666
>
> ip route flush table 666
> ip route add default via 2.2.2.1 dev eth3 proto static table 666
> ip route flush cache
>
>My firewall configuration:
> iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK
>--set-mark 2
> iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128
> iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128
>
>Unfortunately, this does not work. Outgoing packets are fine. Incoming
>packets on port 80 are not de-masqueraded and do not reach the internal
>hosts.
>
>Also, if I change the ip rule above to be based on the source address
>(instead of a mark), connections start working fine.
>
>Here is the output of 'ip rule ls', to prove that I do have fwmark compiled:
> 0: from all lookup local
> 1002: from all fwmark 2 lookup http
> 32766: from all lookup main
> 32767: from all lookup 253
>
>I am wondering if there is some kind of bug related to the interaction
>between fwmark and NAT. Any ideas?
>
>Thanks,
>Miron Cuperman
>
>
>
>_______________________________________________
>LARTC mailing list / LARTC@mailman.ds9a.nl
>http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO:
>http://ds9a.nl/2.4Routing/
>
>_______________________________________________
>LARTC mailing list / LARTC@mailman.ds9a.nl
>http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
>
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
^ permalink raw reply [flat|nested] 6+ messages in thread* RE: [LARTC] Masq/route based on port
2001-12-06 9:58 [LARTC] Masq/route based on port Miron
2001-12-06 15:18 ` Greg Scott
2001-12-06 19:03 ` Miron
@ 2001-12-06 23:18 ` Greg Scott
2001-12-07 6:34 ` Miron
2001-12-07 8:17 ` Miron
4 siblings, 0 replies; 6+ messages in thread
From: Greg Scott @ 2001-12-06 23:18 UTC (permalink / raw)
To: lartc
> This is a home setup, not a server setup. We have no servers on our
> network. The reason we want port 80 on eth2 is because eth2 has more
> download bandwidth. For other protocols we want eth1, because it has
> more symmetric bandwidth.
So anything that comes in from the Internet for port 80, no matter the
source, you want the reply to go back out on ETH2. And anything that
comes in from the Internet other than port 80, you want those replies to
go out ETH1. So the web server process is inside your Linux box?
Did I get that much right? Or do I have it backwards?
The Linux box is your internal LAN's default gateway and you want this
box to decide which Internet interface to use, based on the destination
port your internal client PCs choose?
Hadn't thought about it that way before.
- Greg
-----Original Message-----
From: Miron [mailto:miron@hyper.to]
Sent: Thursday, December 06, 2001 1:03 PM
To: Greg Scott
Cc: lartc@mailman.ds9a.nl
Subject: Re: [LARTC] Masq/route based on port
>I have following setup:
>
>- eth0 is an internal network
>- eth1 is an Internet connection (IP = 1.1.1.128, GW=1.1.1.1)
>- eth2 is another Internet connection (IP = 2.2.2.128, GW=2.2.2.1)
>
>I would like to masquerade port 80 through eth2, but all other traffic
>should be masq'ed through eth1.
>
>My routing configuration:
>
> (default route in main table is 1.1.1.1)
>
> ip rule add fwmark 2 pref 1002 table 666
>
> ip route flush table 666
> ip route add default via 2.2.2.1 dev eth3 proto static table 666
> ip route flush cache
>
>My firewall configuration:
> iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK
>--set-mark 2
> iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128
> iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128
>
>Unfortunately, this does not work. Outgoing packets are fine. Incoming
>packets on port 80 are not de-masqueraded and do not reach the internal
>hosts.
>
>Also, if I change the ip rule above to be based on the source address
>(instead of a mark), connections start working fine.
>
>Here is the output of 'ip rule ls', to prove that I do have fwmark
compiled:
> 0: from all lookup local
> 1002: from all fwmark 2 lookup http
> 32766: from all lookup main
> 32767: from all lookup 253
>
>I am wondering if there is some kind of bug related to the interaction
>between fwmark and NAT. Any ideas?
>
>Thanks,
>Miron Cuperman
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [LARTC] Masq/route based on port
2001-12-06 9:58 [LARTC] Masq/route based on port Miron
` (2 preceding siblings ...)
2001-12-06 23:18 ` Greg Scott
@ 2001-12-07 6:34 ` Miron
2001-12-07 8:17 ` Miron
4 siblings, 0 replies; 6+ messages in thread
From: Miron @ 2001-12-07 6:34 UTC (permalink / raw)
To: lartc
I think we are on the same wavelength, except maybe exactly backwards. ;)
I'm trying to provide fast web browsing for clients that are on the
internal network (talking to servers on the Internet). Anything from
the Internet with source port 80 should go through eth2. Anything else
should go through eth1.
I did some more investigation, and it seems that the Linux box is
dropping all incoming packets arriving into eth2. I can see packets
with tcpdump, but the applications and NAT don't actually get it. It
must be getting dropped in the kernel. It seems that this has something
to do with the default route pointing to eth1. I don't understand why
the fact that the default route is not pointing to an interface should
cause *incoming* packets to be dropped on that interface.
Sorry if my post was confusing.
Greg Scott wrote:
>>This is a home setup, not a server setup. We have no servers on our
>>network. The reason we want port 80 on eth2 is because eth2 has more
>>download bandwidth. For other protocols we want eth1, because it has
>>more symmetric bandwidth.
>>
>
>
>So anything that comes in from the Internet for port 80, no matter the
>source, you want the reply to go back out on ETH2. And anything that
>comes in from the Internet other than port 80, you want those replies to
>go out ETH1. So the web server process is inside your Linux box?
>
>Did I get that much right? Or do I have it backwards?
>
>The Linux box is your internal LAN's default gateway and you want this
>box to decide which Internet interface to use, based on the destination
>port your internal client PCs choose?
>
>Hadn't thought about it that way before.
>
>- Greg
>
>
>
>
>-----Original Message-----
>From: Miron [mailto:miron@hyper.to]
>Sent: Thursday, December 06, 2001 1:03 PM
>To: Greg Scott
>Cc: lartc@mailman.ds9a.nl
>Subject: Re: [LARTC] Masq/route based on port
>
>
>
>>I have following setup:
>>
>>- eth0 is an internal network
>>- eth1 is an Internet connection (IP = 1.1.1.128, GW=1.1.1.1)
>>- eth2 is another Internet connection (IP = 2.2.2.128, GW=2.2.2.1)
>>
>>I would like to masquerade port 80 through eth2, but all other traffic
>>should be masq'ed through eth1.
>>
>>My routing configuration:
>>
>> (default route in main table is 1.1.1.1)
>>
>> ip rule add fwmark 2 pref 1002 table 666
>>
>> ip route flush table 666
>> ip route add default via 2.2.2.1 dev eth3 proto static table 666
>> ip route flush cache
>>
>>My firewall configuration:
>> iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK
>>--set-mark 2
>> iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128
>> iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128
>>
>>Unfortunately, this does not work. Outgoing packets are fine. Incoming
>>packets on port 80 are not de-masqueraded and do not reach the internal
>>hosts.
>>
>>Also, if I change the ip rule above to be based on the source address
>>(instead of a mark), connections start working fine.
>>
>>Here is the output of 'ip rule ls', to prove that I do have fwmark
>>
>compiled:
>
>> 0: from all lookup local
>> 1002: from all fwmark 2 lookup http
>> 32766: from all lookup main
>> 32767: from all lookup 253
>>
>>I am wondering if there is some kind of bug related to the interaction
>>between fwmark and NAT. Any ideas?
>>
>>Thanks,
>>Miron Cuperman
>>
>
>_______________________________________________
>LARTC mailing list / LARTC@mailman.ds9a.nl
>http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
>
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [LARTC] Masq/route based on port
2001-12-06 9:58 [LARTC] Masq/route based on port Miron
` (3 preceding siblings ...)
2001-12-07 6:34 ` Miron
@ 2001-12-07 8:17 ` Miron
4 siblings, 0 replies; 6+ messages in thread
From: Miron @ 2001-12-07 8:17 UTC (permalink / raw)
To: lartc
Okay, I figured it out. The secret is to set rp_filter to zero on the
non-default interface. I guess the kernel thought that the packets were
source-routed because of the fact that the interface received packets it
could not route.
That was frustrating, yet fun... ;)
Miron wrote:
> I think we are on the same wavelength, except maybe exactly backwards. ;)
>
> I'm trying to provide fast web browsing for clients that are on the
> internal network (talking to servers on the Internet). Anything from
> the Internet with source port 80 should go through eth2. Anything
> else should go through eth1.
>
> I did some more investigation, and it seems that the Linux box is
> dropping all incoming packets arriving into eth2. I can see packets
> with tcpdump, but the applications and NAT don't actually get it. It
> must be getting dropped in the kernel. It seems that this has
> something to do with the default route pointing to eth1. I don't
> understand why the fact that the default route is not pointing to an
> interface should cause *incoming* packets to be dropped on that
> interface.
>
> Sorry if my post was confusing.
>
> Greg Scott wrote:
>
>>> This is a home setup, not a server setup. We have no servers on our
>>> network. The reason we want port 80 on eth2 is because eth2 has
>>> more download bandwidth. For other protocols we want eth1, because
>>> it has more symmetric bandwidth.
>>>
>>
>>
>> So anything that comes in from the Internet for port 80, no matter
>> the source, you want the reply to go back out on ETH2. And anything
>> that
>> comes in from the Internet other than port 80, you want those replies to
>> go out ETH1. So the web server process is inside your Linux box?
>> Did I get that much right? Or do I have it backwards?
>> The Linux box is your internal LAN's default gateway and you want
>> this box to decide which Internet interface to use, based on the
>> destination port your internal client PCs choose?
>> Hadn't thought about it that way before.
>>
>> - Greg
>>
>>
>>
>>
>> -----Original Message-----
>> From: Miron [mailto:miron@hyper.to]
>> Sent: Thursday, December 06, 2001 1:03 PM
>> To: Greg Scott
>> Cc: lartc@mailman.ds9a.nl
>> Subject: Re: [LARTC] Masq/route based on port
>>
>>
>>
>>> I have following setup:
>>>
>>> - eth0 is an internal network
>>> - eth1 is an Internet connection (IP = 1.1.1.128, GW=1.1.1.1)
>>> - eth2 is another Internet connection (IP = 2.2.2.128, GW=2.2.2.1)
>>>
>>> I would like to masquerade port 80 through eth2, but all other
>>> traffic should be masq'ed through eth1.
>>>
>>> My routing configuration:
>>>
>>> (default route in main table is 1.1.1.1)
>>>
>>> ip rule add fwmark 2 pref 1002 table 666
>>>
>>> ip route flush table 666
>>> ip route add default via 2.2.2.1 dev eth3 proto static table 666
>>> ip route flush cache
>>>
>>> My firewall configuration:
>>> iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK
>>> --set-mark 2
>>> iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128
>>> iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128
>>>
>>> Unfortunately, this does not work. Outgoing packets are fine.
>>> Incoming packets on port 80 are not de-masqueraded and do not reach
>>> the internal hosts.
>>>
>>> Also, if I change the ip rule above to be based on the source
>>> address (instead of a mark), connections start working fine.
>>>
>>> Here is the output of 'ip rule ls', to prove that I do have fwmark
>>>
>> compiled:
>>
>>> 0: from all lookup local
>>> 1002: from all fwmark 2 lookup http
>>> 32766: from all lookup main
>>> 32767: from all lookup 253
>>>
>>> I am wondering if there is some kind of bug related to the
>>> interaction between fwmark and NAT. Any ideas?
>>>
>>> Thanks,
>>> Miron Cuperman
>>>
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://ds9a.nl/2.4Routing/
^ permalink raw reply [flat|nested] 6+ messages in thread