[LARTC] Wierd Policy Routing Behaviour

[LARTC] Wierd Policy Routing Behaviour

[Sellaro]

Hi there 

I'm trying to set up policy routing in a simple scenario without further
success. My linux router is connected to two different cable ISPs. One
of these links (hereafter named A) permits traffic to SMTP servers while
the other (link B) doesn't. 

What I am trying to set up is: all traffic should flow through link B
and only SMTP traffic through link A. 

What I am doing: 

Marking all packets from my intranet with source port within the range
1024 to 65535 and with destination port 25 with mark 0x3. I am also
marking packets from our internal SMTP server with source port 25 to any
other port with the same mark. Marks are being made in the OUTPUT chain
(using iptables) as follows: 

iptables -t mangle -A OUTPUT -p tcp --sport 1024:65535 --dport 25 -j
MARK --set-mark 3 

iptables  -t mangle -A OUTPUT -p tcp -s <MY SMTP SERVER> --sport 25 -j
MARK --set-mark 3 

Then, I have created a new routing table named SMTP-ROUTE with default
gateway being the default gateway for link A as follows: 

ip ro add default via <A's default GW> table SMTP-ROUTE 

To complete the setup, I've added a rule stating that all packets marked
with 0x3 should use SMTP-ROUTE with this command: 

ip ru add fwmark 3 table SMTP-ROUTE 

Yes, I've flushed the routing cache with: 

ip ro flush cache 

Default route in main routing table is B's gateway. 

I don't know why, but SMTP traffic keeps going through B's gateway,
instead of A's, as expected. 

I've sniffed the network and, in fact, the packets are trying to go out
through B's gateway.

Can anyone please point what I am doing wrong?

Thank you in advance 
-- 
Sellaro

Agente Livre - Linux Community (www.agentelivre.org)

PGP Key ID: 3ADF8645
PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553  2242 A1D0 17C0 3ADF 8645

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Wierd Policy Routing Behaviour

[Martin A. Brown]

Sellaro,

I'm not sure about this, (others on the list may confirm), but I think you 
want to use --set-tos instead of --set-mark.

My understanding is that the mark only exists while the packet is on the 
local machine, but when you set the ToS flags, then you are changing the 
packet.

Best of luck,

-Martin

 : Hi there 
 : 
 : I'm trying to set up policy routing in a simple scenario without further
 : success. My linux router is connected to two different cable ISPs. One
 : of these links (hereafter named A) permits traffic to SMTP servers while
 : the other (link B) doesn't. 
 : 
 : What I am trying to set up is: all traffic should flow through link B
 : and only SMTP traffic through link A. 
 : 
 : What I am doing: 
 : 
 : Marking all packets from my intranet with source port within the range
 : 1024 to 65535 and with destination port 25 with mark 0x3. I am also
 : marking packets from our internal SMTP server with source port 25 to any
 : other port with the same mark. Marks are being made in the OUTPUT chain
 : (using iptables) as follows: 
 : 
 : iptables -t mangle -A OUTPUT -p tcp --sport 1024:65535 --dport 25 -j
 : MARK --set-mark 3 
 : 
 : iptables  -t mangle -A OUTPUT -p tcp -s <MY SMTP SERVER> --sport 25 -j
 : MARK --set-mark 3 
 : 
 : Then, I have created a new routing table named SMTP-ROUTE with default
 : gateway being the default gateway for link A as follows: 
 : 
 : ip ro add default via <A's default GW> table SMTP-ROUTE 
 : 
 : To complete the setup, I've added a rule stating that all packets marked
 : with 0x3 should use SMTP-ROUTE with this command: 
 : 
 : ip ru add fwmark 3 table SMTP-ROUTE 
 : 
 : Yes, I've flushed the routing cache with: 
 : 
 : ip ro flush cache 
 : 
 : Default route in main routing table is B's gateway. 
 : 
 : I don't know why, but SMTP traffic keeps going through B's gateway,
 : instead of A's, as expected. 
 : 
 : I've sniffed the network and, in fact, the packets are trying to go out
 : through B's gateway.
 : 
 : Can anyone please point what I am doing wrong?
 : 
 : Thank you in advance 
 : 

-- 
Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Wierd Policy Routing Behaviour

[Sellaro]

On Thu, 2002-05-30 at 12:22, Martin A. Brown wrote:

> I'm not sure about this, (others on the list may confirm), but I think you 
> want to use --set-tos instead of --set-mark.

I have not tried to use ToS as key for classifying the packets. I'll try
to, eventhough I think there's nothing wrong with --set-mark.
 
> My understanding is that the mark only exists while the packet is on the 
> local machine, but when you set the ToS flags, then you are changing the 
> packet.

But the packets are being marked on the router itself. That's why I
think there is nothing wrong (explicitlly, at least) with the solution
I'm trying to use. Anyway, I'll wait for comments from others on the
list.

Thank you for your help.
-- 
Sellaro

Agente Livre - Linux Community (www.agentelivre.org)

PGP Key ID: 3ADF8645
PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553  2242 A1D0 17C0 3ADF 8645

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Wierd Policy Routing Behaviour

[Alexey Talikov]

Without iproute
iptables -t nat -A POSTROUTING -s $SMTP -p tcp --sport 1024: --dport 25 -j SNAT --to-source $IP_B


30.05.2002 19:05:13, Sellaro <sellaro@email.it> wrote:

>Hi there 
>
>I'm trying to set up policy routing in a simple scenario without further
>success. My linux router is connected to two different cable ISPs. One
>of these links (hereafter named A) permits traffic to SMTP servers while
>the other (link B) doesn't. 
>
>What I am trying to set up is: all traffic should flow through link B
>and only SMTP traffic through link A. 
>
>What I am doing: 
>
>Marking all packets from my intranet with source port within the range
>1024 to 65535 and with destination port 25 with mark 0x3. I am also
>marking packets from our internal SMTP server with source port 25 to any
>other port with the same mark. Marks are being made in the OUTPUT chain
>(using iptables) as follows: 
>
>iptables -t mangle -A OUTPUT -p tcp --sport 1024:65535 --dport 25 -j
>MARK --set-mark 3 
>
>iptables  -t mangle -A OUTPUT -p tcp -s <MY SMTP SERVER> --sport 25 -j
>MARK --set-mark 3 
>
>Then, I have created a new routing table named SMTP-ROUTE with default
>gateway being the default gateway for link A as follows: 
>
>ip ro add default via <A's default GW> table SMTP-ROUTE 
>
>To complete the setup, I've added a rule stating that all packets marked
>with 0x3 should use SMTP-ROUTE with this command: 
>
>ip ru add fwmark 3 table SMTP-ROUTE 
>
>Yes, I've flushed the routing cache with: 
>
>ip ro flush cache 
>
>Default route in main routing table is B's gateway. 
>
>I don't know why, but SMTP traffic keeps going through B's gateway,
>instead of A's, as expected. 
>
>I've sniffed the network and, in fact, the packets are trying to go out
>through B's gateway.
>
>Can anyone please point what I am doing wrong?
>
>Thank you in advance 
>-- 
>Sellaro
>
>Agente Livre - Linux Community (www.agentelivre.org)
>
>PGP Key ID: 3ADF8645
>PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553  2242 A1D0 17C0 3ADF 8645
>
>_______________________________________________
>LARTC mailing list / LARTC@mailman.ds9a.nl
>http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>

-----------------------------------
mailto:alexey_talikov@texlab.com.uz
BR
Alexey Talikov
FORTEK
-----------------------------------


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Wierd Policy Routing Behaviour

[Sellaro]

On Fri, 2002-05-31 at 00:57, Alexey Talikov wrote:

> Without iproute
> iptables -t nat -A POSTROUTING -s $SMTP -p tcp --sport 1024: --dport 25 -j SNAT --to-source $IP_B

Source NAT is not what I want to do. I really want to send SMTP packets
through a different route.

-- 
Sellaro

Agente Livre - Linux Community (www.agentelivre.org)

PGP Key ID: 3ADF8645
PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553  2242 A1D0 17C0 3ADF 8645

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Wierd Policy Routing Behaviour

[Alexey Talikov]

IP: advanced router (CONFIG_IP_ADVANCED_ROUTER) [Y/n/?]
 IP: policy routing (CONFIG_IP_MULTIPLE_TABLES) [Y/n/?]
  IP: use netfilter MARK value as routing key (CONFIG_IP_ROUTE_FWMARK) [Y/n/?]

iptables -t mangle -A PREROUTING -p tcp --dport 25 -j MARK --set-mark 1
echo 201 mail.out >> /etc/iproute2/rt_tables
ip rule add fwmark 1 table mail.out
ip route add default via $IP dev $DEV table mail.out


for more detail see Advanced-Routing HOWTO Chapter 11

31.05.2002 16:24:35, Sellaro <sellaro@email.it> wrote:

>On Fri, 2002-05-31 at 00:57, Alexey Talikov wrote:
>
>> Without iproute
>> iptables -t nat -A POSTROUTING -s $SMTP -p tcp --sport 1024: --dport 25 -j SNAT --to-source 
$IP_B
>
>Source NAT is not what I want to do. I really want to send SMTP packets
>through a different route.
>
>-- 
>Sellaro
>
>Agente Livre - Linux Community (www.agentelivre.org)
>
>PGP Key ID: 3ADF8645
>PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553  2242 A1D0 17C0 3ADF 8645
>

-----------------------------------
mailto:alexey_talikov@texlab.com.uz
BR
Alexey Talikov
FORTEK
-----------------------------------


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Wierd Policy Routing Behaviour

[Sellaro]

> IP: advanced router (CONFIG_IP_ADVANCED_ROUTER) [Y/n/?]
>  IP: policy routing (CONFIG_IP_MULTIPLE_TABLES) [Y/n/?]
>   IP: use netfilter MARK value as routing key (CONFIG_IP_ROUTE_FWMARK)
[Y/n/?]

internet:/usr/src/linux# cat .config | grep ROUTER
CONFIG_IP_ADVANCED_ROUTER=y

internet:/usr/src/linux# cat .config | grep MULTIPLE
CONFIG_IP_MULTIPLE_TABLES=y

internet:/usr/src/linux# cat .config | grep FWMARK  
CONFIG_IP_ROUTE_FWMARK=y

> iptables -t mangle -A PREROUTING -p tcp --dport 25 -j MARK --set-mark 1

OK, I'm marking with 3, not 1, but I think this will not cause any problem:

iptables -t mangle -A PREROUTING -p tcp --dport 25 -j MARK --set-mark 3

> echo 201 mail.out >> /etc/iproute2/rt_tables

OK. But table is named only mail.

> ip rule add fwmark 1 table mail.out

ip ru add fwmark 3 table mail

ip ru ls
0:
from all lookup local 
32765:
from all fwmark        3 lookup mail 
32766:
from all lookup main 
32767:
from all lookup default

> ip route add default via $IP dev $DEV table mail.out

Done.

You've forgoten to mention flushing route cahce. I did it with:
ip ro flush cache

And nothing happens as expected. That's why I am writting to the list.

Thank you for your help.

--
Sellaro

Network Management for the Masses



--
Prendi GRATIS l'email universale che... risparmia: http://www.email.it/f

Sponsor:
Tutti i bulbi in offerta... da Peraga risparmi il 30%!
Clicca qui: http://adv2.email.it/cgi-bin/foclick.cgi?mid=448&d=31-5
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Wierd Policy Routing Behaviour

[Alexey Talikov]

Your don't add $DEV 
ip route add default via $IP dev $DEV table mail.out

31.05.2002 17:03:10, "Sellaro"<sellaro@email.it> wrote:

>> IP: advanced router (CONFIG_IP_ADVANCED_ROUTER) [Y/n/?]
>>  IP: policy routing (CONFIG_IP_MULTIPLE_TABLES) [Y/n/?]
>>   IP: use netfilter MARK value as routing key (CONFIG_IP_ROUTE_FWMARK)
>[Y/n/?]
>
>internet:/usr/src/linux# cat .config | grep ROUTER
>CONFIG_IP_ADVANCED_ROUTER=y
>
>internet:/usr/src/linux# cat .config | grep MULTIPLE
>CONFIG_IP_MULTIPLE_TABLES=y
>
>internet:/usr/src/linux# cat .config | grep FWMARK  
>CONFIG_IP_ROUTE_FWMARK=y
>
>> iptables -t mangle -A PREROUTING -p tcp --dport 25 -j MARK --set-mark 1
>
>OK, I'm marking with 3, not 1, but I think this will not cause any problem:
>
>iptables -t mangle -A PREROUTING -p tcp --dport 25 -j MARK --set-mark 3
>
>> echo 201 mail.out >> /etc/iproute2/rt_tables
>
>OK. But table is named only mail.
>
>> ip rule add fwmark 1 table mail.out
>
>ip ru add fwmark 3 table mail
>
>ip ru ls
>0:
>from all lookup local 
>32765:
>from all fwmark        3 lookup mail 
>32766:
>from all lookup main 
>32767:
>from all lookup default
>
>> ip route add default via $IP dev $DEV table mail.out
>
>Done.
>
>You've forgoten to mention flushing route cahce. I did it with:
>ip ro flush cache
>
>And nothing happens as expected. That's why I am writting to the list.
>
>Thank you for your help.
>
>--
>Sellaro
>
>Network Management for the Masses
>
>
>
>--
>Prendi GRATIS l'email universale che... risparmia: http://www.email.it/f
>
>Sponsor:
>Tutti i bulbi in offerta... da Peraga risparmi il 30%!
>Clicca qui: http://adv2.email.it/cgi-bin/foclick.cgi?midD8&d1-5
>

-----------------------------------
mailto:alexey_talikov@texlab.com.uz
BR
Alexey Talikov
FORTEK
-----------------------------------


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Wierd Policy Routing Behaviour

[Sellaro]

On Fri, 2002-05-31 at 09:15, Alexey Talikov wrote:

> Your don't add $DEV 
> ip route add default via $IP dev $DEV table mail.out

Done. But this is not the problem. I've found a post (reading Advanced
Routing Howto again) from Rusty speaking about MASQ and fwmark problems.
The post is available at this address:
http://lists.samba.org/pipermail/netfilter/2000-November/006089.html

OK. Now we have a little change on our scenario. All hosts from my
INTRANET are able to connect to external SMTP servers properly. However
routing itself is not. Sniffing the netowrk shows that from the router
the packets are going out through the wrong interface.

I've also added the same same marking rule to OUTPUT table, but it
didn't work. Any clues?

Thank you in advance.
-- 
Sellaro

Agente Livre - Linux Community (www.agentelivre.org)

PGP Key ID: 3ADF8645
PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553  2242 A1D0 17C0 3ADF 8645

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Wierd Policy Routing Behaviour

[Alexey Talikov]

Version of kernel iptables and iproute , any patches ???

31.05.2002 18:48:39, Sellaro <sellaro@email.it> wrote:

>On Fri, 2002-05-31 at 09:15, Alexey Talikov wrote:
>
>> Your don't add $DEV 
>> ip route add default via $IP dev $DEV table mail.out
>
>Done. But this is not the problem. I've found a post (reading Advanced
>Routing Howto again) from Rusty speaking about MASQ and fwmark problems.
>The post is available at this address:
>http://lists.samba.org/pipermail/netfilter/2000-November/006089.html
>
>OK. Now we have a little change on our scenario. All hosts from my
>INTRANET are able to connect to external SMTP servers properly. However
>routing itself is not. Sniffing the netowrk shows that from the router
>the packets are going out through the wrong interface.
>
>I've also added the same same marking rule to OUTPUT table, but it
>didn't work. Any clues?
>
>Thank you in advance.
>-- 
>Sellaro
>
>Agente Livre - Linux Community (www.agentelivre.org)
>
>PGP Key ID: 3ADF8645
>PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553  2242 A1D0 17C0 3ADF 8645
>

-----------------------------------
mailto:alexey_talikov@texlab.com.uz
BR
Alexey Talikov
FORTEK
-----------------------------------


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Wierd Policy Routing Behaviour

[Sellaro]

On Fri, 2002-05-31 at 11:33, Alexey Talikov wrote:

> Version of kernel iptables and iproute , any patches ???

Sorry for not mentioning before.

kernel: 2.4.18
iproute: 20010824-7
Patches: only HTB 3 into the kernel, of course.

-- 
Sellaro

Agente Livre - Linux Community (www.agentelivre.org)

PGP Key ID: 3ADF8645
PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553  2242 A1D0 17C0 3ADF 8645

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Wierd Policy Routing Behaviour

[Alexey Talikov]

add input interface in rule because this table general for all interfaces also add ip if possible
iptables -t mangle -A PREROUTING -i $DEV -p tcp --dport 25 -j MARK --set-mark 1
if don't work
try
iptables -t mangle -A FORWARD -i $DEV -p tcp --dport 25 -j MARK --set-mark 1 (don't work for old
iptables and kernel, missing FORWARD and INPUT chain for mangle table)

if you compile netfilter as modules -- all necessary loaded ?

if don't work try to test mark with tc filter 
try to use last iproute2-2.4.7-020116 (possible that the ip route can't see your marks
and last htb-3.6 (Thanks Devik !)

Send me results !!

(Sorry for my pure English :((   )

31.05.2002 19:45:33, Sellaro <sellaro@email.it> wrote:

>On Fri, 2002-05-31 at 11:33, Alexey Talikov wrote:
>
>> Version of kernel iptables and iproute , any patches ???
>
>Sorry for not mentioning before.
>
>kernel: 2.4.18
>iproute: 20010824-7
>Patches: only HTB 3 into the kernel, of course.
>
>-- 
>Sellaro
>
>Agente Livre - Linux Community (www.agentelivre.org)
>
>PGP Key ID: 3ADF8645
>PGP Key Fingerprint: 6AB0 D60B 69B5 B3F9 4553  2242 A1D0 17C0 3ADF 8645
>

-----------------------------------
mailto:alexey_talikov@texlab.com.uz
BR
Alexey Talikov
FORTEK
-----------------------------------


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/