[LARTC] Slipt 2 ISP strange routing problem

[LARTC] Slipt 2 ISP strange routing problem

[guillaume]

Dear all Lartc,

I try to split my Internet access to my 2 ISP with 1 linux (GNU/Debian 
sarge) 3 NIC router,
I want all my users conneted with ISP1 and just some IP connected with ISP2

Here is my configuration:

Internal network: 10.117.71.0/24
                              Interface eth0

ISP1: IP for my linux box: 1.2.3.4/29
        Interface: eth1
        Gateway: 1.2.3.5

ISP2: IP for my linux box: 5.6.7.8/29
        interface: eth2
        gateway: 5.6.7.9    (in fact it's my adsl modem)

You can see below my configuration for iptables and iproute2

I get a strange problem:

All my user can connect to internet with ISP1, there is no problem.

But when I try with 10.117.71.1 (the one routed to ISP2),
I can connect to the gateway (5.6.7.9), that mean that I am correctly 
routed.
But I cannot connect (or ping) internet. I think packets are send but 
when the
web site (google.com for exemple) reply, my packets are not routed back.

I also try to connect to 5.6.7.9 (ISP2 gateway) from other internal IP 
(routed on ISP1),
and it's not possible, but for the moment it's not a problem for me.

I test my connection from my modem (gateway ISP2), no problem.

Is there some kind guru of iptables and iproute2 to help me on this ?

Thank you in advance,
Guillaume

Here is my script for routing and iptables:
#################################
# ISP2
/sbin/ip route flush table 5
/sbin/ip route add table 5 default via 5.6.7.9 dev eth2
/sbin/ip rule add fwmark 5 table 5

echo "firewall constants setup"

# FLUSH the tables
iptables -t nat -F POSTROUTING
iptables -t nat -F PREROUTING
iptables -t mangle -F PREROUTING

## Mangeling Rules ##

# special rules for some IPs to go on second ISP
iptables -t mangle -A PREROUTING -s 10.117.71.1 -j MARK --set-mark 5


# SNAT RULES

iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to 1.2.3.4
iptables -t nat -A POSTROUTING -o eth2 -s 10.117.71.1 -j SNAT --to 5.6.7.8


###############################################
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

# The loopback interface
auto lo
iface lo inet loopback

# (network, broadcast and gateway are optional)
auto eth1 eth0 eth2

# local network (LAN)
iface eth0 inet static
       address 10.117.71.1
       netmask 255.255.255.0

# external network (ISP1)
iface eth1 inet static
       address 1.2.3.4
       netmask 255.255.255.248
       gateway 1.2.3.5

iface eth2 inet static
       address 5.6.7.8
       netmask 255.255.255.248

#######################################

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Slipt 2 ISP strange routing problem

[Robert Kurjata]

Cytowanie guillaume <guillaume.riviere@vslitc.com>:

> Dear all Lartc,
> I try to split my Internet access to my 2 ISP with 1 linux (GNU/Debian 
> sarge) 3 NIC router,
> I want all my users conneted with ISP1 and just some IP connected with ISP2
> Here is my configuration:

[cut out a part]

I would suggest slight change in fw rules below
 
 # special rules for some IPs to go on second ISP
 iptables -t mangle -A PREROUTING -s 10.117.71.1 -j MARK --set-mark 34
 
 # SNAT RULES
 
 iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to 1.2.3.4
 iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to 5.6.7.8

and trying the script below. It was taken from my Multipath/Policy routing case
by wipeing out a multipath part, so its is lsightly too much, but I suppose (I
didn't check) it should work. 
It is supposed to seamlesly integrate with multipath routing that's why so big,
the idea behind is:
1. remove default routing from main table,
2. take care of routing from correct interface (correctness of source IP/ source
interface pair),
3. policy routing of selected clients (table 210)
4. default routing of the others (table 211)
Of course you can leave it for the main table, but for the sake of example
i left it this way.

Hope it can help.

--- cut here ----
#!/bin/bash

# CONFIGURATION
IP=/sbin/ip
PING=/bin/ping

#--------------- LINK PART -----------------
# EXTIFn - interface name
# EXTIPn - outgoing IP
# EXTMn  - netmask length (bits)
# EXTGWn - outgoing gateway
#-------------------------------------------

# LINK 1 - 
EXTIF1EXTIP1EXTM1EXTGW1
# LINK 2 - 
EXTIF2EXTIP2EXTM2EXTGW2
#ROUTING PART
# removing old rules and routes

echo "removing old rules"
${IP} rule del prio 50 table main
${IP} rule del prio 201 from ${EXTIP1}/${EXTM1} table 201
${IP} rule del prio 202 from ${EXTIP2}/${EXTM2} table 202
${IP} rule del prio 210 fwmark 22 table 210
${IP} rule del prio 221 table 221
echo "flushing tables"
${IP} route flush table 201
${IP} route flush table 202
${IP} route flush table 210
${IP} route flush table 221
echo "removing tables"
${IP} route del table 201
${IP} route del table 202
${IP} route del table 210
${IP} route del table 221

case $1 in
stop) exit
      ;;
esac
      
# setting new rules
echo "Setting new routing rules"

# main table w/o default gateway here
${IP} rule add prio 50 table main
${IP} route del default table main

# identified routes here
${IP} rule add prio 201 from ${EXTIP1}/${EXTM1} table 201
${IP} rule add prio 202 from ${EXTIP2}/${EXTM2} table 202

${IP} route add default via ${EXTGW1} dev ${EXTIF1} src ${EXTIP1} proto static
table 201
${IP} route append prohibit default table 201 metric 1 proto static

${IP} route add default via ${EXTGW2} dev ${EXTIF2} src ${EXTIP2} proto static
table 202
${IP} route append prohibit default table 202 metric 1 proto static

# selected clients / select by fwmark4

${IP} rule add prio 210 fwmark 22 table 210

# others
${IP} rule add prio 221 table 221

${IP} route add default table 210 proto static \
	    via ${EXTGW2} dev ${EXTIF2} src ${EXTIP2}

${IP} route add default table 221 proto static \
	    via ${EXTGW1} dev ${EXTIF1} src ${EXTIP1}

${IP} route flush cache




-- 
Robert Kurjata     mailto:rkurjata@ire.pw.edu.pl


-------------------------------------------------
This mail sent through IMP: http://horde.org/imp/
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Slipt 2 ISP strange routing problem

[guillaume]

Robert Kurjata wrote:

>Cytowanie guillaume <guillaume.riviere@vslitc.com>:
>
>  
>
>>Dear all Lartc,
>>I try to split my Internet access to my 2 ISP with 1 linux (GNU/Debian 
>>sarge) 3 NIC router,
>>I want all my users conneted with ISP1 and just some IP connected with ISP2
>>Here is my configuration:
>>    
>>
>
>[cut out a part]
>
>I would suggest slight change in fw rules below
> 
> # special rules for some IPs to go on second ISP
> iptables -t mangle -A PREROUTING -s 10.117.71.1 -j MARK --set-mark 34
> 
> # SNAT RULES
> 
> iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to 1.2.3.4
> iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to 5.6.7.8
>
>and trying the script below. It was taken from my Multipath/Policy routing case
>by wipeing out a multipath part, so its is lsightly too much, but I suppose (I
>didn't check) it should work. 
>It is supposed to seamlesly integrate with multipath routing that's why so big,
>the idea behind is:
>1. remove default routing from main table,
>2. take care of routing from correct interface (correctness of source IP/ source
>interface pair),
>3. policy routing of selected clients (table 210)
>4. default routing of the others (table 211)
>Of course you can leave it for the main table, but for the sake of example
>i left it this way.
>
>Hope it can help.
>  
>
[cut out a part]


Dear Robert,

I try this script (adapted for my network) and I get the same problem:

All my IP routed on my first ISP, no problem
With my 10.117.71.1 routed on my second ISP,
I can connect to my ISP network (I can connect to the gateway website on 
5.6.7.9) but
I cannot ping any external IP addresses.

Do I have to apply some pach to my 2.6.8 kernel ?
I really doesn't know how to invastigate more on  this problem.

In any cases, thank you for this script,
Guillaume
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Slipt 2 ISP strange routing problem

[guillaume]

guillaume wrote:

> Robert Kurjata wrote:
>
>> Cytowanie guillaume <guillaume.riviere@vslitc.com>:
>>
>>  
>>
>>> Dear all Lartc,
>>> I try to split my Internet access to my 2 ISP with 1 linux 
>>> (GNU/Debian sarge) 3 NIC router,
>>> I want all my users conneted with ISP1 and just some IP connected 
>>> with ISP2
>>> Here is my configuration:
>>>   
>>
>>
>> [cut out a part]
>>
>> I would suggest slight change in fw rules below
>>
>> # special rules for some IPs to go on second ISP
>> iptables -t mangle -A PREROUTING -s 10.117.71.1 -j MARK --set-mark 34
>>
>> # SNAT RULES
>>
>> iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to 1.2.3.4
>> iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to 5.6.7.8
>>
>> and trying the script below. It was taken from my Multipath/Policy 
>> routing case
>> by wipeing out a multipath part, so its is lsightly too much, but I 
>> suppose (I
>> didn't check) it should work. It is supposed to seamlesly integrate 
>> with multipath routing that's why so big,
>> the idea behind is:
>> 1. remove default routing from main table,
>> 2. take care of routing from correct interface (correctness of source 
>> IP/ source
>> interface pair),
>> 3. policy routing of selected clients (table 210)
>> 4. default routing of the others (table 211)
>> Of course you can leave it for the main table, but for the sake of 
>> example
>> i left it this way.
>>
>> Hope it can help.
>>  
>>
> [cut out a part]
>
>
> Dear Robert,
>
> I try this script (adapted for my network) and I get the same problem:
>
> All my IP routed on my first ISP, no problem
> With my 10.117.71.1 routed on my second ISP,
> I can connect to my ISP network (I can connect to the gateway website 
> on 5.6.7.9) but
> I cannot ping any external IP addresses.

I  also test a DNAT rule to access to my internal network with my second 
ISP external
IP ... And it works fine, no problem ...

iptables -t nat -I PREROUTING  -p tcp -d 1.2.3.4   --dport 80  -j DNAT 
--to 10.117.71.2:80 # my web server

I don't know how to make this work for Inside -> outside connection ...

Guillaume
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Slipt 2 ISP strange routing problem

[guillaume]

Robert Kurjata wrote:

>Hello guillaume,
>
>Saturday, September 4, 2004, 12:44:04 PM, you wrote:
>
>g> guillaume wrote:
>
>  
>
>>>Robert Kurjata wrote:
>>>
>>>      
>>>
>>>>Cytowanie guillaume <guillaume.riviere@vslitc.com>:
>>>>
>>>> 
>>>>
>>>>        
>>>>
>>>>>Dear all Lartc,
>>>>>I try to split my Internet access to my 2 ISP with 1 linux 
>>>>>(GNU/Debian sarge) 3 NIC router,
>>>>>I want all my users conneted with ISP1 and just some IP connected
>>>>>with ISP2
>>>>>Here is my configuration:
>>>>>  
>>>>>          
>>>>>
>>>>[cut out a part]
>>>>
>>>>I would suggest slight change in fw rules below
>>>>
>>>># special rules for some IPs to go on second ISP
>>>>iptables -t mangle -A PREROUTING -s 10.117.71.1 -j MARK --set-mark 34
>>>>
>>>># SNAT RULES
>>>>
>>>>iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to 1.2.3.4
>>>>iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to 5.6.7.8
>>>>
>>>>and trying the script below. It was taken from my Multipath/Policy
>>>>routing case
>>>>by wipeing out a multipath part, so its is lsightly too much, but I
>>>>suppose (I
>>>>didn't check) it should work. It is supposed to seamlesly integrate
>>>>with multipath routing that's why so big,
>>>>the idea behind is:
>>>>1. remove default routing from main table,
>>>>2. take care of routing from correct interface (correctness of source
>>>>IP/ source
>>>>interface pair),
>>>>3. policy routing of selected clients (table 210)
>>>>4. default routing of the others (table 211)
>>>>Of course you can leave it for the main table, but for the sake of
>>>>example
>>>>i left it this way.
>>>>
>>>>Hope it can help.
>>>> 
>>>>
>>>>        
>>>>
>>>[cut out a part]
>>>
>>>
>>>Dear Robert,
>>>
>>>I try this script (adapted for my network) and I get the same problem:
>>>
>>>All my IP routed on my first ISP, no problem
>>>With my 10.117.71.1 routed on my second ISP,
>>>I can connect to my ISP network (I can connect to the gateway website
>>>on 5.6.7.9) but
>>>I cannot ping any external IP addresses.
>>>      
>>>
>
>g> I  also test a DNAT rule to access to my internal network with my second
>g> ISP external
>g> IP ... And it works fine, no problem ...
>
>g> iptables -t nat -I PREROUTING  -p tcp -d 1.2.3.4   --dport 80  -j DNAT
>g> --to 10.117.71.2:80 # my web server
>
>g> I don't know how to make this work for Inside -> outside connection ...
>
>g> Guillaume
>g> _______________________________________________
>g> LARTC mailing list / LARTC@mailman.ds9a.nl
>g> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>
>Of course I have the patches from Julian Anastasov applied
>http://www.ssi.bg/~ja/#routes , maybe that's the point
>
>  
>
In a first time no,
but now, yes I have apply the routes-2.6.8-10.diff find on 
http://www.ssi.bg/~ja/#routes-2.6 
<http://www.ssi.bg/%7Eja/routes-2.6.8-10.diff>
by patch "patch -p1 < route-2.6.8-10.diff" there is no differences ...
I could not access to Internet with our without the pach,

How can I be sure that this pach is on my current kernel ?

I will try to re-apply the pach and recompile my kernel.

Thank you,
Guillaume
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Slipt 2 ISP strange routing problem

[guillaume]

guillaume wrote:

> Robert Kurjata wrote:
>
>> Hello guillaume,
>>
>> Saturday, September 4, 2004, 12:44:04 PM, you wrote:
>>
>> g> guillaume wrote:
>>
>>  
>>
>>>> Robert Kurjata wrote:
>>>>
>>>>     
>>>>
>>>>> Cytowanie guillaume <guillaume.riviere@vslitc.com>:
>>>>>
>>>>>
>>>>>
>>>>>       
>>>>>
>>>>>> Dear all Lartc,
>>>>>> I try to split my Internet access to my 2 ISP with 1 linux 
>>>>>> (GNU/Debian sarge) 3 NIC router,
>>>>>> I want all my users conneted with ISP1 and just some IP connected
>>>>>> with ISP2
>>>>>> Here is my configuration:
>>>>>>  
>>>>>>         
>>>>>
>>>>> [cut out a part]
>>>>>
>>>>> I would suggest slight change in fw rules below
>>>>>
>>>>> # special rules for some IPs to go on second ISP
>>>>> iptables -t mangle -A PREROUTING -s 10.117.71.1 -j MARK --set-mark 34
>>>>>
>>>>> # SNAT RULES
>>>>>
>>>>> iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to 1.2.3.4
>>>>> iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to 5.6.7.8
>>>>>
>>>>> and trying the script below. It was taken from my Multipath/Policy
>>>>> routing case
>>>>> by wipeing out a multipath part, so its is lsightly too much, but I
>>>>> suppose (I
>>>>> didn't check) it should work. It is supposed to seamlesly integrate
>>>>> with multipath routing that's why so big,
>>>>> the idea behind is:
>>>>> 1. remove default routing from main table,
>>>>> 2. take care of routing from correct interface (correctness of source
>>>>> IP/ source
>>>>> interface pair),
>>>>> 3. policy routing of selected clients (table 210)
>>>>> 4. default routing of the others (table 211)
>>>>> Of course you can leave it for the main table, but for the sake of
>>>>> example
>>>>> i left it this way.
>>>>>
>>>>> Hope it can help.
>>>>>
>>>>>
>>>>>       
>>>>
>>>> [cut out a part]
>>>>
>>>>
>>>> Dear Robert,
>>>>
>>>> I try this script (adapted for my network) and I get the same problem:
>>>>
>>>> All my IP routed on my first ISP, no problem
>>>> With my 10.117.71.1 routed on my second ISP,
>>>> I can connect to my ISP network (I can connect to the gateway website
>>>> on 5.6.7.9) but
>>>> I cannot ping any external IP addresses.
>>>>     
>>>
>>
>> g> I  also test a DNAT rule to access to my internal network with my 
>> second
>> g> ISP external
>> g> IP ... And it works fine, no problem ...
>>
>> g> iptables -t nat -I PREROUTING  -p tcp -d 1.2.3.4   --dport 80  -j 
>> DNAT
>> g> --to 10.117.71.2:80 # my web server
>>
>> g> I don't know how to make this work for Inside -> outside 
>> connection ...
>>
>> g> Guillaume
>> g> _______________________________________________
>> g> LARTC mailing list / LARTC@mailman.ds9a.nl
>> g> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: 
>> http://lartc.org/
>>
>> Of course I have the patches from Julian Anastasov applied
>> http://www.ssi.bg/~ja/#routes , maybe that's the point
>>
>>  
>>
> In a first time no,
> but now, yes I have apply the routes-2.6.8-10.diff find on 
> http://www.ssi.bg/~ja/#routes-2.6 
> <http://www.ssi.bg/%7Eja/routes-2.6.8-10.diff>
> by patch "patch -p1 < route-2.6.8-10.diff" there is no differences ...
> I could not access to Internet with our without the pach,
>
> How can I be sure that this pach is on my current kernel ?
>
> I will try to re-apply the pach and recompile my kernel.
>
> Thank you,
> Guillaume
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>
In fact, now, I am very close to the solution,

I try this:

ip rule add from 10.117.71.190 lookup 5
ip route add 10.117.71.0/24 via 10.117.71.1 dev eth0 table 5
ip route add 0/0 via 5.6.7.9 dev eth2 table 5

And, it works  (with of course the corresponding POSTROUTING nat rule on 
eth2) !
I can access to showmyip.com from 10.117.71.190
and the web site show me 5.6.7.8 ...

I do not see where is the problem with:

ip rule add fwmark 5 table 5

It's not working with the packet MARK ....

Now I have a solution, I search for this just because i would like to know
why with fwmark packet it's not working

Guillaume
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/