[LARTC] Routing with two providers

[LARTC] Routing with two providers

[Martin Ferrari - Decidir IT]

Hi people!

I have a question that perhaps some of you have already faced sometime.

The company I work for has a website, and some web applications. Since we
had real troble with Brazilian customers because of netlag, we contracted a
WorldCom link, which has excellent times with Brazilian and non-Brazilian
sites. But we want to retain the old link since is cheaper and OK for
national (Argentina) traffic.

The question is: how would setup a configuration which can use both links
for incoming traffic, and using the best link for returning packets (or at
least, the lenk they came from)?

We are using linux as firewall/NAT/some routing. The servers are on a DMZ,
NATting with ipchains on the firewall.

I'm using different DNS record for the Brazilian services, so I can point to
the WorldCom IP of the servers, but I couldn't get to work OK the response
packets, they go by the wrong interface, and UDP response packets (DNS) do
strange things (some don't even go out of the firewall box)


Thanks a lot!!!!!

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Routing with two providers

[Arthur van Leeuwen]

On Tue, 9 Apr 2002, Martin Ferrari - Decidir IT wrote:

> I have a question that perhaps some of you have already faced sometime.

See http://mailman.ds9a.nl/pipermail/lartc/2002q2/003111.html

As you may note, this was written 7 days ago.

Doei, Arthur. (Busy trying to check out the LARTC CVS tree and writing a
               patch to it that includes exactly this scenario... it's
               *very* much a FAQ)

-- 
  /\    / |      arthurvl@sci.kun.nl      | Work like you don't need the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there's nobody watching

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

RE: [LARTC] Routing with two providers

[Martin Ferrari - Decidir IT]

> > I have a question that perhaps some of you have already 
> faced sometime.
> 
> See http://mailman.ds9a.nl/pipermail/lartc/2002q2/003111.html
> 
> As you may note, this was written 7 days ago.

Sorry, I'd suscribed months ago, but I'd missed that one.

Do you know if that way UDP will work okay? and NAT?
The Alexey's patches has some influence on this?

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

RE: [LARTC] Routing with two providers

[Arthur van Leeuwen]

On Wed, 10 Apr 2002, Martin Ferrari - Decidir IT wrote:

> > > I have a question that perhaps some of you have already
> > faced sometime.
> >
> > See http://mailman.ds9a.nl/pipermail/lartc/2002q2/003111.html
> >
> > As you may note, this was written 7 days ago.
>
> Sorry, I'd suscribed months ago, but I'd missed that one.

No problem. It'll be included in the HOWTO within days from now.

> Do you know if that way UDP will work okay? and NAT?

Yes. Yes. Yes. Yes.

> The Alexey's patches has some influence on this?

Err... good question. What do you mean with 'the Alexey's patches'?
You need a kernel newer than (say) 2.2.16, and then it'll just work.
The work was done by Alexey, yes, but it's been included in the
kernel for quite some time already.

Doei, Arthur.

-- 
  /\    / |      arthurvl@sci.kun.nl      | Work like you don't need the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there's nobody watching

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

RE: [LARTC] Routing with two providers

[Martin Ferrari - Decidir IT]

> > The Alexey's patches has some influence on this?
> 
> Err... good question. What do you mean with 'the Alexey's patches'?
> You need a kernel newer than (say) 2.2.16, and then it'll just work.
> The work was done by Alexey, yes, but it's been included in the
> kernel for quite some time already.

Oops, I'd confused Alexey with Julian Anastasov.. I meant Julian' patches
(http://www.linuxvirtualserver.org/~julian) which apply to 2.2 and 2.4
kernels...

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

RE: [LARTC] Routing with two providers

[Arthur van Leeuwen]

On Wed, 10 Apr 2002, Martin Ferrari - Decidir IT wrote:

> > > The Alexey's patches has some influence on this?
> >
> > Err... good question. What do you mean with 'the Alexey's patches'?
> > You need a kernel newer than (say) 2.2.16, and then it'll just work.
> > The work was done by Alexey, yes, but it's been included in the
> > kernel for quite some time already.
>
> Oops, I'd confused Alexey with Julian Anastasov.. I meant Julian' patches
> (http://www.linuxvirtualserver.org/~julian) which apply to 2.2 and 2.4
> kernels...

Yes, Julian's patches do have an influence on multipath routing. They make
it quite a bit nicer to use in the case of interfaces going down and coming
back up again, which can and does happen. They also make multipath routing
and masquerading play even nicer with each other, although I haven't seen
problems on stock kernels with that. May very well be that the tests I've
done have been on networks with highly active but not highly demanding i
users, thereby making sure the route cache stays up to date enough...
don't really know.

Doei, Arthur.

-- 
  /\    / |      arthurvl@sci.kun.nl      | Work like you don't need the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there's nobody watching

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Routing with two providers

[Jason A. Pattie]

Arthur van Leeuwen wrote:

>On Wed, 10 Apr 2002, Martin Ferrari - Decidir IT wrote:
>
>Yes, Julian's patches do have an influence on multipath routing. They make
>it quite a bit nicer to use in the case of interfaces going down and coming
>back up again, which can and does happen. They also make multipath routing
>and masquerading play even nicer with each other, although I haven't seen
>problems on stock kernels with that. May very well be that the tests I've
>done have been on networks with highly active but not highly demanding i
>users, thereby making sure the route cache stays up to date enough...
>don't really know.
>
>Doei, Arthur.
>
The only problem that I have had with Julian's patches is interoperation 
with FreeS/WAN.  I am still not able to make that work, although I 
haven't worked on it in awhile.  The last I remember is that with the 
patches applied, the moment FreeS/WAN starts, all network traffic goes 
out the ipsec0 interface instead of continuing to be routed via eth0 (or 
whichever interface).  This happens without a tunnel brought up.  And 
for some reason, I was not able to assign a metric to the route using 
either the 'route' command or 'ip route'.

-- 
Jason A. Pattie
pattieja@pcxperience.com


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Routing with two providers

[Arthur van Leeuwen]

On Wed, 10 Apr 2002, Jason A. Pattie wrote:

> Arthur van Leeuwen wrote:
>
> >On Wed, 10 Apr 2002, Martin Ferrari - Decidir IT wrote:
> >
> >Yes, Julian's patches do have an influence on multipath routing. They make
> >it quite a bit nicer to use in the case of interfaces going down and coming
> >back up again, which can and does happen. They also make multipath routing
> >and masquerading play even nicer with each other, although I haven't seen
> >problems on stock kernels with that. May very well be that the tests I've
> >done have been on networks with highly active but not highly demanding i
> >users, thereby making sure the route cache stays up to date enough...
> >don't really know.
> >
> >Doei, Arthur.
> >
> The only problem that I have had with Julian's patches is interoperation
> with FreeS/WAN.  I am still not able to make that work, although I
> haven't worked on it in awhile.  The last I remember is that with the
> patches applied, the moment FreeS/WAN starts, all network traffic goes
> out the ipsec0 interface instead of continuing to be routed via eth0 (or
> whichever interface).  This happens without a tunnel brought up.  And
> for some reason, I was not able to assign a metric to the route using
> either the 'route' command or 'ip route'.

Sorry, can't help you there. Never played with that particular setup...

Doei, Arthur.

-- 
  /\    / |      arthurvl@sci.kun.nl      | Work like you don't need the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there's nobody watching

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Routing with two providers

[Julian Anastasov]

	Hello,

On Wed, 10 Apr 2002, Jason A. Pattie wrote:

> The only problem that I have had with Julian's patches is interoperation
> with FreeS/WAN.  I am still not able to make that work, although I
> haven't worked on it in awhile.  The last I remember is that with the
> patches applied, the moment FreeS/WAN starts, all network traffic goes
> out the ipsec0 interface instead of continuing to be routed via eth0 (or
> whichever interface).  This happens without a tunnel brought up.  And

	Hm. IIRC, the default updown script in FreeSWAN creates
routes with the "route" utility. That means they are
"from all to remote_net via XXX dev ipsecX". FreeSWAN is ready
for this, it just forwards the gw->gw traffic via the configured
nexthop without encryption, so it looks like it is not related to
the route patches. Is that correct?

> for some reason, I was not able to assign a metric to the route using
> either the 'route' command or 'ip route'.

	If you try to add different metric to the different
alternative routes this is not possible by design. All alternative
routes have same metric value. This is the difference between
"ip route add" and "ip route append".

Regards

--
Julian Anastasov <ja@ssi.bg>

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Routing with two providers

[Jason A. Pattie]

Julian Anastasov wrote:

>>The only problem that I have had with Julian's patches is interoperation
>>with FreeS/WAN.  I am still not able to make that work, although I
>>haven't worked on it in awhile.  The last I remember is that with the
>>patches applied, the moment FreeS/WAN starts, all network traffic goes
>>out the ipsec0 interface instead of continuing to be routed via eth0 (or
>>whichever interface).  This happens without a tunnel brought up.  And
>>
>
>	Hm. IIRC, the default updown script in FreeSWAN creates
>routes with the "route" utility.
>
That is correct.

> That means they are
>"from all to remote_net via XXX dev ipsecX". FreeSWAN is ready
>for this, it just forwards the gw->gw traffic via the configured
>nexthop without encryption, so it looks like it is not related to
>the route patches. Is that correct?
>
I don't know.  If the patch is not in the kernel, everything works fine. 
 The moment I boot with the patched kernel, normal networking stops 
working the once ipsec is started.  The defaultroute routes are added 
apparently before any tunnels are brought up.  An ipsecN interface is 
bound to whichever interface (eth0, eth1, ..., ethN) you have specified. 
 Routes are then added that mirror the routes to these devices for the 
network of that device.  Without the patch, this problem can be 
duplicated if you first bring down normal networking (ifdown eth0) but 
leave FreeS/WAN running.  Then restart normal networking (ifup eth0) and 
the routes will be reversed in the routing table apparently giving 
precedence to ipsecN routes.  This causes all normal traffic to be 
attempted to be sent through the ipsec interface instead of the normal 
ethN interface that the ipsecN interface is bound to.  Apparently 
something very similar happens with the patches in place, because if the 
ipsec routes are removed manually and reinserted into the running kernel 
AFTER the routes for the normal network interface, things start working 
again.  The only way I could get that to work was by assigning a metric 
(somehow) to the normal networking route (or maybe it was the ipsec 
networking route).  Then the normal networking route took precedence 
over the ipsec networking route.

>>for some reason, I was not able to assign a metric to the route using
>>either the 'route' command or 'ip route'.
>>
>If you try to add different metric to the different
>alternative routes this is not possible by design. All alternative
>routes have same metric value. This is the difference between
>"ip route add" and "ip route append".
>
Hmm.  Didn't realize there was a difference.

-- 
Jason A. Pattie
pattieja@pcxperience.com


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

RE: [LARTC] Routing with two providers

[Martin Ferrari - Decidir IT]

> > I have a question that perhaps some of you have already 
> faced sometime.

Well, after sometime working fine, now I can say that it worked!!! Thanks
Arthur and thanks Julian... When I first asked I really did not understand
Julian's answer, after Arthur's response, I re-read a lot of things
(specially the nano-howto) and finally understood them.

Now I have a setup of two upstream links, load balancing to the outside and
respecting the incoming route when answering...


Martín.

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/