[LARTC] two upstreams without nat

[LARTC] two upstreams without nat

[Tomas Bonnedahl]

im in the process of configurating our network to have two upstream providers, it will be loadbalanced
under normal operation and a complete failover if one of the lines would fail.

internet		internet
  |			  |
border			border
  |                       |
      |- core router - |
	     |
	    lan


the "problem" im having is that i will not do nat on the core router, but on the border routers. 
the multipath default route is on the core router. from what i understand, could be totally wrong,
you have to have nat, at least connection tracking on the core to make the multipath route per
flow and not per packet.

any insight of this?


-tomas bonnedahl



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] two upstreams without nat

[William L. Thomson Jr.]

On Wed, 2003-06-25 at 04:35, Tomas Bonnedahl wrote:

> the "problem" im having is that i will not do nat on the core router, but on the border routers.

I was faced with the same problem and ended up doing two rounds of
NAT/PAT. The next step to that is to stop doing any NAT on the routers
and let the core router deal with all that. From my experience a
properly designed and dialed in Linux router can perform better than
most other name brand dedicated routers.

Now I am not saying it will be out a $100,000 Cisco router. The
performance should easily be equal to or greater than your existing
routers.

For example when I had my setup in CA my Linux router through put
latency was about half that of my Cisco 827 ADSL router, or either of my
Netopia SDSL routers.
 
> the multipath default route is on the core router.

Linux router, correct.

>  from what i understand, could be totally wrong,
> you have to have nat, at least connection tracking on the core to make the multipath route per
> flow and not per packet.

Correct, sort of. NAT will keep the path in cache, which will allow
packets to keep traveling the same router.

The word flow is much better than connection. You will not get per
connection load balancing. Either way using multipath it will be per
packet load balancing. However with NAT and Julian's patches the NAT
routes are cached which will allow further packets to flow or traverse
the same path.

I have seen others, I think even Julian, said that it is possible to
accomplish without NAT. That has not been my experience. Based on my
experience I would say that NAT is a must.

-- 
Sincerely,
William L. Thomson Jr.
Support Group
Obsidian-Studios, Inc.
3548 Jamestown Ln.
Jacksonville, FL 32223
Phone/Fax  904.260.2445
http://www.obsidian-studios.com

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] two upstreams without nat

[Aaron Dewell]

Perhaps I missed the original point of the first message, but why exactly
don't you just use BGP, as it was basically designed for this purpose?

There are at least two good implementations of BGP for Linux, one of which
is easy to use, the other obfuscated.  (Zebra and GateD)  Of course, that
requires having globally routable address space in the first place, but I
assume that you do.

Is there a reason not to use BGP in this case?

Aaron

On Wed, 25 Jun 2003, William L. Thomson Jr. wrote:
> On Wed, 2003-06-25 at 04:35, Tomas Bonnedahl wrote:
>
> > the "problem" im having is that i will not do nat on the core router, but on the border routers.
>
> I was faced with the same problem and ended up doing two rounds of
> NAT/PAT. The next step to that is to stop doing any NAT on the routers
> and let the core router deal with all that. From my experience a
> properly designed and dialed in Linux router can perform better than
> most other name brand dedicated routers.
>
> Now I am not saying it will be out a $100,000 Cisco router. The
> performance should easily be equal to or greater than your existing
> routers.
>
> For example when I had my setup in CA my Linux router through put
> latency was about half that of my Cisco 827 ADSL router, or either of my
> Netopia SDSL routers.
>
> > the multipath default route is on the core router.
>
> Linux router, correct.
>
> >  from what i understand, could be totally wrong,
> > you have to have nat, at least connection tracking on the core to make the multipath route per
> > flow and not per packet.
>
> Correct, sort of. NAT will keep the path in cache, which will allow
> packets to keep traveling the same router.
>
> The word flow is much better than connection. You will not get per
> connection load balancing. Either way using multipath it will be per
> packet load balancing. However with NAT and Julian's patches the NAT
> routes are cached which will allow further packets to flow or traverse
> the same path.
>
> I have seen others, I think even Julian, said that it is possible to
> accomplish without NAT. That has not been my experience. Based on my
> experience I would say that NAT is a must.
>
>
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] two upstreams without nat

[William L. Thomson Jr.]

On Wed, 2003-06-25 at 13:44, Aaron Dewell wrote:
>  Of course, that
> requires having globally routable address space in the first place, but I
> assume that you do.

Most of the time this is the exact reason why BGP is not used. I have
considered purchasing a block. However I doubt many ISP's would want to
route those IP's to DSL lines or equivalent. Maybe a T-1.

So if you have DSL lines or equivalent and a small block of static or
dynamic IP addresses then BGP is not an option.

-- 
Sincerely,
William L. Thomson Jr.
Support Group
Obsidian-Studios, Inc.
3548 Jamestown Ln.
Jacksonville, FL 32223
Phone/Fax  904.260.2445
http://www.obsidian-studios.com

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] two upstreams without nat

[William L. Thomson Jr.]

On Wed, 2003-06-25 at 19:02, Aaron Dewell wrote:
> DSL can be an option, if you have at least a /24, which if you have 3 routers,
> you probably do.  Some ISPs will think about it over DSL (business class).

Covad no
BellSouth no
PacBell no

assume mom and pops

Sonic no
ATG no

So for most no. Not at least for DSL

> If you are using residential class (1 IP, static or not), then sure, you have
> to do something really, really obscure to multi-home, but then you are kinda
> getting what you deserve then, IMHO.

True, but those still may have the need or want.

> The proper solution is to get more IP addresses from your provider, then
> announce those via BGP.  That's usually not a big problem unless you have a
> perceived lack of BGP clue, or a crappy provider.

Yes but part of the idea is multiple providers. Next step for me will be
to load balance a SDSL and CDMA wireless connection. At my previous
location in CA I had two separate SDSL providers and a separate ADSL
provider. All had different networks, but would still travel the same
few miles to the CO.

The wireless way will provide a further level of redundancy in case a
physical line is broken between the CO and me.

> You don't need to purchase the block from ARIN, just get some from your
> provider.

I do and they allocate them to you. However most providers have down
time for servicing and etc. Especially on DSL lines business class or
not. So multiple providers is an added benefit since both usually are
not down at the same time. Except for the scenario above thus wireless.

One provider will not route for the other provider. So getting a block
from one and having another route them for you as well is not an option.

If you know of any ISPs that differ please let me know.

-- 
Sincerely,
William L. Thomson Jr.
Support Group
Obsidian-Studios, Inc.
3548 Jamestown Ln.
Jacksonville, FL 32223
Phone/Fax  904.260.2445
http://www.obsidian-studios.com

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] two upstreams without nat

[Julian Anastasov]

	Hello,

On Wed, 25 Jun 2003, Tomas Bonnedahl wrote:

> im in the process of configurating our network to have two upstream 
> providers, it will be loadbalanced
> under normal operation and a complete failover if one of the lines would 
> fail.
>
> internet		internet
>   |			  |
> border			border
>   |                       |
>       |- core router - |
> 	     |
> 	    lan

	The problem is that the multipath route assumes one packet
can go through any of the defined nexthops. This is valid for setups
with many ISPs allowing spoofing, you can safely send packet with
saddr=IP1 via ISP2 and this works as long as ISP1 is UP to receive
the replies from remote hosts. But for your setup there is one
requirement: once one connection is routed via Border X all its
packets should go via the same border router for NAT purposes.
Even the related ICMP traffic. And this works ... for some time
until the routing cache is flushed and different Border router is
selected from the multipath route. The connection usually dies.

> the "problem" im having is that i will not do nat on the core router, but on the border routers.
> the multipath default route is on the core router. from what i understand, could be totally wrong,
> you have to have nat, at least connection tracking on the core to make the multipath route per
> flow and not per packet.

	I'm not sure the Linux conntracking has such feature. IIRC,
the netfilter follows the routing decision, i.e. nobody guarantees
that packets from one connection will continue to use same nexthop
while it is alive. Considering the fact that all cache entries
expire (on user request or on timeout) the current implementation
can not guarantee such persistence for the forwarded connections
that have multiple possible paths. This works only if the core router
performs NAT and when the NAT implementation does not blindly
follow the multipath route decision. The problem can be solved in
this way: the NAT connection requests route with saddr=the external
IP address attached for this connection. As result, the multipath
route is avoided and the traffic for one connection is bound to
one ISP (usually): http://www.ssi.bg/~ja/#routes

> any insight of this?
>
>
> -tomas bonnedahl

Regards

--
Julian Anastasov <ja@ssi.bg>


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] two upstreams without nat

[Tomas Bonnedahl]

On Thu, Jun 26, 2003 at 09:50:45AM -0600, Aaron Dewell wrote:
> On Thu, 26 Jun 2003, Tomas Bonnedahl wrote:
> > i dont have any addresses nor do i own an AS, i know there are private ASNs to
> > use but this seems like a more complicated solution than a mere multipath default
> > route to the two upstream providers.
 
> An ASN can be gotten from ARIN with the justification "I'm multihomed to ASN #X
> and #Y" and $500.  Or you can use a private AS and have your upstreams filter
> it out, also reasonably common.

i didnt know it was that easy really, this might be an option. 

> BGP is not complicated at all to use, that's a myth.  It's a fairly simple
> protocol, and even easier to set up.  Define one external peer per router, one
> internal peer (each other), this is all done by AS.  Set up the routes you want
> advertised.  In this case, you want everything, so no inbound filtering.  Done.
> 3 configuration options in Zebra's bgpd.  Less complicated than setting up NAT.

i assume i will only advertise the core (some /28) since the lan is still a 
private network. i probably wont be able to get a whole /24 from my upstream. 

> Think about it - if you have two IP addresses total, one assigned by each
> upstream, and using two default routes, anything connection-oriented is
> broken immediately (TCP comes to mind).  Anything connectionless (i.e. UDP)
> will likely work fine.  Web, ssh, IMAP, POP3, SMTP are all TCP.  Those not
> working make it basically useless.

why wont it work? from what i understand, you could get a "per flow" with julians
patches so the core-router doesnt varies on a per packet basis and thus make established
connections to fail.

> Otherwise, you have to have selective routes.  Route this block of the internet
> through provider X, that block through provider Y.  No failover, no redundancy,
> no point.  Or, you could point default and provider X and a lower priority to
> provider Y, but then you have to learn by IGP at your core when provider X dies.
> That means advertising default from the borders with your IGP, which is a
> workable solution, but could get messy if you're not pretty good at whatever
> IGP you are using, making the assumption that your IGP will do it.  However,
> two problems:  1.  Your second connection is idle until the primary fails, thus
> wasting money.  2.  All TCP connections reset when you fail over to the backup,
> and reset again when you resume to the primary.

i thought the multihop path was designed to solve this issue with redundancy and 
failover? my very first thought in this was to use ospf as IGP but i couldnt come
up with something to use upstream to see if the providers still were under normal
operation. 

just to sum it up: use something like ospf as IGP and use BGP upstream. were you
assuming that i would get a /24 from my isp and use for lan or should i do nat
on the core router from the lan?

thanks, tomas
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/