Re: [LARTC] Dead Gateway Detection & BGP

[Grant Taylor <gtaylor@riverviewtech.net>]

(Before any one questions why I withheld information and went down the 
road that I did, I'd like to say that I had fully intended to respond 
with more detail, however other things going on both at work and home 
prevented me from doing so before now.  I also sort of paused because of 
the discussion that arose out of the road that I did go down.)

On 8/26/2007 12:29 PM, Rangi Biddle wrote:
>          +-----------------+
>          | Uplink Provider |
>          +--------+--------+
>                   |
>         +---------+---------+
>         |                   |
> +-------+-------+   +-------+-------+
> | Cisco  Router |   | Cisco  Router |
> +-------+-------+   +-------+-------+
>         |                   |
> +-------+-------+   +-------+-------+
> | Firewall # 1  |   | Firewall # 2  |
> +---------------+   +-------+-------+
> 
> Initially, the first task I was designated was to setup BGP routing 
> on 2 firewalls.  Each firewall is connected to its own Cisco router 
> provided by the uplink provider and the uplink provider is only 
> providing a default gateway/router to each of the firewalls.  Now, 
> having had minimal experience with BGP (minimal in terms of the 
> broadness of what is possible with BGP) and using the information 
> provided by the uplink provider I have setup BGP.

Question:
  - Are there multiple providers in this situation or one single 
provider that has chosen to do this type of set up.
  - If there are multiple providers, are they in any sort of peering 
relationship between them?
  - Is there suppose to be any sort of redundancy amongst the two Cisco 
routers or are they to be two purely independent non redundant connections?
  - What type of connections are there in to the two Cisco routers?
  - Are the Cisco routers actually routing, or just bridging between two 
layer 1 technologies?
  - Is ethernet being used between the Cisco routers and the Debian 
firewalls?
  - What type of (if any) IP address range overlap are we looking at?

Answers to each of these questions will most likely beget more questions 
until finally a much clearer picture of what ultimately is being done 
emerges.  This is also part of why I was wanting to do this off mailing 
list as some of these answers are not appropriate for a public form that 
is archived and search able.

> What I have been recently informed of is that the 2 firewalls must do 
> some sort of failover between them when either of the default 
> gateway’s are no longer responsive.  I had initially looked into 
> using heartbeat (which I am still considering) to do the failover or 
> possibly using vrrpd (Virtual Router Redundancy Protocol Daemon). 
> This however isn’t what I am contacting this list about.  What I need 
> to do at minimal, is at least for the failover, is to detect when the 
> default gateway of (say) firewall 1 is no longer available and 
> perform failover to firewall 2 and vice versa.  As far as  I am aware 
> the only DGD support available is still through the patches that 
> Julian Anastasov wrote for the 2.4 kernel series or by writing a 
> script that uses arping to determine the last hop available.

Hum.  I'm not entirely sure what is suppose to be redundant here, the 
Cisco routers, the Debian firewalls, a logical router (or routers) that 
are presented to your systems behind the firewalls, what.  Will you 
please clarify?

> What other options are there?

More than you might initially think.

> I have done a fair amount of searching the internet only to come back 
> to these 2 possibilities.  Surely there must be something else ….

Well, in my opinion, what you have proposed is a couple of different 
solutions to the same piece of the puzzle.

Presuming that you are dealing with T-1s from your provider(s), let's 
start with a modified version of your above network layout.

          +-----------------+
          | Uplink Provider |
          +--------+--------+
                   |
         +---------+---------+
         |                   |
+-------+-------+   +-------+-------+
|   Atlas 550   +---+   Atlas 550   |
+-------+---+---+   +---+---+-------+
         |   |           |   |
         |    \         /    |
         |     \       /     |
         |      \     /      |
         |       \   /       |
         |        \ /        |
         |         X         |
         |        / \        |
         |       /   \       |
         |      /     \      |
         |     /       \     |
         |    /         \    |
         |   |           |   |
+-------+---+---+   +---+---+-------+
| Cisco  Router +---+ Cisco  Router |
+-------+---+---+   +---+---+-------+
         |   |           |   |
         |    \         /    |
         |     \       /     |
         |      \     /      |
         |       \   /       |
         |        \ /        |
         |         X         |
         |        / \        |
         |       /   \       |
         |      /     \      |
         |     /       \     |
         |    /         \    |
         |   |           |   |
+-------+---+---+   +---+---+-------+
|    Switch     +---+    Switch     |
+-------+---+---+   +---+---+-------+
         |   |           |   |
         |    \         /    |
         |     \       /     |
         |      \     /      |
         |       \   /       |
         |        \ /        |
         |         X         |
         |        / \        |
         |       /   \       |
         |      /     \      |
         |     /       \     |
         |    /         \    |
         |   |           |   |
+-------+---+---+   +---+---+-------+
| Firewall # 1  +---+ Firewall # 2  |
+-------+---+---+   +---+---+-------+
         |   |           |   |
         |    \         /    |
         |     \       /     |
         |      \     /      |
         |       \   /       |
         |        \ /        |
         |         X         |
         |        / \        |
         |       /   \       |
         |      /     \      |
         |     /       \     |
         |    /         \    |
         |   |           |   |
+-------+---+---+   +---+---+-------+
|    Switch     +---+    Switch     |
+-------+-------+   +-------+-------+
         |                   |
    ...--+--...--(LAN)--...--+--...

Now that the ASCII art is out of the way, let's have some explanation as 
to what each piece of the puzzle is for.

Physical Layer
--------------

The "Atlas 550"s are devices to switch / route T-1 on a phone company / 
circuit level.  In other words they can take a T-1 in and give a T-1 out 
based on different conditions with in the circuit on a given interface. 
  In short the Atlas 550 will allow you to route an inbound T-1 the 
primary interface if the equipment that the primary interface is 
connected to is up and handling traffic.  If the equipment that the 
primary interface connected to is not up and handling traffic route the 
T-1 out the secondary interface.  If for some reason the equipment that 
the secondary interface is connected to is not handling traffic route 
the T-1 out the tertiary interface to the backup Atlas in hopes that the 
cabling between the original Atlas and the primary and secondary 
equipment is down and that the backup Atlas has functioning cable.

The Cisco routers are similarly configured with two T-1 WICs each so 
that each can connect to both Atlas 550s.  Also there is a similar setup 
between the Cisco routers and the ethernet switches and each other.

Likewise the switches have a similar set up to connect to the firewall 
boxen as well as the firewall boxen do to the internal LAN switch(es).

Data Layer
----------
Each Atlas 550s can redundantly route their inbound T-1 to two different 
routers configured redundantly for each other or to the other Atlas 550.

Each Cisco router can redundantly route their inbound T-1s to two 
different switches configured redundantly for each other or to the other 
router.

Each switch can redundantly switch their inbound network segments to two 
different firewalls configured redundantly for each other or to the 
other switch.

Each firewall can redundantly filter their inbound network segments to 
two different switches configured redundantly for each other or to the 
other firewall.

Each switch can redundantly switch their inbound network segment to the 
internal LAN or to the other switch.

Network Layer
-------------
Each Atlas 550 would be configured to be able to handle the others T-1 
in the event that the other is unable to reach its desired router.

Each Cisco router would be configured to be able to handle the other 
routers circuit in addition to its own circuit, thus you could have a 
Cisco router die with out adversely effecting your network.  If I could, 
I would probably use HSRP or VRRP between the Cisco routers so that they 
could be redundant for each other.

Each switch is used for basic network connectivity allowing for more 
intermediary equipment.  If this is the only equipment you are going t 
have you could take the core switches out of the mix and go from the 
Cisco routers straight in to the firewalls.  However these switches will 
allow for more future expansion and other options down the road.  For 
example, either of the switches, if managed, would allow you to mirror 
traffic from one port to another for sniffing.

Each firewall would be able to filter traffic for its primary circuit as 
well as backup filter for the other firewalls backup circuit.  I would 
use VRRP to allow multiple physical firewalls to be redundant for each 
others IP address.  For example, make firewall A be primary for IP 1 and 
secondary for IP 2 while making firewall B be primary for IP 2 and 
secondary for IP 1.  Thus each firewall is redundant on its WAN facing 
side.  Do something similar for the LAN facing side.  If you decide that 
one connection from your provider is primary and the other is backup, 
you could route inbound traffic through one firewall while routing 
outbound traffic through the other firewall for load balancing / 
distribution reasons.  If you have the ethernet switches in place you 
could even insert a third firewall ans an inactive backup system to be 
used if either of the primary systems go down.  I would recommend that 
you use ConnTrackd to synchronize the firewall state between the two (or 
more) firewalls.

Each switch is used to allow connectivity between the two (or more) 
firewalls with the internal LAN.

As you can see there really is not a single point of failure between 
where the provider leaves off and the workstations pick up.

> Thanks in advance to anyone that replies as I know that this topic 
> seems to be coming up more and more frequently on the lists and must 
> be getting somewhat tedious for most.

*nod*

> Regards,

*nod*

Chew on this and let me know what you think.



Grant. . . .
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc