[LARTC] Advanced Policy Routing not working properly

[LARTC] Advanced Policy Routing not working properly

[Andre D. Correa]

Hi list, I'm trying to setup a Linux box with a complicated source 
routing and could use a hand from you.

The box has 4 NICs and lots of VLANs attached. It is a firewall and 
router in the following scenario: (obs: IP addresses have being changed 
for security purposes)


- eth0 holds the default route (GW: 200.1.0.1, Firewall: 200.1.0.2);
- The box is routing and sometimes source routing, with no problems;
- We got our own ASN with a IP range assigned: 101.30.0.0/20;
- We have a Cisco router responsible for BGP sessions of our ASN. This 
router is already talking to our neighbors and connects to the Firewall 
on eth2.887 (Router: 101.30.15.249, Firewall: 101.30.15.250);
- We have old ISP's IP addresses used on lots of VLAN interfaces, ex: 
200.1.2.0/26, 200.1.3.0/24, etc;
- The default route is still pointing to our old ISP and cannot be 
changed by now;

So far so good, but:

- We created a testing VLAN, eth2.6, and assigned the address 
101.30.0.1/28 to the Firewall and 101.30.0.2 to a testing machine 
(machine-X);
- if we create a source routing like this:

ip route add default via 101.30.15.249 table MyASN  # IP of BGP router
ip rule add from 101.30.0.0/28 table MyASN

we can see the Internet and the Internet see us through our BGP router 
and neighbors, BUT we cannot see hosts at IP addresses of our old ISP 
(those directly connected to the Firewall). The reason is simple, table 
MyASN has no entry to these old addresses. The easy way to go is to 
insert static routes on MyASN, but it is a bad solution when you have 
lots of subnets in use and changes occur frequently.

The old and new addresses (from my old ISP and from my ASN) must 
communicate but I cannot keep updating MyASN table.

I tried some workarounds with no good results and here is where I need a 
hand.

All the workarounds I tried expect that in the above scenario if a host 
on old ISP's IP address, lets say 200.1.2.2, pings my testing server: 
machine-X on 101.30.0.2, packets should show up on the sender host 
interface and go out on machine-x interface. I expect this as the _main_ 
table has a route to machine-x (directly connected to the Firewall) so 
the box should know where to send packets. It doesn't happen like this. 
The packets goes nowhere. They come on the sender host interface but 
never go out on machine-x interface. If I insert a route to 200.1.2.2 on 
table MyASN I start to see traffic coming and going.

Why is this happening? Shouldn't the box just forward traffic when there 
is a route in the _main_ table regardless of existing or not a route of 
return? Or shouldn't it, at least, send this traffic to its default gateway?

Any comments and suggestions are appreciated.

Regards.


--------------------------------------------------------------------
Andre D. Correa, CISSP          |  Visite meus projetos pessoais:
andre.correa (at) pobox.com     |  Visit my personal projects:
http://andre.hiperlinks.com.br  |  - http://www.malware.com.br/
Sao Paulo / SP / Brazil         |  - http://www.linuximq.net/
--------------------------------------------------------------------

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Advanced Policy Routing not working properly

[Martin A. Brown]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings Andre Correa,

 : ip route add default via 101.30.15.249 table MyASN  # IP of BGP router
 : ip rule add from 101.30.0.0/28 table MyASN
 : 
 : we can see the Internet and the Internet see us through our BGP 
 : router and neighbors, BUT we cannot see hosts at IP addresses of 
 : our old ISP (those directly connected to the Firewall). The 
 : reason is simple, table MyASN has no entry to these old 
 : addresses. The easy way to go is to insert static routes on 
 : MyASN, but it is a bad solution when you have lots of subnets in 
 : use and changes occur frequently.

By altering the manner in which you make your routing decisions, you 
should be able to solve your problem fairly easily.  The key is to 
understand how to take advantage of the routing policy database.  
Given your description, you have a main routing table (table 254) 
and a table named MyASN (let's say table 250).

You have a number of locally connected networks, and you can manage 
these networks all in a single routing table...let's call it 
legacynets (table 240).  The legacynets routing table should only 
contain routes for the locally connected networks in your old 
netblock.

  # ip rule add prio 32000 from 101.30.0.0/28 table MyASN
  # ip rule add prio 31000 from all table legacynets
  # ip rule show
  0:      from all lookup local 
  31000:  from all lookup legacynets 
  32000:  from 101.30.0.0/28 lookup MyASN 
  32766:  from all lookup main 
  32767:  from all lookup default 

Now, ask yourself what happens at route lookup time.  Let's say we 
have a single inbound packet from 101.30.0.0/28 to 200.1.3.17.

  * kernel checks the routing cache; if cached route is found select
    that route, otherwise perform routing lookup
  * check the routing policy database, to determine which routing 
    table to select for first lookup
  * priority 0 (highest) requires us to check the local routing 
    table (lookup in local routing table for 200.1.3.17); no route 
    found? return to routing policy database to see which is the
    next table to select for lookup
  * lookup route for 200.1.3.17 in table known as legacynets

I hope that it's obvious at this point how you can generalize this 
solution for your network.  Here are a few items:

  * copy_routing_table is a quick-n-dirty function which populates 
    one routing table based on the contents of another [0]
  * you can, alternatively use the "throw" route to escape a routing 
    table and continue to traverse the RPDB, in the event that you
    have a destination that you'd like to handle separately:

      ip route add throw 200.1.3.17 table legacynets

    A route like the above would mean that lookups in the routing 
    table for legacynets would (effectively) return to the RPDB for 
    the selection of the next viable routing table.

 : All the workarounds I tried expect that in the above scenario if 
 : a host on old ISP's IP address, lets say 200.1.2.2, pings my 
 : testing server: machine-X on 101.30.0.2, packets should show up 
 : on the sender host interface and go out on machine-x interface. I 
 : expect this as the _main_ table has a route to machine-x 
 : (directly connected to the Firewall) so the box should know where 
 : to send packets. It doesn't happen like this. The packets goes 
 : nowhere. They come on the sender host interface but never go out 
 : on machine-x interface. If I insert a route to 200.1.2.2 on table 
 : MyASN I start to see traffic coming and going.

Without twiddles to the sysctl settings, a (well-behaved) Linux 
router will not send packets out the same interface on which it 
receives them*.

 : Why is this happening? Shouldn't the box just forward traffic 
 : when there is a route in the _main_ table regardless of existing 
 : or not a route of return? Or shouldn't it, at least, send this 
 : traffic to its default gateway?

With more knowledge about the routing process (see also [1]) and 
routing policy database, it should be a bit more obvious why this 
happens.  I suggest that for each packet (or flow) that is not 
working, you walk through the route selection process.  This will 
help you internalize the kernel's routing logic and the solution 
should jump to your mind.  (Hint:  It probably involves a route 
lookup in a routing table containing a default route before a 
routing table containing a more specific route.)

Good luck,

- -Martin

   * For those who like to quibble, it is certainly possible to do 
     this, but it is a fairly sane default to suppress logical 
     one-armed router scenarios.  If you know what you are doing,
     there's nothing wrong with this configuration.

 [0] # - - - - - - - - - - -
       copy_routing_table () {
     # - - - - - - - - - - -
     #
     # -- accepts one parameter:
     #
     #    $1:  table identifier for the routing table to create
     #
       test "$#" -lt "1"     && return
       DTABLE=$1
     
       test "$#" -gt "1"     && STABLE="$2"
       test "$STABLE" = ""   && STABLE="main"
     
       ip route flush table "$DTABLE"
       ip route show table "$STABLE" | grep -Ev '^default' \
         | while read ROUTE ; do
           ip route add table "$DTABLE" $ROUTE
       done
     
     }

 [1] http://linux-ip.net/html/routing-selection.html


- -- 
Martin A. Brown
http://linux-ip.net/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: pgf-0.72 (http://linux-ip.net/sw/pine-gpg-filter/)

iD8DBQFFrOT/HEoZD1iZ+YcRAgwTAJ4qOm6DECDdvmAyk2qQ2FkSWClzAwCgiTiP
hZRW7ypLM65/pj+D0JmlMcA=hZeL
-----END PGP SIGNATURE-----
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc