Re: [LARTC] "Bug" in howto 4.2.1 Split access and other advice

[Stef Coene <stef.coene@docum.org>]

Hi,

I was rereading the pages in the howto about multiple ISP's and I also found 
some strange stuff in it.  Maybe you can create a patch for Bert to update 
the howto ?

Stef

PS.  Het wordt hier echt wel een lage-landen onderonsje :)
English : lot of low-country (Belgium, The Netherlands) people on the list.

On Friday 05 July 2002 19:31, Ard van Breemen wrote:
> Hi,
> http://lartc.org/HOWTO//cvs/2.4routing/html/lartc.rpdb.multiple-links.html
> I am not sure who wrote this part or what it was based upon, but
> since I am working a lot longer now with ip rules, I think I want
> to add some stuff:
> The example 4.2.1 refers to the picture above, and does a plain
> ip rule add from .... table ....
> The problem with the exampe is that if you connect from the
> inside (local network) to your if1 ip or if2 ip, that in this
> example the replies to the local-network are going out if1 or
> if2... That is not what you want.
>
> If we carefully study the ip rule set, we see the first number.
> This number is the priority, and if you use this priority number
> in your rule command, it will insert it in the rule list after the
> rule with the same or lower priority.
>
> I am using this to differentiate between known routing, and
> default routing.
>
> So first we set up the link local routing (the things you can
> reach directly). Actually you don't have to do a thing for that,
> except setting up the interface.
> List your local routing to understand:
> ip route show table main
>
> Then we put the default routing into table default.
> And between the main and the default rule we put rules that
> differentiate the default routing per provider.
>
> example:
> ############################################################
> # set up table main by upping the interfaces
> ifup eth0   # local net
> ifup eth1   # provider 1
> ifup eth2   # provider 2
>
> # Now set up the default routes in a failover fashion, with the
> # most important route first(f.i. gw-provider1):
> ip route add default via gw-provider1    table default
> # Secondary route when the first gw fails:
> ip route append default via gw-provider2 table default
> # (The append is needed because else the routes will clash)
> # So we have a table with two default routes wich will failover
> # for eachother (takes about 10 minutes in default config)
>
> # We now have a simple failover, which for most of us will not
> # work, since most providers will have src-ip filtering.
> # We are going to fix that now:
> ip route add default via gw-provider1 table rt-provider1
> ip route add default via gw-provider2 table rt-provider2
> # We now have 2 tables each with a single different default gw.
> # They are not used, that is what we are going to solve now:
>
> ip rule add from ip-eth1 table rt-provider1 prio 32766
> ip rule add from ip-eth2 table rt-provider2 prio 32766
>
>
> # That's it.
> ############################################################
>
> So let's think about it, and look at it:
> ard@erwin(slave):~$ /sbin/ip rule list
> 0:      from all lookup local
> 32766:  from all lookup main
> 32766:  from <someip> lookup <sometable>
> 32767:  from all lookup default
>
> This is not what we did above, but it is a rule list from a
> working environment (does www.telegraaf.nl ring a bell?).
>
> What we should have seen was this:
> /sbin/ip rule list
> 0:      from all lookup local
> 32766:  from all lookup main
> 32766:  from <ip-eth1> lookup rt-provider1
> 32766:  from <ip-eth2> lookup rt-provider2
> 32767:  from all lookup default
>
> The main difference with the example in the document is:
> - We do *not* have a default route in main
> - We *have* default routes in the default table
> - We have rules *after* main, not before main
>
> So what is the catch:
> The only catch is that if you do not have point-to-point
> connections with your provider, but a /24 for example, then
> requests coming in from provider2 for the ip-eth1, will go out
> from your eth2 and not from your eth1. This might be a problem if
> your /24 is filtered by your ISP.
> The solution to that is the essence of this story: move the
> calling of your default route tables from the rules to the last
> possible moment.
> So to fix the catch you get two more routing tables:
> (With a provider3 added for clarity)
> ############################################################
> ip route add net-provider2/24 via gw-provider1 table use-gw-provider1
> ip route add net-provider3/24 via gw-provider1 table use-gw-provider1
> ip rule add from ip-eth1 table use-gw-provider1 prio 32765
>
> ip route add net-provider1/24 via gw-provider2 table use-gw-provider2
> ip route add net-provider3/24 via gw-provider2 table use-gw-provider2
> ip rule add from ip-eth2 table use-gw-provider2 prio 32765
>
> ip route add net-provider1/24 via gw-provider3 table use-gw-provider3
> ip route add net-provider2/24 via gw-provider3 table use-gw-provider3
> ip rule add from ip-eth3 table use-gw-provider3 prio 32765
> ############################################################
>
> I hope this makes some sense, and I hope it also is clear that
> this is only needed for the link-local network of your provider
> only if it is filtered!
>
>
> Next thing: I was talking about failover earlier:
> If a gateway is not available (ie, it does not reply to arps),
> linux will think it is dead within a few minutes, and use the
> other gateway. But only if it reaches the default table.
> It reaches that, when it does not have a clue of the outgoing
> src-ip yet. So if an application makes a connection to a website,
> and the first gateway is considered dead, it will connect to the
> website using the second gateway, and therefore bind to ip-eth2.
>
>
> Last thing: this failover thingie can also be a "loadbalanced"
> thingie as explained in "4.2.2 Load balancing".
> However: due to bugs in the equalizeing code, I recommend against
> it. Somewhere inside the kernel it cannot clearly come up with a
> route, which results in a lot of "cannot happen 777".
> Next to that: the usage counts of the devices are not correctly
> incremented and decremented. You have to be very careful and craft
> an extra non-multipath route before, then remove the existing
> multi-path route, before bringing down a network device. Else it
> ends up in an endless "device still in use, waiting". And you
> will not be able to use the device anymore until you reset some
> sense into the machine...

-- 

stef.coene@docum.org
 "Using Linux as bandwidth manager"
     http://www.docum.org/
     #lartc @ irc.openprojects.net
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/