From mboxrd@z Thu Jan 1 00:00:00 1970 From: Brian Burch Date: Wed, 08 Jan 2014 18:52:26 +0000 Subject: Re: How to override the default source ipv4 address on packets originating from a router Message-Id: <52CD9E6A.9030205@pingtoo.com> List-Id: References: <52B43271.7040807@PingToo.com> In-Reply-To: <52B43271.7040807@PingToo.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: lartc@vger.kernel.org On 08/01/14 17:19, Joel Gerber wrote: > You're going to have to re-acquaint me with your network layout. I'm having a hard time remembering what all of your interfaces are being used for and how they fit into your network. Oh! Sorry, Joel. I forgot this is a simple mailing list, without a cute threaded web interface. I "helpfully" snipped out our initial discussion to lighten the payload... I will resend my last post within the entire reply, then you will have everything in one place. Discard this one and the "full" version will follow soon. Thanks, Brian > Joel Gerber > Network Specialist > Network Operations > Eastlink > E: Joel.Gerber@corp.eastlink.ca T: 519.786.1241 > > -----Original Message----- > From: lartc-owner@vger.kernel.org [mailto:lartc-owner@vger.kernel.org] On Behalf Of Brian Burch > Sent: January-08-14 11:18 AM > To: Linux Advanced Routing List > Subject: Re: How to override the default source ipv4 address on packets originating from a router > > On 21/12/13 12:53, Brian Burch wrote: >> On 20/12/13 18:46, Brian Burch wrote: >>> On 20/12/13 13:05, Joel Gerber wrote: >>>> Your problem is one I've come up against a number of times in the >>>> past. You are definitely on the right route using "policy routing" >>>> (otherwise known as utilizing IP Rules), but the snag you're hitting >>>> is that the source address of a locally generated packet isn't >>>> chosen until a destination route has already been selected. Using >>>> the from statement in an ip rules table is only useful when you're >>>> routing packets from other subnets. >>> >>> What a quick and helpful reply! Not only did my problem motivate you >>> to respond, but I am very relieved to find the answer isn't just RTFM. >>> >>> Your explanation about the timing for kernel selection of the source >>> address is crucial and now my failed experiments all make sense. Your >>> suggested solution also makes sense, although it is counter-intuitive... >>> at least to me! >>> >>>> Now, just so you know, the local table should normally be left alone. >>>> It's used for locally connected subnets only. It is given the >>>> highest priority on purpose... don't mess with it! :) >>> >>> I read somewhere (possibly the howto guide or the ip-cref) a >>> statement that the kernel treats this table as read-only. My system >>> is running ubuntu 13.04 with a custom non-pae 3.8.0 kernel, so it is >>> reasonably modern. When I thought more about the purpose of this >>> table, I decided there would be no point in trying to modify it, even >>> if that was possible. >>> >>>> There are a few methods you could try here, but I'm only going to >>>> recommend one of them. Have your main routing table hold the routes >>>> that your local machine will use, and then have a separate table for >>>> all routed hosts. Then, use ip rule statements to force any traffic >>>> coming in from a routed host into that routing table. >>>> >>>> IE: on firewall-router (assuming eth1 is facing your static subnet >>>> hosts): >>>> >>>> # ip rule list >>>> 0: from all lookup local >>>> 32765: iif eth1 lookup static-hosts >>>> 32766: from all lookup main >>>> 32767: from all lookup default >>>> >>>> Have table main include all of the routes your local firewall-router >>>> should use, and have static-hosts contain all of the routes that >>>> your static-subnet-hosts should use. >>> >>> Great! I understand the principle you propose and will try to >>> implement it over the weekend. I will update this thread when I have >>> something to report. >> >> Excellent news!! My router/firewall is currently collecting 200+ >> updated packages from the repository server! >> >> The solution was achieved using ONLY policy routing, based on Joel's >> suggested approach. >> >> However, I hit some fairly nasty snags along the way and had to do a >> lot of trial and error fiddling to make it work. I am certain my >> working solution can be made more elegant, but that will take a couple >> of hours pen-and-pencil time to map out the current routing table >> entries in selection order and then perform some experiments. >> >> I don't have time to do it at the moment, but will get started over >> the next couple of days. If I get stuck, I will post the "current best" >> configuration and ask for help to improve it. > > > > I tried to improve my "current best" definitions, but my efforts to remove apparent redundancy have all failed. Here is the setup: > > brian@chenin:~$ ip address show > 1: lo: mtu 65536 qdisc noqueue state UNKNOWN > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 > inet 127.0.0.1/8 scope host lo > inet6 ::1/128 scope host > valid_lft forever preferred_lft forever > 2: eth1: mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000 > link/ether 00:15:8a:01:91:e3 brd ff:ff:ff:ff:ff:ff > inet 217.154.193.209/28 brd 217.154.193.223 scope global eth1 > inet6 fe80::215:8aff:fe01:91e3/64 scope link > valid_lft forever preferred_lft forever > 3: eth0: mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000 > link/ether 00:40:63:f5:a5:72 brd ff:ff:ff:ff:ff:ff > inet 172.16.101.2/24 brd 172.16.101.255 scope global eth0 > inet6 fe80::240:63ff:fef5:a572/64 scope link > valid_lft forever preferred_lft forever > > > brian@chenin:~$ ip rule list > 0: from all lookup local > 32764: from 217.154.193.209 lookup CHENIN_TO_INTERNET > 32765: from all iif eth1 lookup SUBNET_217 > 32766: from all lookup main > 32767: from all lookup default > > > brian@chenin:~$ ip route list table CHENIN_TO_INTERNET > 163.1.221.67 via 172.16.101.1 dev eth0 > > > brian@chenin:~$ ip route list table SUBNET_217 default via 172.16.101.1 dev eth0 > > > brian@chenin:~$ ip route list table main > 163.1.221.67 dev eth0 scope link src 217.154.193.209 > 169.254.0.0/16 dev eth0 scope link metric 1000 > 172.16.101.0/24 dev eth0 proto kernel scope link src 172.16.101.2 > 217.154.193.208/28 dev eth1 proto kernel scope link src 217.154.193.209 > > > I know Joel's suggestion did not include my CHENIN_TO_INTERNET rule, but without it chenin simply times out on its http gets. > > I also tried keeping CHENIN_TO_INTERNET, and removing the explicit route in the main table, but that fails too, although I wasn't very surprised. > > I have tried trimming these rules and routes to discover the minimum requirement, but this seems to be the smallest working subset. > > > Here is my theory to account for what is happening: > > 1. The SUBNET_217 default route MUST be able to handle ANY traffic > arriving on eth1. When I changed the rule to match on just > 217.154.193.208/28, the forwarded traffic became very glitchy, with poor > response times and some dropped tcp sessions. > > 2. The route to 163.1.221.67 in the main table is required to assign the > eth1 source address 217.154.193.209 to its outbound packets. This works > because the kernel cannot find any other matching routes from chenin to > 163.1.221.67 (which might encourage it to assign eth0's 172.16.101.2 as > the source address). > > 4. Having assigned the 217.154.193.209 source address, the packet must > be routed to the next hop. The CHENIN_TO_INTERNET rule matches the > explicit source host address at a higher priority than the SUBNET_217 > rule, so 172.16.101 is selected. > > 5. The main table has a route to 172.16.101.1 via eth0, so the packet is > sent on its way. > > > That seems to be three passes through the policy routing tables. Am I > correct? > > I still feel it should be possible to improve the solution, but I don't > understand what is going on well enough to see how. I would be very > grateful if anyone can enlighten me. > > Thanks, > > Brian > -- > To unsubscribe from this list: send the line "unsubscribe lartc" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe lartc" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html >