Re: Wrong routing when combining ip rule with SNAT

[Vigneswaran R <vignesh@atc.tcs.com>]

On 09/14/2013 09:10 PM, Nikolaus Rath wrote:
> On 09/14/2013 06:41 AM, Pascal Hambourg wrote:
>> Vigneswaran R a écrit :
>>> Hello Nikolaus,
>>>
>>> I have a doubt. It seems, rath of ebox is assigned with IP address in
>>> the range 192.168.12.0/24. However, IP address of vostro seems to be
>>> 192.168.17.47 (assuming /24). Ebox doesn't have any route to this range.
>>> So it try to use default route via eth0.
>> Correct.
>>
>>> What I assume is, 'vostro' has IP addresses in (atleast) two ranges
>>> (192.168.12.0/24, 192.168.17.0/24). In the default routing table, the
>>> src IP is set to 192.168.12.x (for the packets originating from vostro).
>>> However, the 'tovpn' table didn't specify the src IP. So, when the
>>> 'tovpn' table is being used, the packets may have got the src IP as
>>> 192.168.17.x.
>>>
>>> I think, you can avoid this by explicitly specifying the src IP when
>>> adding the route to 'tovpn' table,
>>>
>>>       ip route add default via 192.168.12.1 src 192.168.12.x table tovpn
>> This won't work. It's too late. The source address has already been
>> selected by the TCP layer when the packet was created and won't be
>> changed when the packet is re-routed due to the mark.
> I see. Out of curiosity: how is the source address selected when the
> packet is created, and in which situation would the source entry in the
> routing table become effective?

Following is my understanding.

For a packet created by local process, it seems, there are two places in 
which the IP Rules have been evaluated and various routing tables have 
been consulted.

1. Just before getting into OUTPUT chain
   -- Here, the outgoing Interface, SRC IP have been decided for the packet

2. After processing OUTPUT chain and just before getting into 
POSTROUTING chain
   -- Here, the outgoing Interface is getting modified, if necessary. 
However, the SRC IP didn't.

So, in our case,

1. Just before getting into OUTPUT chain
   -- At this point, the IP Rule (based on FWMARK) doesn't evaluate to 
true as the MARK is not yet set.
   -- Due to that 'tovpn' table is not consulted
   -- This means, there is no explicit route to 190.93.249.164. So, it 
uses default route 'via 192.168.17.1 dev br0' (based on main table entry).
   -- This results in SRC IP being 192.168.17.47 and the outgoing 
Interface being 'br0'.

2. After processing OUTPUT chain and just before getting into 
POSTROUTING chain
   -- At this point, the IP Rule (based on FWMARK) evaluates to true as 
the MARK is added by OUTPUT chain.
   -- Now, the 'tovpn' table is also consulted which has route to 
190.93.249.164 'via 192.168.12.1 dev rath'.
   -- In my observation, only the outgoing Interface is updated at this 
point.
   -- This results in SRC IP being 192.168.17.47 and the outgoing 
Interface being 'rath'.

The following workaround solves our purpose. However ugly and not 
scalable, I think.

Using some condition for IP Rule which evaluates to true at the 1st step 
itself. eg., (matching destination IP).

     ip rule add to 190.93.249.164 table tovpn


Regards,
Vignesh