Prioritizing IPs on interface with multiple addresses

Prioritizing IPs on interface with multiple addresses

[Kristian Evensen]

Hello,

I am currently working on configuring an embedded system that will be
used as an access point for other devices. In order to reduce the
number of external devices, we want to connect the LAN on the embedded
system to a switch. Then, two separate networks will also be connected
to this switch, and the LAN interface assigned one IP from each
network. The IPs are static and network addresses are not overlapping.
One interface is the main interface for all traffic from clients,
while the other is used as fallback and for some monitoring traffic.
The clients that connect to this AP will be assigned IPs using DHCP
and traffic from them will be NAT'ed.

Initially, this setup works fine. The devices connected to this AP are
able to send traffic through the intended network and to the correct
hosts. If I disconnect from the main network, the routing tables are
updated and traffic is routed through the secondary network. However,
when I connect to the main network again, things break. The problem is
that there is an inconsistency between the order in the routing table
and the order of IP addresses assigned to the interface, which causes
problems when I do NAT (MASQUERADE). The default route (with the
lowest metric) points to the main network, but the first IP address
belongs to the secondary network. So what happens is that the packets
have the MAC-address of the first hop in the main network, but a
source IP address from the second network (chosen by the NAT). This
causes the traffic to be discarded by the network. Deleting (and then
later adding) the IP of the secondary network is not an option as it
is needed for the monitoring traffic.

My question is, is there some way to prioritize the different IP
addresses assigned to an interface? For example, is there an
equivalent to a metric, index or something similar? Based on my
understanding, ip addr is only able to append addresses. I believe
this would solve the problem, as it would then be possible to ensure
that addresses and the routing table are in sync.

Another soulution would be to monitor network events and create/delete
SNAT rules on-demand, but this is a big hack if you ask me and I would
like to try to avoid it.

Thanks in advance for any help!
Kristian

Re: Prioritizing IPs on interface with multiple addresses

[Andrew Beverley]

On Mon, 2012-10-22 at 11:36 +0200, Kristian Evensen wrote:
> Hello,
> 
> I am currently working on configuring an embedded system that will be
> used as an access point for other devices. In order to reduce the
> number of external devices, we want to connect the LAN on the embedded
> system to a switch. Then, two separate networks will also be connected
> to this switch, and the LAN interface assigned one IP from each
> network. The IPs are static and network addresses are not overlapping.
> One interface is the main interface for all traffic from clients,
> while the other is used as fallback and for some monitoring traffic.
> The clients that connect to this AP will be assigned IPs using DHCP
> and traffic from them will be NAT'ed.
> 
> Initially, this setup works fine. The devices connected to this AP are
> able to send traffic through the intended network and to the correct
> hosts. If I disconnect from the main network, the routing tables are
> updated and traffic is routed through the secondary network. However,
> when I connect to the main network again, things break. The problem is
> that there is an inconsistency between the order in the routing table
> and the order of IP addresses assigned to the interface, which causes
> problems when I do NAT (MASQUERADE). The default route (with the
> lowest metric) points to the main network, but the first IP address
> belongs to the secondary network. So what happens is that the packets
> have the MAC-address of the first hop in the main network, but a
> source IP address from the second network (chosen by the NAT). This
> causes the traffic to be discarded by the network. Deleting (and then
> later adding) the IP of the secondary network is not an option as it
> is needed for the monitoring traffic.

I have to admit that I'm struggling to get my head round this, and I
suspect others are as well given lack of replies. Could you provide an
ascii diagram and either write more succinctly or try and simplify the
problem you are having? 

> My question is, is there some way to prioritize the different IP
> addresses assigned to an interface? For example, is there an
> equivalent to a metric, index or something similar?

That said, I don't know if there is any way of doing this. Can you not
achieve it with iptables rules and SNAT?

>  Based on my
> understanding, ip addr is only able to append addresses.

Well it can delete them as well, or have I misunderstood?

> Another soulution would be to monitor network events and create/delete
> SNAT rules on-demand, but this is a big hack if you ask me and I would
> like to try to avoid it.

Ah, you've already thought of SNAT. Is there not a way of doing it
without adding and deleting rules? For example can you use packet
marking somehow?

Andy