NAT, not doing route_me_harder?

NAT, not doing route_me_harder?

[Balazs Scheidler]

Hi,

I was wondering what the reason is for NAT not rerouting modified packets?

If anything important is modified by a mangle rule that affects routing, the
routing decision is automatically redone as this code fragment shows:

        ret = ipt_do_table(pskb, hook, in, out, &packet_mangler, NULL);
        /* Reroute for ANY change. */
        if (ret != NF_DROP && ret != NF_STOLEN && ret != NF_QUEUE
            && ((*pskb)->nh.iph->saddr != saddr
                || (*pskb)->nh.iph->daddr != daddr
                || (*pskb)->nfmark != nfmark
                || (*pskb)->nh.iph->tos != tos))
                return ip_route_me_harder(pskb) == 0 ? ret : NF_DROP;

NAT doesn't do anything like this. So given an SNAT rule changes the source
address in POSTROUTING, the routing tables are not looked up again, so
source address dependant policy routing rules are not applied.

It might not be the best to change this by default, but it could be
implemented by a match, e.g.

iptables -t nat -A POSTROUTING -p tcp -d 0/0 --dport 25 -m reroute -j SNAT --to-source 1.2.3.4

-m reroute would flag the packet as one which needs rerouting (using for
example a flag in nfcache). Packets flagged as such would be rerouted after
do_bindings() is called.

Opinions?

-- 
Bazsi
PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1

Re: NAT, not doing route_me_harder?

[Henrik Nordstrom]

Balazs Scheidler wrote:
> Hi,
>
> I was wondering what the reason is for NAT not rerouting modified packets?
>
> If anything important is modified by a mangle rule that affects routing,
> the routing decision is automatically redone as this code fragment shows:

[snip]

This is done only in the OUTPUT chain, and only because the TCP kernel has 
already routed locally originating packets before they first hit netfilter.

> NAT doesn't do anything like this. So given an SNAT rule changes the source
> address in POSTROUTING, the routing tables are not looked up again, so
> source address dependant policy routing rules are not applied.

It sure does, in the same spot as mangle, which only is when there is a 
destnination nat transformations applied to a locally originated packet.

in ip_nat_local_fn():
        ret = ip_nat_fn(hooknum, pskb, in, out, okfn);
        if (ret != NF_DROP && ret != NF_STOLEN
            && ((*pskb)->nh.iph->saddr != saddr
                || (*pskb)->nh.iph->daddr != daddr))
                return ip_route_me_harder(pskb) == 0 ? ret : NF_DROP;

For the other cases (including mangle), all the transformations that are 
assumed to affect routing is done in PREROUTING. SNAT is not among them.

There is a number of ways to route SNAT:ed packets differently if needed. The 
method I use is usually to use the nfmark of mangle PREROUTING or OUTPUT in 
combination with SNAT in POSTROUTING. Mangle marks the packet telling that 
this should be NAT:ed according to policy X, this nfmark is then used in 
routing to route the packet in the correct direction and by nat POSTROUTING 
to apply the correct NAT rule.

Regards
Henrik

Re: NAT, not doing route_me_harder?

[Balazs Scheidler]

> This is done only in the OUTPUT chain, and only because the TCP kernel has 
> already routed locally originating packets before they first hit netfilter.
> 
> > NAT doesn't do anything like this. So given an SNAT rule changes the source
> > address in POSTROUTING, the routing tables are not looked up again, so
> > source address dependant policy routing rules are not applied.
> 
> It sure does, in the same spot as mangle, which only is when there is a 
> destnination nat transformations applied to a locally originated packet.
> 
> in ip_nat_local_fn():
>         ret = ip_nat_fn(hooknum, pskb, in, out, okfn);
>         if (ret != NF_DROP && ret != NF_STOLEN
>             && ((*pskb)->nh.iph->saddr != saddr
>                 || (*pskb)->nh.iph->daddr != daddr))
>                 return ip_route_me_harder(pskb) == 0 ? ret : NF_DROP;
> 
> For the other cases (including mangle), all the transformations that are 
> assumed to affect routing is done in PREROUTING. SNAT is not among them.
> 
> There is a number of ways to route SNAT:ed packets differently if needed. The 
> method I use is usually to use the nfmark of mangle PREROUTING or OUTPUT in 
> combination with SNAT in POSTROUTING. Mangle marks the packet telling that 
> this should be NAT:ed according to policy X, this nfmark is then used in 
> routing to route the packet in the correct direction and by nat POSTROUTING 
> to apply the correct NAT rule.

But what happens when you initiate a connection on the host running
netfilter, thus you have no PREROUTING chain?

Scenario:

I have a default route to gateway A on interface a, but I want my SMTP
traffic to leave the box on a different interface b with a different gateway
B. Of course this means a different source address is to be assigned to the
outgoing connection. Assume it is not possible to set the bind address of
the MTA (or setting it affects mail delivery in other directions).

I have source based routing, that makes packets go to the correct direction
based on their source address.

If I'm doing SNAT in POSTROUTING, the routing decision is not redone, thus
it leaves with the specified source address, but on the wrong interface.

I think I now understand, have my packets marked in local OUTPUT, route
based on that mark, and SNAT based on the marks. Is this the way you
suggested? Hmm.. this sounds reasonable on the programmer's perspective, but
is difficult to maintain from the user's: it needs two rules.

iptables -t mangle -A OUTPUT -p tcp ! -s <intranet> -d 0/0 --dport 25 -j MARK --set-mark 100
iptables -t nat -A POSTROUTING -m mark --mark 100 -j SNAT --to-source <b ip address>

instead of:

iptables -t nat -A POSTROUTING -p tcp ! -s <intranet> -d 0/0 --dport 25 -m reroute -j SNAT --to-source <b ipaddress>

Hmm... as I think some more, the 2nd case might not even be possible, as the
nat rule is triggered only once during a session, and it would mean that syn
would be routed correctly, but packets following it would not. Solution:
instead of nfcache the flag could be stored in ip_conntrack.

-- 
Bazsi
PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1

Re: NAT, not doing route_me_harder?

[Henrik Nordstrom]

Balazs Scheidler wrote:

> But what happens when you initiate a connection on the host running
> netfilter, thus you have no PREROUTING chain?

You have the OUTPUT chain.

> If I'm doing SNAT in POSTROUTING, the routing decision is not redone, thus
> it leaves with the specified source address, but on the wrong interface.

See my previous reply.

> I think I now understand, have my packets marked in local OUTPUT, route
> based on that mark, and SNAT based on the marks. Is this the way you
> suggested? Hmm.. this sounds reasonable on the programmer's perspective,
> but is difficult to maintain from the user's: it needs two rules.

Yes, it requires three custom rules rather than two (there is also the routing 
policy rule)

Having NAT reroute all packets due to source nat transformations would be a 
significant performance impact only to support the corner cases where it is 
handy..

Regards
Henrik

Re: NAT, not doing route_me_harder?

[Balazs Scheidler]

On Wed, Jun 26, 2002 at 12:04:23PM +0200, Henrik Nordstrom wrote:
> Balazs Scheidler wrote:
> > I think I now understand, have my packets marked in local OUTPUT, route
> > based on that mark, and SNAT based on the marks. Is this the way you
> > suggested? Hmm.. this sounds reasonable on the programmer's perspective,
> > but is difficult to maintain from the user's: it needs two rules.
> 
> Yes, it requires three custom rules rather than two (there is also the routing 
> policy rule)
> 
> Having NAT reroute all packets due to source nat transformations would be a 
> significant performance impact only to support the corner cases where it is 
> handy..

Why? The rerouting would be triggered only if the user requests it, so
normal path would not be affected. And as routing decisions are heavily
cached, it is said (I think it was Harald who said that) that routing
decisions are not expensive. It would add a simple bit-test in normal path,
and a second routing decision if explicitly requested:

something like this in ip_nat_fn(), after do_bindings is called:

	saddr = (*pskb)->nh.iph->saddr;
	daddr = (*pskb)->nh.iph->daddr;

	ret = do_bindings(ct, ctinfo, info, hooknum, pskb);
	if (ret != NF_DROP && ret != NF_STOLEN && (ct->flags & IP_NAT_REROUTE)) {
	        if (((*pskb)->nh.iph->saddr != saddr || (*pskb)->nh.iph->daddr != daddr))
	                ret = (ip_route_me_harder(pskb) == 0) ? ret : NF_DROP;
	
	}
	return ret;

This could also be extended with the local output case, so ip_nat_fn() and
ip_nat_local_fn() could be merged. (the if condition would become:

if (ret != NF_DROP && ret != NF_STOLEN && (hooknum == NF_IP_LOCAL_OUT || ct->flags & IP_NAT_REROUTE)) {
	...
}


-- 
Bazsi
PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1