Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Henrik Nordstrom]

---------- Forwarded message ----------
Date: Wed, 14 Jan 2004 23:50:44 +0100 (CET)
From: Henrik Nordstrom <hno@marasystems.com>
To: Rusty Russell <rusty@rustcorp.com.au>
Cc: Harald Welte <laforge@netfilter.org>
Subject: Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's
    brain broke!) 

On Wed, 14 Jan 2004, Rusty Russell wrote:

> > And there are lots of users that have multiple DSL-dynip links these
> > days, trying to statically or dynamically balance web requests between
> > them, etc.
> 
> In that case, the interfaces are different, (ppp0 vs ppp1) so no
> problem.  You need something more complex to trigger the problem
> AFAICT.

Consider a firewall connected with ethernet to multiple DSL links not
using PPP-over-ethernet.

But i can not say I consider such setup sane in the DSL-dynip case, nor
how to overcome such trivial issues such as address assignment as such
setups generally use DHCP over the modem to assign the address and it is a
little hard to direct the DHCP request to the correct modem over a
broadcast ethernet.

A better case is the case of having multiple links from the same ethernet 
and wanting to SNAT based on the route taken. Masquerade with route lookup 
does this kind of magically by the route local source address field.

Hmm.. doesn't the SKB already have the route information by the time of
POSTROUTING? If so, why not simply use the route information already
there? MASQUERADE should never change the route taken, only the source
address of the packet so there should not be a need for an additional 
route lookup, and as the packet is already routed there should be no need 
to dig into the interface to find addressing informations..

I.e. why not use something like the following to find the source address 
in MASQUERADE:

    ((struct rtable*)skb->dst)->rt_src

Only worry is if there is POSTROUTING cases where skb->dst is not an
rtable (superset of dst_entry).

Regards
Henrik

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Patrick McHardy]

Henrik Nordstrom wrote:
> I.e. why not use something like the following to find the source address 
> in MASQUERADE:
> 
>     ((struct rtable*)skb->dst)->rt_src

AFAICS in case of a input routed packet rt_src is the source ip of the
packet.

Best regards,
Patrick


> 
> Only worry is if there is POSTROUTING cases where skb->dst is not an
> rtable (superset of dst_entry).
> 
> Regards
> Henrik
> 

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Henrik Nordstrom]

On Thu, 15 Jan 2004, Patrick McHardy wrote:

> Henrik Nordstrom wrote:
> > I.e. why not use something like the following to find the source address 
> > in MASQUERADE:
> > 
> >     ((struct rtable*)skb->dst)->rt_src
> 
> AFAICS in case of a input routed packet rt_src is the source ip of the
> packet.

MASQUERADE is never called in INPUT.

Regards
Henrik

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Patrick McHardy]

Henrik Nordstrom wrote:
> 
> MASQUERADE is never called in INPUT.

That is true, but if you want to save the routing call in MASQUERADE all
you've got is the route that was determined at input routing (for
non-local packets).

Regards,
Patrick


> 
> Regards
> Henrik
> 

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Henrik Nordstrom]

On Thu, 15 Jan 2004, Patrick McHardy wrote:

> That is true, but if you want to save the routing call in MASQUERADE all
> you've got is the route that was determined at input routing (for
> non-local packets).

Right.. so that shortcut can not be taken.

Regards
Henrik

MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 1790 bytes --]

On Fri, Jul 25, 2003 at 09:56:57PM +0400, kuznet@ms2.inr.ac.ru wrote:

> Hello!

Hi Alexey,  I have to follow-up on this old thraed.

> > Hmm, what's your routing setup?  And what kernel?  It's possible with
> > wierd setups, like source routing.
> 
> Unlikely, source address is unspecified here. Most likely, it is fwmark.
> 
> Unrelated: giving out->ifindex is a bug, by the way. It can screw up
> the things a lot. In this context, if you want to be sure that packet
> will go out expected interface you do plain lookup and drop packet
> if it gave you some strange route.

Your proposed change (key.oif = 0 instead of out->ifindex) went into
2.4.23, and we've received a number of bug reports like
https://bugzilla.netfilter.org/cgi-bin/bugzilla/show_bug.cgi?id=144
http://www.ussg.iu.edu/hypermail/linux/kernel/0312.0/0465.html
http://www.ussg.iu.edu/hypermail/linux/kernel/0312.0/0408.html

This means that ip_route_output_key() returns a route with a different
outgoing interface than the skb->dst->dev of our to-be-masqueraded
packet.  

Why was it wrong to specify skb->dst->dev->ifindex of the previous
'real' routing decision as key to our current routing decision?

As an example case where I would suspect problems: The packet could be
coming from a local socket, and the socket be bound to a specific
interface (sk->bound_dev_if).

Please comment, thanks.

> Alexey

-- 
- Harald Welte <laforge@netfilter.org>             http://www.netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Julian Anastasov]

	Hello,

On Sun, 11 Jan 2004, Harald Welte wrote:

> As an example case where I would suspect problems: The packet could be
> coming from a local socket, and the socket be bound to a specific
> interface (sk->bound_dev_if).

	IMO, the real example is that the people use multipath routes
and providing oif was the only way MASQUERADE to meet the netfilter
and firewalling expectations of not changing the output device during
hooks. OTOH, providing oif=0 is the valid approach for selecting the
right route but as long as the above expectation exists there are
two options for the users:

- provide oif learned from the input route (as before the discussed
change). May be in 99% of the setups it selects the right route.
I think, we should use this, at least for 2.4.

- use CONNMARK or similar functionality to keep the connection
bound to its path. As long as CONNMARK is not a standard feature
there is no safe way to use multipath routes with MASQUERADE and
SNAT in the latest kernels. Even before this change it was risky
to rely on the routing cache to keep NAT connections bound to
its path in the multipath route - the cache entries expire.

Regards

--
Julian Anastasov <ja@ssi.bg>

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Henrik Nordstrom]

On Sun, 11 Jan 2004, Julian Anastasov wrote:

> - use CONNMARK or similar functionality to keep the connection
> bound to its path. As long as CONNMARK is not a standard feature
> there is no safe way to use multipath routes with MASQUERADE and
> SNAT in the latest kernels. Even before this change it was risky
> to rely on the routing cache to keep NAT connections bound to
> its path in the multipath route - the cache entries expire.

As the aurhor of CONNMARK I certainly do not mind having this progress 
beyond path-o-matic extras..

While it was invented to solve a special-case issue, it has over the years 
found many additional and more general uses. Today it is in use for

a) Multihomed setups of a shared network with limited routing tables

b) Reliable and easy to understand multipath+NAT routing.

c) Interception routing without NAT, routing specific TCP sessions
(including RELATED and ICMP messages) a special path, usually for 
interception caching outside of the router without loss of addressing 
information.

and probably several other applications I do not know about or simply 
forgotten.

Regards
Henrik

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Patrick McHardy]

Julian Anastasov wrote:

>- provide oif learned from the input route (as before the discussed
>change). May be in 99% of the setups it selects the right route.
>I think, we should use this, at least for 2.4.
>
>  
>
Why should we do a route lookup at all ? MASQUERADE doesn't need the
dst_entry but only the interface address. Using ifa_list->ifa_local
of the outgoing in_device seems like the simplest solution to me.

Regards,
Patrick

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Rusty Russell]

In message <4001F33A.9000802@trash.net> you write:
> Julian Anastasov wrote:
> 
> >- provide oif learned from the input route (as before the discussed
> >change). May be in 99% of the setups it selects the right route.
> >I think, we should use this, at least for 2.4.
> >
> >  
> >
> Why should we do a route lookup at all ? MASQUERADE doesn't need the
> dst_entry but only the interface address. Using ifa_list->ifa_local
> of the outgoing in_device seems like the simplest solution to me.

You take all the fun out.

Yes, this is the best.  It always does *something*, and is
predictable.  People with really complex routing shouldn't really use
MASQUERADE, since it's designed for a specific, simple case.

Patch?

Cheers,
Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Patrick McHardy]

[-- Attachment #1: Type: text/plain, Size: 561 bytes --]

Rusty Russell wrote:
> You take all the fun out.
> 
> Yes, this is the best.  It always does *something*, and is
> predictable.  People with really complex routing shouldn't really use
> MASQUERADE, since it's designed for a specific, simple case.

I'm not sure about this, as Julian mentioned it breaks some setups,
but that's your decision.

> 
> Patch?

Patches for 2.4+2.6 attached. Both patches compile, 2.6 version is also
tested.

Best regards,
Patrick

> 
> Cheers,
> Rusty.
> --
>   Anyone who quotes me in their sig is an idiot. -- Rusty Russell.
> 


[-- Attachment #2: 2.4-masq-ifa_local.diff --]
[-- Type: text/plain, Size: 2543 bytes --]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/01/13 05:05:13+01:00 kaber@trash.net 
#   Use indev->ifa_list->ifa_local for MASQUERADE
# 
# net/ipv4/netfilter/ipt_MASQUERADE.c
#   2004/01/13 05:05:11+01:00 kaber@trash.net +16 -24
#   Use indev->ifa_list->ifa_local for MASQUERADE
# 
diff -Nru a/net/ipv4/netfilter/ipt_MASQUERADE.c b/net/ipv4/netfilter/ipt_MASQUERADE.c
--- a/net/ipv4/netfilter/ipt_MASQUERADE.c	Tue Jan 13 05:09:32 2004
+++ b/net/ipv4/netfilter/ipt_MASQUERADE.c	Tue Jan 13 05:09:32 2004
@@ -1,10 +1,13 @@
-/* Masquerade.  Simple mapping which alters range to a local IP address
-   (depending on route). */
+/* Masquerade.  Simple mapping which alters range to a local IP address. */
+#include <linux/kernel.h>
 #include <linux/config.h>
 #include <linux/types.h>
 #include <linux/ip.h>
 #include <linux/timer.h>
 #include <linux/module.h>
+#include <linux/skbuff.h>
+#include <linux/netdevice.h>
+#include <linux/inetdevice.h>
 #include <linux/netfilter.h>
 #include <net/protocol.h>
 #include <net/checksum.h>
@@ -67,9 +70,8 @@
 	enum ip_conntrack_info ctinfo;
 	const struct ip_nat_multi_range *mr;
 	struct ip_nat_multi_range newrange;
+	struct in_device *indev;
 	u_int32_t newsrc;
-	struct rtable *rt;
-	struct rt_key key;
 
 	IP_NF_ASSERT(hooknum == NF_IP_POST_ROUTING);
 
@@ -84,30 +86,20 @@
 
 	mr = targinfo;
 
-	key.dst = (*pskb)->nh.iph->daddr;
-	key.src = 0; /* Unknown: that's what we're trying to establish */
-	key.tos = RT_TOS((*pskb)->nh.iph->tos)|RTO_CONN;
-	key.oif = 0;
-#ifdef CONFIG_IP_ROUTE_FWMARK
-	key.fwmark = (*pskb)->nfmark;
-#endif
-	if (ip_route_output_key(&rt, &key) != 0) {
-                /* Funky routing can do this. */
-                if (net_ratelimit())
-                        printk("MASQUERADE:"
-                               " No route: Rusty's brain broke!\n");
-                return NF_DROP;
-        }
-        if (rt->u.dst.dev != out) {
-                if (net_ratelimit())
-                        printk("MASQUERADE:"
-                               " Route sent us somewhere else.\n");
+	indev = in_dev_get(out);
+	if (indev == NULL)
+		return NF_DROP;
+	read_lock(&indev->lock);
+	if (indev->ifa_list == NULL) {
+		read_unlock(&indev->lock);
+		in_dev_put(indev);
 		return NF_DROP;
 	}
+	newsrc = indev->ifa_list->ifa_local;
+	read_unlock(&indev->lock);
+	in_dev_put(indev);
 
-	newsrc = rt->rt_src;
 	DEBUGP("newsrc = %u.%u.%u.%u\n", NIPQUAD(newsrc));
-	ip_rt_put(rt);
 
 	WRITE_LOCK(&masq_lock);
 	ct->nat.masq_index = out->ifindex;

[-- Attachment #3: 2.6-masq-ifa_local.diff --]
[-- Type: text/plain, Size: 2415 bytes --]

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/01/13 05:08:46+01:00 kaber@trash.net 
#   Use indev->ifa_list->ifa_local for MASQUERADE
# 
# net/ipv4/netfilter/ipt_MASQUERADE.c
#   2004/01/13 05:05:49+01:00 kaber@trash.net +18 -28
#   Use indev->ifa_list->ifa_local for MASQUERADE
# 
diff -Nru a/net/ipv4/netfilter/ipt_MASQUERADE.c b/net/ipv4/netfilter/ipt_MASQUERADE.c
--- a/net/ipv4/netfilter/ipt_MASQUERADE.c	Tue Jan 13 05:09:17 2004
+++ b/net/ipv4/netfilter/ipt_MASQUERADE.c	Tue Jan 13 05:09:17 2004
@@ -1,10 +1,13 @@
-/* Masquerade.  Simple mapping which alters range to a local IP address
-   (depending on route). */
+/* Masquerade.  Simple mapping which alters range to a local IP address. */
+#include <linux/kernel.h>
 #include <linux/config.h>
 #include <linux/types.h>
 #include <linux/ip.h>
 #include <linux/timer.h>
 #include <linux/module.h>
+#include <linux/skbuff.h>
+#include <linux/netdevice.h>
+#include <linux/inetdevice.h>
 #include <linux/netfilter.h>
 #include <net/protocol.h>
 #include <net/checksum.h>
@@ -71,8 +74,8 @@
 	enum ip_conntrack_info ctinfo;
 	const struct ip_nat_multi_range *mr;
 	struct ip_nat_multi_range newrange;
+	struct in_device *indev;
 	u_int32_t newsrc;
-	struct rtable *rt;
 
 	IP_NF_ASSERT(hooknum == NF_IP_POST_ROUTING);
 
@@ -87,33 +90,20 @@
 
 	mr = targinfo;
 
-	{
-		struct flowi fl = { .nl_u = { .ip4_u =
-					      { .daddr = (*pskb)->nh.iph->daddr,
-						.tos = (RT_TOS((*pskb)->nh.iph->tos) |
-							RTO_CONN),
-#ifdef CONFIG_IP_ROUTE_FWMARK
-						.fwmark = (*pskb)->nfmark
-#endif
-					      } } };
-		if (ip_route_output_key(&rt, &fl) != 0) {
-			/* Funky routing can do this. */
-			if (net_ratelimit())
-				printk("MASQUERADE:"
-				       " No route: Rusty's brain broke!\n");
-			return NF_DROP;
-		}
-		if (rt->u.dst.dev != out) {
-			if (net_ratelimit())
-				printk("MASQUERADE:"
-				       " Route sent us somewhere else.\n");
-			return NF_DROP;
-		}
-	}
+	indev = in_dev_get(out);
+	if (indev == NULL)
+		return NF_DROP;
+	read_lock(&indev->lock);
+	if (indev->ifa_list == NULL) {
+		read_unlock(&indev->lock);
+		in_dev_put(indev);
+		return NF_DROP;
+	}
+	newsrc = indev->ifa_list->ifa_local;
+	read_unlock(&indev->lock);
+	in_dev_put(indev);
 
-	newsrc = rt->rt_src;
 	DEBUGP("newsrc = %u.%u.%u.%u\n", NIPQUAD(newsrc));
-	ip_rt_put(rt);
 
 	WRITE_LOCK(&masq_lock);
 	ct->nat.masq_index = out->ifindex;

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Julian Anastasov]

	Hello,

On Mon, 12 Jan 2004, Rusty Russell wrote:

> > Why should we do a route lookup at all ? MASQUERADE doesn't need the
> > dst_entry but only the interface address. Using ifa_list->ifa_local
> > of the outgoing in_device seems like the simplest solution to me.
>
> You take all the fun out.

	Selecting link local addresses is not always a fun :)
The IP addresses are sorted by their scope.

> Yes, this is the best.  It always does *something*, and is
> predictable.  People with really complex routing shouldn't really use
> MASQUERADE, since it's designed for a specific, simple case.

	Don't do that for 2.4, the users will not be happy.
There are setups with non-default multipath routes, multipath
routes with nexthops sharing same outdev, WAN interfaces which
have their private link layer IPs at first position.

	MASQUERADE is better for some cases where SNAT can
not be used at all: using source address autoselection based
on the scope and the nexthop's GW IP.

	I vote for backing out the oif change and to return
to the previous state - the routing is still smarter than any
iptables rules.

> Cheers,
> Rusty.

Regards

--
Julian Anastasov <ja@ssi.bg>

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 1783 bytes --]

On Mon, Jan 12, 2004 at 03:30:14PM +1100, Rusty Russell wrote:
> In message <4001F33A.9000802@trash.net> you write:
> > Julian Anastasov wrote:
> > 
> > >- provide oif learned from the input route (as before the discussed
> > >change). May be in 99% of the setups it selects the right route.
> > >I think, we should use this, at least for 2.4.
> > >
> > >  
> > >
> > Why should we do a route lookup at all ? MASQUERADE doesn't need the
> > dst_entry but only the interface address. Using ifa_list->ifa_local
> > of the outgoing in_device seems like the simplest solution to me.
> 
> You take all the fun out.
> 
> Yes, this is the best.  It always does *something*, and is
> predictable.  People with really complex routing shouldn't really use
> MASQUERADE, since it's designed for a specific, simple case.

No, I really disagree with that.  I still don't understand what the
problem was with the old solution.  We haven't received any complaints,
at least not that I can remember.

And as long as we don't provide a more sophisticated MASQUERADE
replacement target, we shouldn't change the behaviour at all.

You cannot use SNAT for the dynamic IP address case, because it doesn't
flush the tables.

And there are lots of users that have multiple DSL-dynip links these
days, trying to statically or dynamically balance web requests between
them, etc.

> Cheers,
> Rusty.

-- 
- Harald Welte <laforge@netfilter.org>             http://www.netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Rusty Russell]

In message <20040113115428.GO20206@sunbeam.de.gnumonks.org> you write:
> > Yes, this is the best.  It always does *something*, and is
> > predictable.  People with really complex routing shouldn't really use
> > MASQUERADE, since it's designed for a specific, simple case.
> 
> No, I really disagree with that.  I still don't understand what the
> problem was with the old solution.  We haven't received any complaints,
> at least not that I can remember.

I'm not so sure.  We know Patrick's solution will work.  Yes it might
break things.

> And as long as we don't provide a more sophisticated MASQUERADE
> replacement target, we shouldn't change the behaviour at all.
> 
> You cannot use SNAT for the dynamic IP address case, because it doesn't
> flush the tables.

We should probably do "-j SNAT --dynamic" for this case.

> And there are lots of users that have multiple DSL-dynip links these
> days, trying to statically or dynamically balance web requests between
> them, etc.

In that case, the interfaces are different, (ppp0 vs ppp1) so no
problem.  You need something more complex to trigger the problem
AFAICT.

Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Julian Anastasov]

	Hello,

On Mon, 12 Jan 2004, Patrick McHardy wrote:

> Why should we do a route lookup at all ? MASQUERADE doesn't need the
> dst_entry but only the interface address. Using ifa_list->ifa_local
> of the outgoing in_device seems like the simplest solution to me.

	Because it is possible the output interface to be used for
many logical subnets. In such cases it is desired maddr to be
the preferred source address for the target. If the users do not
want to fill the routing cache with such entries they can add
SNAT rule for each path. The drawback is that SNAT has only -o
match, there is no GW match. As result, we can not properly assign
maddr in setups that have two GWs on same output interface.

> Regards,
> Patrick

Regards

--
Julian Anastasov <ja@ssi.bg>

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[kuznet]

Hello!

> This means that ip_route_output_key() returns a route with a different
> outgoing interface than the skb->dst->dev of our to-be-masqueraded
> packet. =20
>
> Why was it wrong to specify skb->dst->dev->ifindex of the previous
> 'real' routing decision as key to our current routing decision?

Because in the situtation when ip_route_output_key() with oif=0 returns result
different of previous ip_route_output_key() you get undefined result.

I expected that you do ip_route_output_key() on _rewritten_ address,
in this case you have to _drop_ packet if routing contradicts to firewalling
rules or to send the packet via that interface which routng requires.
You cannot just blindly send packet to some random interface, when
firewalling rules are illegal.

> As an example case where I would suspect problems: The packet could be
> coming from a local socket, and the socket be bound to a specific
> interface (sk->bound_dev_if).

This is right observation. F.e. you could use skb->sk ? skb->sk->bound_dev_if : 0.

Alexey

Re: MASQUERADE: Route sent us somewhere else (was Re: Fw: Rusty's brain broke!)

[Julian Anastasov]

	Hello,

On Wed, 14 Jan 2004 kuznet@ms2.inr.ac.ru wrote:

> > Why was it wrong to specify skb->dst->dev->ifindex of the previous
> > 'real' routing decision as key to our current routing decision?
>
> Because in the situtation when ip_route_output_key() with oif=0 returns result
> different of previous ip_route_output_key() you get undefined result.
>
> I expected that you do ip_route_output_key() on _rewritten_ address,

	Nobody does output route on maddr, it is present only in 2.2

> in this case you have to _drop_ packet if routing contradicts to firewalling
> rules or to send the packet via that interface which routng requires.
> You cannot just blindly send packet to some random interface, when
> firewalling rules are illegal.

	Ah, no, you explain the 2.2 algorithm. 2.4 uses output
route only for the first packet, just to select maddr. While
2.2 uses 2nd route call by maddr for all next packets, 2.4
relies only on the routing cache for them.

	IMO, the rules should be:

1. packet should not change its outdev after routing (firewalling
expectation)

	In the discussed case, oif is provided only for the
1st packet in connection, providing oif=0 easily breaks
the above rule when multipath route is used - the input
route can return oif1 while the output route can return oif2.
Note the main thing: the goal before now was that oif is
provided to keep one packet to its outdev, not two packets
to same outdev, because:

2. different packets from one NAT connection still can be routed
through alive path (possibly different outdev) after the initial
path fails. This looks valid as long as we route by maddr which is
true for 2.2 but not for 2.4. Currently, such routing by maddr for
all next packets is not implemented in 2.4 with the excuse of
saving one routing call. With the current tools it can be done
with the cost of 2nd route call (the 1st is the input route).

	Now I'll remind you about my lsrc routing argument which
allows to route properly NAT connections by providing lsrc=maddr
in a ROUTING hook, only with single input route - a combination
from the current input and output route logic. It can be used
exactly from NAT because relying on the routing cache for binding
long living connections to its path is wrong.

	If this sounds complex, returning to the previous way
of providing oif still works better than now.

> Alexey

Regards

--
Julian Anastasov <ja@ssi.bg>