Re: Re-routing packets via netfilter (ip_rt_bug)

Re: Re-routing packets via netfilter (ip_rt_bug)

[Herbert Xu]

Patrick McHardy <kaber@trash.net> wrote:
> --- 70652aa8f30bea3ea83594cc4a47a11f7a8db89d/net/core/netfilter.c  (mode:100644 sha1:e51cfa46950cf8f1f4dea42be94e71d76d8c3c5b)
> +++ a469360c577fdf6919b9a771521eca120103db45/net/core/netfilter.c  (mode:100644 sha1:85936a0b23d9ea42e2cd9d45e8254c2f780eb786)
> @@ -611,7 +611,8 @@
>        /* some non-standard hacks like ipt_REJECT.c:send_reset() can cause
>         * packets with foreign saddr to appear on the NF_IP_LOCAL_OUT hook.
>         */
> -       if (inet_addr_type(iph->saddr) == RTN_LOCAL) {
> +       if (inet_addr_type(iph->saddr) == RTN_LOCAL ||
> +           inet_addr_type(iph->daddr) == RTN_LOCAL) {
>                fl.nl_u.ip4_u.daddr = iph->daddr;
>                fl.nl_u.ip4_u.saddr = iph->saddr;
>                fl.nl_u.ip4_u.tos = RT_TOS(iph->tos);

You'll still BUG if the destination is multicast/broadcast.  I'm also
unsure whether ipt_REJECT isn't susceptible to the same problem.
Luckily ipt_MIRROR is no more :)

What are the issues with getting rid of the ip_route_input path
altogether?

The only thing we gain over calling ip_route_output is the ability
to set the input device.  But even that is just a fake derived from
the source address in a deterministic way.  Therefore any effects
achievable through using ip_route_input can also be done by simply
reconfiguring the policy routing table to look at the local source
addresses instead of (or in conjunction with) the input device.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Re-routing packets via netfilter (ip_rt_bug)

[Patrick McHardy]

Herbert Xu wrote:
> You'll still BUG if the destination is multicast/broadcast.  I'm also
> unsure whether ipt_REJECT isn't susceptible to the same problem.
> Luckily ipt_MIRROR is no more :)

ipt_REJECT is fine, ip_route_input() is only used in NF_IP_FORWARD.
But you're right about multicast/broadcast still causing problems,
I'll have another look.

> What are the issues with getting rid of the ip_route_input path
> altogether?
> 
> The only thing we gain over calling ip_route_output is the ability
> to set the input device.  But even that is just a fake derived from
> the source address in a deterministic way.  Therefore any effects
> achievable through using ip_route_input can also be done by simply
> reconfiguring the policy routing table to look at the local source
> addresses instead of (or in conjunction with) the input device.

No, ip_route_me_harder() can be called for packets with non-local
source. ip_route_output_slow() rejects non-local source addresses,
so the only way to use them for policy routing is by using
ip_route_input().

Regards
Patrick

Re: Re-routing packets via netfilter (ip_rt_bug)

[Herbert Xu]

On Mon, Apr 25, 2005 at 05:28:57PM +0200, Patrick McHardy wrote:
> 
> No, ip_route_me_harder() can be called for packets with non-local
> source. ip_route_output_slow() rejects non-local source addresses,
> so the only way to use them for policy routing is by using
> ip_route_input().

You're right.  But then we can't call ip_route_output in the case
where saddr is foreign but daddr is local.  Nor can we call
ip_route_input since the output will be ip_rt_bug.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Re-routing packets via netfilter (ip_rt_bug)

[Patrick McHardy]

Herbert Xu wrote:
> You're right.  But then we can't call ip_route_output in the case
> where saddr is foreign but daddr is local.  Nor can we call
> ip_route_input since the output will be ip_rt_bug.

In that case we need to use saddr=0, which shouldn't make any difference
with sane routing.

Regards
Patrick

Re: Re-routing packets via netfilter (ip_rt_bug)

[Herbert Xu]

On Tue, Apr 26, 2005 at 02:08:18AM +0200, Patrick McHardy wrote:
> Herbert Xu wrote:
> >You're right.  But then we can't call ip_route_output in the case
> >where saddr is foreign but daddr is local.  Nor can we call
> >ip_route_input since the output will be ip_rt_bug.
> 
> In that case we need to use saddr=0, which shouldn't make any difference
> with sane routing.

Makes sense.  But what about the case where saddr is foreign but
daddr is broadcast/multicast?

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Re-routing packets via netfilter (ip_rt_bug)

[Patrick McHardy]

Herbert Xu wrote:
> Makes sense.  But what about the case where saddr is foreign but
> daddr is broadcast/multicast?

Looks like we have no choice but to also use saddr=0 and
ip_route_output() in this case.

Regards
Patrick

Re: Re-routing packets via netfilter (ip_rt_bug)

[Herbert Xu]

On Tue, Apr 26, 2005 at 03:17:27PM +0200, Patrick McHardy wrote:
> 
> Looks like we have no choice but to also use saddr=0 and
> ip_route_output() in this case.

Let's look at the bigger picture.  There are three users of
ip_route_me_harder: nat, mangle and queue.  They're all done
in LOCAL_OUT.

For nat/mangle, the source address cannot change so it's
guaranteed to be a local IP address.  On the face of it,
queue could be changing the source address.  However, the
more I think about it the more I reckon that it should
be disallowed.

If the user is changing the source address in LOCAL_OUT/queue
then he's doing SNAT in LOCAL_OUT.  This violates some fundamental
assumptions in netfilter.  The user also isn't going to have
an easy time setting up the reverse DNAT since the corresponding
location on the reverse side does not have a ip_route_me_harder
call.

Even if there is really a demand for SNAT in LOCAL_OUT, we
should probably be implementing it properly rather than having
the user craft their own in user-space.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Re-routing packets via netfilter (ip_rt_bug)

[Patrick McHardy]

Herbert Xu wrote:
> Let's look at the bigger picture.  There are three users of
> ip_route_me_harder: nat, mangle and queue.  They're all done
> in LOCAL_OUT.
> 
> For nat/mangle, the source address cannot change so it's
> guaranteed to be a local IP address.  On the face of it,
> queue could be changing the source address.  However, the
> more I think about it the more I reckon that it should
> be disallowed.

The ipt_REJECT target can send TCP RSTs with foreign source which
go through LOCAL_OUT. Restricting it to this case and adding proper
checks to ipt_REJECT would relieve us of having to handle the last
case you pointed out (foreign saddr, broadcast/multicast daddr), but
it shouldn't be hard to also handle this case.

> If the user is changing the source address in LOCAL_OUT/queue
> then he's doing SNAT in LOCAL_OUT.  This violates some fundamental
> assumptions in netfilter.  The user also isn't going to have
> an easy time setting up the reverse DNAT since the corresponding
> location on the reverse side does not have a ip_route_me_harder
> call.

These assumptions are only for stateful NAT, the mangle table seems
to try to deal with stateless NAT by rerouting in LOCAL_OUT when
saddr/daddr changed. But it could also just be some left-over
cut-n-pasted from ip_nat_standalone.c, I don't think anyone is doing
stateless NAT with netfilter.

Regards
Patrick

Re: Re-routing packets via netfilter (ip_rt_bug)

[Herbert Xu]

On Wed, Apr 27, 2005 at 02:56:48AM +0200, Patrick McHardy wrote:
> 
> The ipt_REJECT target can send TCP RSTs with foreign source which
> go through LOCAL_OUT. Restricting it to this case and adding proper

Couldn't we feed the TCP RST packets with foreign sources through
the FORWARD table? We're lying to the routing system already by
telling it that the packet is forwarded.  So I don't see anything
wrong with lying to netfilter as well :)

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Re-routing packets via netfilter (ip_rt_bug)

[Patrick McHardy]

Herbert Xu wrote:
> Couldn't we feed the TCP RST packets with foreign sources through
> the FORWARD table? We're lying to the routing system already by
> telling it that the packet is forwarded.  So I don't see anything
> wrong with lying to netfilter as well :)

Forwarded packets can't have any NAT manips in LOCAL_OUT, so it
should work. I'm not sure about it though because it would be
the only place where packets just appear in FORWARD, usually
all packets enters through PRE_ROUTING or LOCAL_OUT.

Regards
Patrick

Re: Re-routing packets via netfilter (ip_rt_bug)

[Herbert Xu]

On Wed, Apr 27, 2005 at 12:26:13PM +0200, Patrick McHardy wrote:
> 
> Forwarded packets can't have any NAT manips in LOCAL_OUT, so it
> should work. I'm not sure about it though because it would be
> the only place where packets just appear in FORWARD, usually
> all packets enters through PRE_ROUTING or LOCAL_OUT.

It's also the only place where we generate a packet with a non-local
source address :)
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Re-routing packets via netfilter (ip_rt_bug)

[Jozsef Kadlecsik]

On Wed, 27 Apr 2005, Herbert Xu wrote:

> On Wed, Apr 27, 2005 at 12:26:13PM +0200, Patrick McHardy wrote:
> >
> > Forwarded packets can't have any NAT manips in LOCAL_OUT, so it
> > should work. I'm not sure about it though because it would be
> > the only place where packets just appear in FORWARD, usually
> > all packets enters through PRE_ROUTING or LOCAL_OUT.
>
> It's also the only place where we generate a packet with a non-local
> source address :)

Besides the REJECT target, TARPIT in patch-o-matic-ng also generates
packets with non-local source addresses. We cannot assume that REJECT is
the only one which can create such packets.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: Re-routing packets via netfilter (ip_rt_bug)

[Herbert Xu]

On Wed, Apr 27, 2005 at 12:41:01PM +0200, Jozsef Kadlecsik wrote:
> > >
> > > Forwarded packets can't have any NAT manips in LOCAL_OUT, so it
> > > should work. I'm not sure about it though because it would be
> > > the only place where packets just appear in FORWARD, usually
> > > all packets enters through PRE_ROUTING or LOCAL_OUT.
> >
> > It's also the only place where we generate a packet with a non-local
> > source address :)
> 
> Besides the REJECT target, TARPIT in patch-o-matic-ng also generates
> packets with non-local source addresses. We cannot assume that REJECT is
> the only one which can create such packets.

Any reason why it can't be fed through the FORWARD chain as opposed
to LOCAL_OUT? In general, is there anything that's generating packets
with foreign addresses that can't be fed through FORWARD?

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Re-routing packets via netfilter (ip_rt_bug)

[Herbert Xu]

On Wed, Apr 27, 2005 at 09:35:42PM +1000, herbert wrote:
>
> > Besides the REJECT target, TARPIT in patch-o-matic-ng also generates
> > packets with non-local source addresses. We cannot assume that REJECT is
> > the only one which can create such packets.
> 
> Any reason why it can't be fed through the FORWARD chain as opposed
> to LOCAL_OUT? In general, is there anything that's generating packets
> with foreign addresses that can't be fed through FORWARD?

Here is another reason why these packets should go through FORWARD.
They were generated in response to packets in INPUT/FORWARD/OUTPUT.
The original packet has not undergone SNAT in any of these cases.

However, if we feed the response packet through LOCAL_OUT it will
be subject to DNAT.  This creates a NAT asymmetry and we may end
up with the wrong destination address.

By pushing it through FORWARD it will only undergo SNAT which is
correct since the original packet would have undergone DNAT.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Re-routing packets via netfilter (ip_rt_bug)

[Patrick McHardy]

Herbert Xu wrote:
> Here is another reason why these packets should go through FORWARD.
> They were generated in response to packets in INPUT/FORWARD/OUTPUT.
> The original packet has not undergone SNAT in any of these cases.
> 
> However, if we feed the response packet through LOCAL_OUT it will
> be subject to DNAT.  This creates a NAT asymmetry and we may end
> up with the wrong destination address.
> 
> By pushing it through FORWARD it will only undergo SNAT which is
> correct since the original packet would have undergone DNAT.

This is only a problem since the recent NAT changes, but I agree
that we should fix it by moving these packets to FORWARD.

Regards
Patrick

RE: Re-routing packets via netfilter (ip_rt_bug)

[Yair Itzhaki]

I'm afraid I'm not following you.
Where did you want to set saddr=0 ?

Yair


> -----Original Message-----
> From: Herbert Xu [mailto:herbert@gondor.apana.org.au]
> Sent: Tuesday, April 26, 2005 02:39
> To: Patrick McHardy
> Cc: Yair Itzhaki; linux-kernel@vger.kernel.org; 
> netfilter-devel@lists.netfilter.org; netdev@oss.sgi.com
> Subject: Re: Re-routing packets via netfilter (ip_rt_bug)
> 
> 
> On Tue, Apr 26, 2005 at 02:08:18AM +0200, Patrick McHardy wrote:
> > Herbert Xu wrote:
> > >You're right.  But then we can't call ip_route_output in the case
> > >where saddr is foreign but daddr is local.  Nor can we call
> > >ip_route_input since the output will be ip_rt_bug.
> > 
> > In that case we need to use saddr=0, which shouldn't make 
> any difference
> > with sane routing.
> 
> Makes sense.  But what about the case where saddr is foreign but
> daddr is broadcast/multicast?
> 
> Cheers,
> -- 
> Visit Openswan at http://www.openswan.org/
> Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
> Home Page: http://gondor.apana.org.au/~herbert/
> PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
> 

RE: Re-routing packets via netfilter (ip_rt_bug)

[Yair Itzhaki]

Can anyone propose a patch that I can start checking?

I have come up with the following:

--- net/core/netfilter.c.orig   2005-04-18 21:55:30.000000000 +0300
+++ net/core/netfilter.c        2005-05-02 17:35:20.000000000 +0300
@@ -622,9 +622,10 @@
        /* some non-standard hacks like ipt_REJECT.c:send_reset() can cause
         * packets with foreign saddr to appear on the NF_IP_LOCAL_OUT hook.
         */
-       if (inet_addr_type(iph->saddr) == RTN_LOCAL) {
+       if ((inet_addr_type(iph->saddr) == RTN_LOCAL) ||
+           (inet_addr_type(iph->daddr) == RTN_LOCAL)) {
                fl.nl_u.ip4_u.daddr = iph->daddr;
-               fl.nl_u.ip4_u.saddr = iph->saddr;
+               fl.nl_u.ip4_u.saddr = 0;
                fl.nl_u.ip4_u.tos = RT_TOS(iph->tos);
                fl.oif = (*pskb)->sk ? (*pskb)->sk->sk_bound_dev_if : 0;
 #ifdef CONFIG_IP_ROUTE_FWMARK

Please advise,
Yair


> -----Original Message-----
> From: Patrick McHardy [mailto:kaber@trash.net]
> Sent: Wednesday, April 27, 2005 14:05
> To: Herbert Xu
> Cc: Jozsef Kadlecsik; netdev@oss.sgi.com; 
> netfilter-devel@lists.netfilter.org; Yair Itzhaki; 
> linux-kernel@vger.kernel.org
> Subject: Re: Re-routing packets via netfilter (ip_rt_bug)
> 
> 
> Herbert Xu wrote:
> > Here is another reason why these packets should go through FORWARD.
> > They were generated in response to packets in INPUT/FORWARD/OUTPUT.
> > The original packet has not undergone SNAT in any of these cases.
> > 
> > However, if we feed the response packet through LOCAL_OUT it will
> > be subject to DNAT.  This creates a NAT asymmetry and we may end
> > up with the wrong destination address.
> > 
> > By pushing it through FORWARD it will only undergo SNAT which is
> > correct since the original packet would have undergone DNAT.
> 
> This is only a problem since the recent NAT changes, but I agree
> that we should fix it by moving these packets to FORWARD.
> 
> Regards
> Patrick
> 

Re: Re-routing packets via netfilter (ip_rt_bug)

[Ric Wheeler]

Patrick, Hebert,

This issues stills seems to be in the latest trees - is this patch or a 
variation on it still bumping around?

Thanks!

Yair Itzhaki wrote:

>Can anyone propose a patch that I can start checking?
>
>I have come up with the following:
>
>--- net/core/netfilter.c.orig   2005-04-18 21:55:30.000000000 +0300
>+++ net/core/netfilter.c        2005-05-02 17:35:20.000000000 +0300
>@@ -622,9 +622,10 @@
>        /* some non-standard hacks like ipt_REJECT.c:send_reset() can cause
>         * packets with foreign saddr to appear on the NF_IP_LOCAL_OUT hook.
>         */
>-       if (inet_addr_type(iph->saddr) == RTN_LOCAL) {
>+       if ((inet_addr_type(iph->saddr) == RTN_LOCAL) ||
>+           (inet_addr_type(iph->daddr) == RTN_LOCAL)) {
>                fl.nl_u.ip4_u.daddr = iph->daddr;
>-               fl.nl_u.ip4_u.saddr = iph->saddr;
>+               fl.nl_u.ip4_u.saddr = 0;
>                fl.nl_u.ip4_u.tos = RT_TOS(iph->tos);
>                fl.oif = (*pskb)->sk ? (*pskb)->sk->sk_bound_dev_if : 0;
> #ifdef CONFIG_IP_ROUTE_FWMARK
>
>Please advise,
>Yair
>
>
>  
>
>>-----Original Message-----
>>From: Patrick McHardy [mailto:kaber@trash.net]
>>Sent: Wednesday, April 27, 2005 14:05
>>To: Herbert Xu
>>Cc: Jozsef Kadlecsik; netdev@oss.sgi.com; 
>>netfilter-devel@lists.netfilter.org; Yair Itzhaki; 
>>linux-kernel@vger.kernel.org
>>Subject: Re: Re-routing packets via netfilter (ip_rt_bug)
>>
>>
>>Herbert Xu wrote:
>>    
>>
>>>Here is another reason why these packets should go through FORWARD.
>>>They were generated in response to packets in INPUT/FORWARD/OUTPUT.
>>>The original packet has not undergone SNAT in any of these cases.
>>>
>>>However, if we feed the response packet through LOCAL_OUT it will
>>>be subject to DNAT.  This creates a NAT asymmetry and we may end
>>>up with the wrong destination address.
>>>
>>>By pushing it through FORWARD it will only undergo SNAT which is
>>>correct since the original packet would have undergone DNAT.
>>>      
>>>
>>This is only a problem since the recent NAT changes, but I agree
>>that we should fix it by moving these packets to FORWARD.
>>
>>Regards
>>Patrick
>>
>>    
>>
>
>  
>