[LARTC] Policy routing and strange packets traversing.

[LARTC] Policy routing and strange packets traversing.

[Tomasz Wrona]

Hi,

Please suppose following config:

Two external interfaces for two different providers
On each of them configured NAT for specific IP addr.
ie.

4: eth2: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc htb qlen 100
    inet 1.1.1.30/30 brd 1.1.1.31 scope global eth2
6: eth4: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc htb qlen 100
    inet 2.2.2.66/27 brd 2.2.2.95 scope global eth4

On eth2 NAT outgoing traffic to 1.1.1.30
On eth4 NAT outgoing traffic to 2.2.2.66


Rules related for this interfaces and traffic:

# To be sure that traffic goes to proper gateway
22:     from 1.1.1.30 lookup 1
22:     from 2.2.2.66 lookup 2
...
# This rules are unnecessary I think but used for diagnostics gateways
#by me.
30:     from all to 1.1.1.29 lookup 1
30:     from all to 2.2.2.65 lookup 2

#Balance tables distributes traffic from LAN.
70:     from all iif eth1 lookup balance


# ip r l ta 1
default via 1.1.1.29 dev eth2
# ip r l ta 2
default via 2.2.2.65 dev eth4
# ip r l ta balance
default
        nexthop via 1.1.1.29  dev eth2 weight 2
        nexthop via 2.2.2.65  dev eth4 weight 3

So. Everything works but I have observed some behaviour what
I can't understand..

What I expected was that trafic nated to 1.1.1.30 goes throught eth2
and traffic nated to 2.2.2.66 goes throught eth4.

Unfortunatelly when become listening on eth4 with following command:
tcpdump -n -i eth4 src 1.1.1.30
I can see trafiic which I am not expecting on this interface:
1.1.1.30.3145 > 217.98.144.187.20: P 1608:2144(536) ack 1 win 16616 (DF)
1.1.1.30.4282 > 212.77.100.17.5555: . ack 1889 win 17520 (DF)

The simmilar is on eth2:
tcpdump -n -i eth2 src 2.2.2.66
2.2.2.66.6114 > 217.17.41.85.8074: P 58257:58281(24) ack 530714947 win 7506 (DF)

Of course more packets have correct sources [1.1.1.30 for eth2 and
2.2.2.66 on eth4] but I cant see the reason there are some missed
packets...
I did experiment and attached iptables DROP rule on POSTROUTING on
eth2 and eth4 interfaces to catch bad sourced packets but they didnt
catch anything what says for me this "bad" traffic didnt really go
through incorrect interfaces.

So that I am confused on this packet traversing.. Could someone explain
this behaviour ? Is it OK or I have missed something ?


Regards,
tw                   
--

-----------
 ck.eter.tym.pl

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Policy routing and strange packets traversing.

[Julian Anastasov]

	Hello,

On Sat, 1 Mar 2003, Tomasz Wrona wrote:

> Rules related for this interfaces and traffic:

	This looks a bit strange, it is not needed:

> # To be sure that traffic goes to proper gateway
> 22:     from 1.1.1.30 lookup 1
> 22:     from 2.2.2.66 lookup 2

	You already have link routes to these IPs in table main

> ...
> # This rules are unnecessary I think but used for diagnostics gateways
> #by me.

	Yes, you don't need them:

> 30:     from all to 1.1.1.29 lookup 1
> 30:     from all to 2.2.2.65 lookup 2
>
> #Balance tables distributes traffic from LAN.

	Don't expect from Netfilter to use correctly the routing,
you have to avoid using "iif" when playing with Netfilter. Just
use "from XXX".

> 70:     from all iif eth1 lookup balance
>
>
> # ip r l ta 1
> default via 1.1.1.29 dev eth2
> # ip r l ta 2
> default via 2.2.2.65 dev eth4
> # ip r l ta balance
> default
>         nexthop via 1.1.1.29  dev eth2 weight 2
>         nexthop via 2.2.2.65  dev eth4 weight 3
>
> So. Everything works but I have observed some behaviour what
> I can't understand..

	I don't know what works but in theory it should not work,
you don't have routes that restrict each ISP traffic through its
gateway. May be in your case each of the ISPs allow spoofing.

> What I expected was that trafic nated to 1.1.1.30 goes throught eth2
> and traffic nated to 2.2.2.66 goes throught eth4.

	Then specify it to be so:

ip rule add prio 20 from 1.1.1.30/30 table 1
ip rule add prio 20 from 2.2.2.66/27 table 2

	but you will need rules "from all to all" for
proper default route selelection and source IP autoselection for
the masquerading. The normal kernel can not give you this, you
need other solutions, eg:

http://www.ssi.bg/~ja/#routes

dgd-usage.txt contains example for rules and routes you can use.

> Unfortunatelly when become listening on eth4 with following command:
> tcpdump -n -i eth4 src 1.1.1.30
> I can see trafiic which I am not expecting on this interface:
> 1.1.1.30.3145 > 217.98.144.187.20: P 1608:2144(536) ack 1 win 16616 (DF)
> 1.1.1.30.4282 > 212.77.100.17.5555: . ack 1889 win 17520 (DF)
>
> The simmilar is on eth2:
> tcpdump -n -i eth2 src 2.2.2.66
> 2.2.2.66.6114 > 217.17.41.85.8074: P 58257:58281(24) ack 530714947 win 7506 (DF)
>
> Of course more packets have correct sources [1.1.1.30 for eth2 and
> 2.2.2.66 on eth4] but I cant see the reason there are some missed
> packets...
> I did experiment and attached iptables DROP rule on POSTROUTING on
> eth2 and eth4 interfaces to catch bad sourced packets but they didnt
> catch anything what says for me this "bad" traffic didnt really go
> through incorrect interfaces.

	May be it is the POST_ROUTING who is guilty for selecting
wrong nexthop and you can not notice it, this mistake is visible
on device output.

> So that I am confused on this packet traversing.. Could someone explain
> this behaviour ? Is it OK or I have missed something ?

	You can read about such issues, use the above URL

> Regards,
> tw
> --
>
> -----------
>  ck.eter.tym.pl

Regards

--
Julian Anastasov <ja@ssi.bg>

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Policy routing and strange packets traversing.

[Tomasz Wrona]

On Sat, 1 Mar 2003, Julian Anastasov wrote:

> 	This looks a bit strange, it is not needed:
>
> > # To be sure that traffic goes to proper gateway
> > 22:     from 1.1.1.30 lookup 1
> > 22:     from 2.2.2.66 lookup 2

Why, It's the same what You pointed me below... ?


> 	You already have link routes to these IPs in table main
> 	Yes, you don't need them:
>
> > 30:     from all to 1.1.1.29 lookup 1
> > 30:     from all to 2.2.2.65 lookup 2

OK, but I process main table after all manual typed rules... but never
mind its not issue ;)



> > #Balance tables distributes traffic from LAN.
>
> 	Don't expect from Netfilter to use correctly the routing,
> you have to avoid using "iif" when playing with Netfilter. Just
> use "from XXX".

Hmmm... I  cant understand what has netfilter to do with "iif" parameter ?
What I want to achieve is to catch all incoming traffic on eth1..

> > 70:     from all iif eth1 lookup balance
> >
> >
> > # ip r l ta 1
> > default via 1.1.1.29 dev eth2
> > # ip r l ta 2
> > default via 2.2.2.65 dev eth4
> > # ip r l ta balance
> > default
> >         nexthop via 1.1.1.29  dev eth2 weight 2
> >         nexthop via 2.2.2.65  dev eth4 weight 3
> >
> > So. Everything works but I have observed some behaviour what
> > I can't understand..
>
> 	I don't know what works but in theory it should not work,
> you don't have routes that restrict each ISP traffic through its
> gateway. May be in your case each of the ISPs allow spoofing.

Well,look at top of the mail for first rules what You say are wrong..


> > What I expected was that trafic nated to 1.1.1.30 goes throught eth2
> > and traffic nated to 2.2.2.66 goes throught eth4.
>
> 	Then specify it to be so:
>
> ip rule add prio 20 from 1.1.1.30/30 table 1
> ip rule add prio 20 from 2.2.2.66/27 table 2

This is exact the same what firs rules on top of mail, Am I really wrong ?


> 	but you will need rules "from all to all" for
> proper default route selelection and source IP autoselection for
> the masquerading.
>
Balance table catches all traffic from LAN to inet.Thats all what I need.


> The normal kernel can not give you this, you
> need other solutions, eg:
>
> http://www.ssi.bg/~ja/#routes
>
> dgd-usage.txt contains example for rules and routes you can use.

Hmm... Maybe I am wrong but It's related to NAT multiple gateways on
single interface not on different what I have...
There shouldn't be problem what I read in this article.


> > Unfortunatelly when become listening on eth4 with following command:
>
> 	May be it is the POST_ROUTING who is guilty for selecting
> wrong nexthop and you can not notice it, this mistake is visible
> on device output.
>
> > So that I am confused on this packet traversing.. Could someone explain
> > this behaviour ? Is it OK or I have missed something ?
>
> 	You can read about such issues, use the above URL


I will dig it still.

Thank You for support
tw
-- 

----------------
 ck.eter.tym.pl

"Never let shooling disturb Your education"


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Policy routing and strange packets traversing.

[Julian Anastasov]

	Hello,

On Sun, 2 Mar 2003, Tomasz Wrona wrote:

> > 	This looks a bit strange, it is not needed:
> >
> > > # To be sure that traffic goes to proper gateway
> > > 22:     from 1.1.1.30 lookup 1
> > > 22:     from 2.2.2.66 lookup 2
>
> Why, It's the same what You pointed me below... ?

	OK, I overlooked it

> > > 30:     from all to 1.1.1.29 lookup 1
> > > 30:     from all to 2.2.2.65 lookup 2
>
> OK, but I process main table after all manual typed rules... but never
> mind its not issue ;)

	It is not good to put table main after other rules, it can
be used only to override route in table main. For example, why
traffic from 1.1.1.29 to some internal IP should go to the ISP
gateway (table 1)?

> > 	Don't expect from Netfilter to use correctly the routing,
> > you have to avoid using "iif" when playing with Netfilter. Just
> > use "from XXX".
>
> Hmmm... I  cant understand what has netfilter to do with "iif" parameter ?
> What I want to achieve is to catch all incoming traffic on eth1..

	There are some places that can use output rerouting where
the iif parameter is ignored. And second, the normal kernel relies
on the routing cache to keep persistence for each NAT connection to
its selected nexthop. There is no guarantee that it will work for the
whole connection life.

> > 	but you will need rules "from all to all" for
> > proper default route selelection and source IP autoselection for
> > the masquerading.
> >
> Balance table catches all traffic from LAN to inet.Thats all what I need.

	It does not work all the time.

> > http://www.ssi.bg/~ja/#routes
> >
> > dgd-usage.txt contains example for rules and routes you can use.
>
> Hmm... Maybe I am wrong but It's related to NAT multiple gateways on
> single interface not on different what I have...

	Not exactly true, it is related to making sure each NAT
conn is bound to its allowed path(s), no matter how many interfaces
are used. Selecting different nexthop should be allowed only if
it is alternative allowed from the routing rules.

> There shouldn't be problem what I read in this article.

	Then why you see traffic to the wrong gateway?

Regards

--
Julian Anastasov <ja@ssi.bg>

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/