[LARTC] ipchains iproute2 and port based routing

[LARTC] ipchains iproute2 and port based routing

[Balle Marco]

Hi all!

I try to make port based routing, because a have two connections to the
internet.
My router is a "one disk floppy router for linux". It is a big router
project www.fli4l.de. I try also to make a opt, it is like a plugin for this
router.
This project uses Kernel 2.2.19 compiled with libc5 (because it is small and
you can use one floppy disk).

At the moment, iproute2 is not implementated. So i downloaded a old libc5
distribution and the kernel 2.2.19.
I compiled the kernel with the iproute 2 related options. I also compiled on
this system the iproute2 package.

On my router (at the moment on harddisk) i added the new kernel and the "ip"
from iproute2.

Everything seems to work. I tried a destination host based routing - works
percect.
But not the port based routing.

How is my configuration?

Three nics:

eth0 - 192.168.0.0/24 - local LAN - masqeraded
eth1 - ppp0 - dialup! - A-DSL Provider
eth2 - static IP - S-DSL Provider - routet to another router.

This do not work for me:

Treid to route all SSH Traffic to eth2 and WEB Traffic to ppp0:

first, i mark the pakets with ipchains in the input chain [mark 1 is eth2 |
mark 2 is ppp0]:

ipchains -A input -p tcp -s 192.168.0.0/24 -d 0/0 22 -m 1
ipchains -A input -p tcp -s 192.168.0.0/24 -d 0/0 80 -m 2


second, i added two rules:

echo 200 t-dsl >> /etc/iproute2/rt_tables
echo 201 s-dsl >> /etc/iproute2/rt_tables

ip ru add fwmark 1 table s-dsl
ip ru add fwmark 2 table a-dsl


at last, i setup the routes:

ip ro add 0/0 dev eth2 table s-dsl
ip ro add 0/0 dev ppp0 table a-dsl


The maqerading is also setup:
ipchains -A forward -s 192.168.0.0/24 -j MASQ


There nothing happens. If i try to connect with ssh or http from a other
host, nothing happens.

If a setup this with two "desternation host rules" it works.
I think there is something wrong with ipchains and the marking of packets or
ip rule can not read the mark.

How can i test, if the packtes get marked?
Anybody knows a other solution?
Is there a mistake?



Best Regards

Marco

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] ipchains iproute2 and port based routing

[Martin A. Brown]

Marco,

 : ip ro add 0/0 dev eth2 table s-dsl
 : ip ro add 0/0 dev ppp0 table a-dsl

You need to specify a default gateway here, or else you are telling your 
box to route 0/0 directly out the interface--which means it will arp for 
every address on the Internet on your local ethernet!

ip route add 0/0 via x.x.x.x table s-dsl
ip route add 0/0 via x.x.x.x table a-dsl

should do it.  You can use the "dev $DEVNAME" if you wish.

 : The maqerading is also setup:
 : ipchains -A forward -s 192.168.0.0/24 -j MASQ

 : How can i test, if the packtes get marked?

Look at the verbose ipchains output ("ipchains -nvL forward") output to 
see if the usage counter on the particular chain increments.

-Martin

-- 
Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com




_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

RE: [LARTC] ipchains iproute2 and port based routing

[Marco Balle]

Hi Martin!

I send this mail with a other E-Mal account - because I am now at home,
but I am Marco!!

Okay, I tried this. But is does not work.

It is very strange, because I made a tcpdump and the result shows it is
the masq?
The configuration:

ipchains -A input -p icmp -s 192.168.0.0/24 -m 2
ip ru add fwmark 2 table 10
ip route add default via x.x.x.x dev eth2 table 10
ipchains -A forward -s 192.168.0.0/24 -j MASQ
* x.x.x.x is the default gateway!

here the tcpdump on eth2 during a ping from internal 192.168.0.31 to a
host in the internet (ping 62.154.89.102 - 4 times timeout):

tcpdump: listening on eth2
19:20:28.532089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request
19:20:28.572089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply
19:20:33.532089 arp who-has x.x.x.x tell y.y.y.y
19:20:33.532089 arp reply x.x.x.x is-at 0:0:c0:b1:a9:90
19:20:33.852089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request
19:20:33.882089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply
19:20:38.852089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request
19:20:38.892089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply
19:20:43.862089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request
19:20:43.902089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply

y.y.y.y = the ip of eth2
x.x.x.x = the gateway

You can see, the ping goes out and returns on the eth2 interface. But it
will not be masqueraded to the internal host 192.168.0.31.
On this host, I started the ping.
Other strange thing: after the return of the first reply, there is a
pause of 5 seconds. After that comes a arp request.

And anything else: if I delete the rule fwmark 2 table 10, the client
(192.168.0.31) shows during a ping to outside:
192.168.0.1 (ip of eth0): no route to host

The ip rule seems to work and the ip route too because the icmp packet
goes out and comes back. But why will it not be route to the internal
host, which has sent it?

I really do not know what is wrong here.

If I do:

ip ru add default via x.x.x.x dev eth2

Everything works well - everything goes over eth2.


You wrote:
> : ip ro add 0/0 dev eth2 table s-dsl
> : ip ro add 0/0 dev ppp0 table a-dsl
>
>You need to specify a default gateway here, or else you are telling
your
>box to route 0/0 directly out the interface--which means it will arp
for
>every address on the Internet on your local ethernet!
>
>ip route add 0/0 via x.x.x.x table s-dsl
>ip route add 0/0 via x.x.x.x table a-dsl
>
>should do it.  You can use the "dev $DEVNAME" if you wish.
>
> : The maqerading is also setup:
> : ipchains -A forward -s 192.168.0.0/24 -j MASQ
>
> : How can i test, if the packtes get marked?
>
>Look at the verbose ipchains output ("ipchains -nvL forward") output to
>see if the usage counter on the particular chain increments.

And with ipchains -nvL, i can see the packets will be marked in the
input chain. This seems to work too.

Hope anybody have I idea.


Best Regards

Marco




_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

RE: [LARTC] ipchains iproute2 and port based routing

[Martin A. Brown]

Hi Marco,

Let's work through this together--your description is good--I'm a bit 
puzzled too.

 : ipchains -A input -p icmp -s 192.168.0.0/24 -m 2
 : ip ru add fwmark 2 table 10
 : ip route add default via x.x.x.x dev eth2 table 10
 : ipchains -A forward -s 192.168.0.0/24 -j MASQ

You don't write a rule to allow explicitly for the returning echo-reply 
packets.  If you have a default policy of DENY on your input chain, then 
the packet can appear on eth2 (see your tcpdump below) but still be 
denied.

If you want to make sure that the policy never gets called (easier to see 
if packets are hitting the last rule than if they are hitting the chain 
policy):

  ipchains -A input -i eth2 -j DENY -l

Now you'll have a chain that logs the packets and has a counter--this will 
make it easier to see if ipchains is part of the problem.

 : here the tcpdump on eth2 during a ping from internal 192.168.0.31 to a
 : host in the internet (ping 62.154.89.102 - 4 times timeout):
 : 
 : tcpdump: listening on eth2
 : 19:20:28.532089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request
 : 19:20:28.572089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply
 : 19:20:33.532089 arp who-has x.x.x.x tell y.y.y.y
 : 19:20:33.532089 arp reply x.x.x.x is-at 0:0:c0:b1:a9:90
 : 19:20:33.852089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request
 : 19:20:33.882089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply
 : 19:20:38.852089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request
 : 19:20:38.892089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply
 : 19:20:43.862089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request
 : 19:20:43.902089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply
 : 
 : y.y.y.y = the ip of eth2
 : x.x.x.x = the gateway

 : You can see, the ping goes out and returns on the eth2 interface. But it
 : will not be masqueraded to the internal host 192.168.0.31.

Minor note:  I'd say "de-masqueraded" here because the packet was 
masqueraded on the outbound trip, and will now be associated with the 
existing masqueraded connection.

To view the masquerading table (if you have any doubt about whether the 
entry exists or not):

  ipchains -nML

 : On this host, I started the ping.
 : Other strange thing: after the return of the first reply, there is a
 : pause of 5 seconds. After that comes a arp request.

Now that strikes me as very odd.  I don't have an easy answer for this 
one...unless you started the ping like this:  

  ping -i 5 L-EB1.L.DE.net.dtag.de

 : And anything else: if I delete the rule fwmark 2 table 10, the client
 : (192.168.0.31) shows during a ping to outside:
 : 192.168.0.1 (ip of eth0): no route to host

What does "ip rule show" tell you after you have removed the above rule?

 : The ip rule seems to work and the ip route too because the icmp packet
 : goes out and comes back. But why will it not be route to the internal
 : host, which has sent it?

See my speculation above....ipchains interference?

 : I really do not know what is wrong here.
 : If I do:
 : ip ru add default via x.x.x.x dev eth2
 : Everything works well - everything goes over eth2.

By the way, you are using "ip route flush cache" every time you make 
changes to the routing tables/RPDB, right?

OK, so:

 - remove all extraneous ip rule entries (from the RPDB)
 - add the one RPDB rule you want
 - check all your routing tables
 - ip route flush cache
 - try again from the internal host
 - check the counters on the ipchains

Also, could you show me what "ip rule show" says on the affected box?

-Martin

-- 
Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com






_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

RE: [LARTC] ipchains iproute2 and port based routing

[Martin A. Brown]

Aigh!  I think I may have spotted the problem.

Your routing table number 10 doesn't know anything about 192.168.0.0/24 
does it?

Make sure that each routing table has routes for the destinations it is 
supposed to be able to reach!

 : ipchains -A input -p icmp -s 192.168.0.0/24 -m 2
 : ip ru add fwmark 2 table 10
 : ip route add default via x.x.x.x dev eth2 table 10
 : ipchains -A forward -s 192.168.0.0/24 -j MASQ
 : * x.x.x.x is the default gateway!

-Martin

-- 
Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

RE: [LARTC] ipchains iproute2 and port based routing

[Marco Balle]

Hi Martin!

Thanks for your feedback.

Okay, i will try to go step by step.


> : ipchains -A input -p icmp -s 192.168.0.0/24 -m 2
> : ip ru add fwmark 2 table 10
> : ip route add default via x.x.x.x dev eth2 table 10
> : ipchains -A forward -s 192.168.0.0/24 -j MASQ
>
>You don't write a rule to allow explicitly for the returning echo-reply
>packets.  If you have a default policy of DENY on your input chain,
then
>the packet can appear on eth2 (see your tcpdump below) but still be
>denied.

I made every new try a ipchains -F  -  there was no other chain(s).


>If you want to make sure that the policy never gets called (easier to
see
>if packets are hitting the last rule than if they are hitting the chain
>policy):
>
>  ipchains -A input -i eth2 -j DENY -l
>
>Now you'll have a chain that logs the packets and has a counter--this
will
>make it easier to see if ipchains is part of the problem.

Okay, it seems there is a problem. In this DENY chain I get after every
ping 4 more packets (one ping - 4 tries).
It seems ipchains deny the incoming icmp packets on eth2. But why?
I tried also to specify the source ip with some other chains, and it is
the packet, that comes from the host 62.154.89.102 - exactly the packet
I am waiting for.

A ipchains -nML shows a open masq connection to the host, I ping'd:

IP masquerading entries
prot expire   source               destination          ports
ICMP 00:57.85 192.168.0.31         62.154.89.102        512 (61009) -> 8


>Now that strikes me as very odd.  I don't have an easy answer for this
>one...unless you started the ping like this:
>
>  ping -i 5 L-EB1.L.DE.net.dtag.de

I used the ip address.


> : And anything else: if I delete the rule fwmark 2 table 10, the
client
> : (192.168.0.31) shows during a ping to outside:
> : 192.168.0.1 (ip of eth0): no route to host
>
>What does "ip rule show" tell you after you have removed the above
rule?

0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup 253

Yes, it is okay - if I delete the rule, the network could not be
reached, because I have no default route ( also in the table main). 
It was only a _simple_ test to see if ip rules works. 
With this rule:

0:      from all lookup local
32765:  from all fwmark        2 lookup 10
32766:  from all lookup main
32767:  from all lookup 253

there is a timeout. It shows me, the marking of packets works and the ip
rules can see the marked packets.


>By the way, you are using "ip route flush cache" every time you make
>changes to the routing tables/RPDB, right?

Yes, i do.


---- cut ---- your next Mail ---- cut ----

>Aigh!  I think I may have spotted the problem.
>
>Your routing table number 10 doesn't know anything about 192.168.0.0/24
>does it?
>
>Make sure that each routing table has routes for the destinations it is
>supposed to be able to reach!
>
> : ipchains -A input -p icmp -s 192.168.0.0/24 -m 2
> : ip ru add fwmark 2 table 10
> : ip route add default via x.x.x.x dev eth2 table 10
> : ipchains -A forward -s 192.168.0.0/24 -j MASQ
> : * x.x.x.x is the default gateway!

Well, if I look into the rules table I see:

0:      from all lookup local
32765:  from all fwmark        2 lookup 10
32766:  from all lookup main
32767:  from all lookup 253

Okay! If I start the ping, the icmp packet comes into the input chain
with source address ex. 192.168.0.31 (my local host).
(ipchains -A input -p icmp -s 192.168.0.0/24 -m 2)
ipchains looks: 
- is the source address in network 192.168.0.0/24? Yes.
- is a icmp packet? Yes. 
Okay it will be marked with 2.

Now the ip rules number 32765 (all fwmark 2 lookup 10) see it and
activates routing table 10.

The routing table 10:
default via x.x.x.x dev eth2
x.x.x.x is the gateway.

Okay, it goes out on eth2 to the gateway - to the target host. It
answers a icmp packet. It arrives on eth2.

Not it is going hot :)
Normaly it should do the following:


Comes into the input chain. Ipchains looks:

- is the source address in network 192.168.0.0/24? No.

The source address it _not_ in the net 192.168.0.0/24, so it will not be
marked.

Now it is ip rules turn:

0:      from all lookup local
32765:  from all fwmark        2 lookup 10
32766:  from all lookup main
32767:  from all lookup 253

okay, not local, not marked with 10 - is comes into the main routing
table!

And this is the main routing table:
217.5.98.39 dev ppp0  proto kernel  scope link  src 80.131.187.122
n.n.n.n/28 dev eth2  proto kernel  scope link  src y.y.y.y
192.168.1.0/24 dev eth1  proto kernel  scope link  src 192.168.1.1
192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.1

n.n.n.n/28 is the net and y.y.y.y the ip address of eth2

In the main routing table, there is the net 192.168.0.0/24, so it should
go on!

But okay. This is not the problem.

It seems, ipchains DENY this packet. But why?


Here a ipchains -L:

Chain input (policy ACCEPT):
target     prot opt     source                destination
ports
-          icmp ------  192.168.0.0/24       anywhere              any
->   any
DENY       all  ----l-  anywhere             anywhere              n/a
Chain forward (policy DENY):
target     prot opt     source                destination
ports
MASQ       all  ------  192.168.0.0/24       anywhere              n/a
Chain output (policy ACCEPT):

The deny chain, is your chain to monitor :)
Without it (the deny chain) it is exactly the same siduation.

Wth denys ipchains this incoming packet on eth2?


Marco




_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

RE: [LARTC] ipchains iproute2 and port based routing

[Martin A. Brown]

Hi again, Marco,

 : I made every new try a ipchains -F  -  there was no other chain(s).

Got it.

 : Okay, it seems there is a problem. In this DENY chain I get after every
 : ping 4 more packets (one ping - 4 tries).
 : It seems ipchains deny the incoming icmp packets on eth2. But why?
 : I tried also to specify the source ip with some other chains, and it is
 : the packet, that comes from the host 62.154.89.102 - exactly the packet
 : I am waiting for.
 : 
 : A ipchains -nML shows a open masq connection to the host, I ping'd:
 : 
 : IP masquerading entries
 : prot expire   source               destination          ports
 : ICMP 00:57.85 192.168.0.31         62.154.89.102        512 (61009) -> 8

All is well.

 : 0:      from all lookup local
 : 32765:  from all fwmark        2 lookup 10
 : 32766:  from all lookup main
 : 32767:  from all lookup 253
 : 
 : there is a timeout. It shows me, the marking of packets works and the ip
 : rules can see the marked packets.

Looks right.

 : >By the way, you are using "ip route flush cache" every time you make
 : >changes to the routing tables/RPDB, right?
 :
 : Yes, i do.

This is just a common problem--so I wanted to ask.

 : >Aigh!  I think I may have spotted the problem.
 : >Your routing table number 10 doesn't know anything about 192.168.0.0/24
 : >does it?
 : >Make sure that each routing table has routes for the destinations it is
 : >supposed to be able to reach!
 : > : ipchains -A input -p icmp -s 192.168.0.0/24 -m 2
 : > : ip ru add fwmark 2 table 10
 : > : ip route add default via x.x.x.x dev eth2 table 10
 : > : ipchains -A forward -s 192.168.0.0/24 -j MASQ
 : > : * x.x.x.x is the default gateway!
 : Well, if I look into the rules table I see:
 : 0:      from all lookup local
 : 32765:  from all fwmark        2 lookup 10
 : 32766:  from all lookup main
 : 32767:  from all lookup 253


<I snipped much of your mail with which I agree>

 : But okay. This is not the problem.
 : It seems, ipchains DENY this packet. But why?
 : 
 : Here a ipchains -L:


 : Chain input (policy ACCEPT):
 : target     prot opt     source                destination
 : ports
 : -          icmp ------  192.168.0.0/24       anywhere              any
 : ->   any
 : DENY       all  ----l-  anywhere             anywhere              n/a
 : Chain forward (policy DENY):
 : target     prot opt     source                destination
 : ports
 : MASQ       all  ------  192.168.0.0/24       anywhere              n/a
 : Chain output (policy ACCEPT):

I was suggesting the "ipchains -A input -j DENY -l" chain to make sure 
that any packet passing through is explicitly logged and dropped instead 
of implicitly.  I'm sure you'll see lots of DENY traffic in your 
/var/log/messages when using this rule, and things definitely won't work.  
Sorry if that was at all unclear--this was intended as a diagnosing tool.

 : The deny chain, is your chain to monitor :)
 : Without it (the deny chain) it is exactly the same siduation.
 : Wth denys ipchains this incoming packet on eth2?

It doesn't look to me like the input chain is your problem, but rather 
your forward chain.  The default policy is deny.  Try changing that to 
allow specifically what you want to allow.

You could always try that very same diagnosing ipchains rule in your 
forward chain, i.e. "ipchains -A forward -j DENY -l".  Then you'll see 
that the de-masqueraded packet is denied passing through the forward 
chain.  (At least that's my guess....)

This, of course, is the beauty of using iptables--much less worrying with 
iptables rules than with ipchains rules (in general), but you are using 
kernel 2.2.19, I believe, so iptables is not an option for you.

Let us know how you fare,

-Martin

-- 
Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

RE: [LARTC] ipchains iproute2 and port based routing

[Marco Balle]

Next :)


>You could always try that very same diagnosing ipchains rule in your
>forward chain, i.e. "ipchains -A forward -j DENY -l".  Then you'll see
>that the de-masqueraded packet is denied passing through the forward
>chain.  (At least that's my guess....)

I did. I understand the deny chain now - it was my mistake.

In the forward chain, I added the deny chain:
ipchains -A input -i eth2 -j DENY -l
But no packets arrive there.

I write it down, the short version:

Chain input (policy ACCEPT):
target     prot opt     source                destination
ports
-          icmp ------  192.168.0.0/24       anywhere              any
->   any
Chain forward (policy ACCEPT):
target     prot opt     source                destination
ports
MASQ       all  ------  192.168.0.0/24       anywhere              n/a
DENY       all  ----l-  anywhere             anywhere              n/a
Chain output (policy ACCEPT):

So the default policy is accept. With a ping of 4 tries, the forward -
MASQ chain added 4 pakets and the icmp mark chain added also 4 packets.
But no one in the DENY chain.


The same with the deny chain in the input chain:
ipchains -A forward -j DENY -l

Chain input (policy ACCEPT):
target     prot opt     source                destination
ports
-          icmp ------  192.168.0.0/24       anywhere              any
->   any
DENY       all  ----l-  anywhere             anywhere              n/a
Chain forward (policy ACCEPT):
target     prot opt     source                destination
ports
MASQ       all  ------  192.168.0.0/24       anywhere              n/a
Chain output (policy ACCEPT):

There with the same ping, 4 packets added in the MASQ, in the icmp _and_
in the input deny chain.

Hmm, if I don't make anything wrong, the packets get lost after the
input and before the forward chain.


What do you think?


Now it is time to go to bed, its 11:30pm here. 
I am at home tomorrow at 5pm CET (hope so) and will try again - so long
to it works, the next day is free for me, so I have the whole night
tomorrow.



Marco




_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/