Advanced routing problem

Advanced routing problem

[Jaap Winius]

Hi all,

After having configured my first iptables firewall, it looks as though
I've already found something it can't handle. Well, I hope not...

All was fine until a secondary external interface was added. I was
hoping that I could configure the firewall to allow remote users to use
either external interface to access a Windows server on a private net
behind the firewall using VNC.

At first I couldn't access the firewall via the secondary external
interface at all, because the firewall was routing its responses out
its default route -- the primary external interface -- and the ISP was
dropping those packets (hot potato routing). However, after installing
the advanced routing package (iproute), I configured a second default
route and solved that problem.

I then set out adjust the firewall, thinking that all I'd need to do
would be to duplicate some rules to accommodate the new interface (see
rules below). However, that didn't work because, despite my efforts,
the firewall loses track of where the packets come from and sends them
out the primary external interface anyway.

Is there a way around this behavior (advanced DNAT or something), or is
the only solution to use a public IP address for the server?

Any hints towards a solution would be appreciated.

Cheers,

Jaap Winius
The Netherlands

======================

# 192.168.13.2 is the Windows server.

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 5900 \
-j DNAT --to-destination 192.168.13.2

iptables -t filter -A FORWARD -i eth0 -p tcp --dport 5900 \
-m state --state NEW -j ACCEPT

iptables -t filter -A FORWARD -d 192.168.13.2 -i eth0 -o eth1 -p tcp \
-m tcp --dport 5900 -m state --state NEW,ESTABLISHED -j ACCEPT

iptables -t filter -A FORWARD -s 192.168.13.2 -i eth1 -o eth0 -p tcp \
-m tcp --sport 5900 -m state --state ESTABLISHED -j ACCEPT

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 1723 \
-j DNAT --to-destination 192.168.13.2

iptables -t filter -A FORWARD -i eth0 -p tcp --dport 1723 \
-m state --state NEW -j ACCEPT

iptables -t filter -A FORWARD -d 192.168.13.2 -i eth0 -o eth1 -p tcp \
-m tcp --dport 1723 -m state --state NEW,ESTABLISHED -j ACCEPT

iptables -t filter -A FORWARD -s 192.168.13.2 -i eth1 -o eth0 -p tcp \
-m tcp --sport 1723 -m state --state ESTABLISHED -j ACCEPT

iptables -t nat -A PREROUTING -i eth0 -p 47 \
-j DNAT --to-destination 192.168.13.2

iptables -t filter -A FORWARD -d 192.168.13.2 -i eth0 -o eth1 -p 47 \
-j ACCEPT

iptables -t filter -A FORWARD -s 192.168.13.2 -i eth1 -o eth0 -p 47 \
-j ACCEPT

iptables -t nat -A POSTROUTING -s 192.168.13.2 -o eth0 -p 47 \
-j MASQUERADE

======================

Re: Advanced routing problem

[Martijn Lievaart]

Jaap Winius wrote:
> Hi all,
>   

Ha Jaap,

> After having configured my first iptables firewall, it looks as though
> I've already found something it can't handle. Well, I hope not...
>
> All was fine until a secondary external interface was added. I was
> hoping that I could configure the firewall to allow remote users to use
> either external interface to access a Windows server on a private net
> behind the firewall using VNC.
>
> At first I couldn't access the firewall via the secondary external
> interface at all, because the firewall was routing its responses out
> its default route -- the primary external interface -- and the ISP was
> dropping those packets (hot potato routing). However, after installing
> the advanced routing package (iproute), I configured a second default
> route and solved that problem.
>   

That's strange. that means you have some very funny routing. You'ld 
better explain how your routing is set up. Also post the output of 'ip ro'.

M4

Re: Advanced routing problem

[Jaap Winius]

Quoting Martijn Lievaart <m@rtij.nl>:

(nice website, interesting hobby)

> That's strange. that means you have some very funny routing. You'd
> better explain how your routing is set up. Also post the output of
> 'ip ro'.

Here's the output from 'ip ro':

196.190.250.17 dev ppp0  proto kernel  scope link  src 63.58.236.234
88.215.195.176/29 dev eth0  proto kernel scope link  src 88.215.195.178
63.58.50.0/25 via 63.58.236.234 dev ppp0  scope link
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.1
192.168.14.0/24 dev eth2  proto kernel  scope link  src 192.168.14.1
192.168.13.0/24 dev eth1  proto kernel  scope link  src 192.168.13.1
10.0.0.0/8 dev eth0  proto kernel  scope link  src 10.0.0.100
default via 88.215.195.177 dev eth0

The route to 63.58.50.0/25 is to make some of that ISP's servers
available, including an SMTP relay that would not be accessible any
other way.

The second routing table, secnet, has only one line:

   default via 63.58.236.234 dev ppp0

The output of 'ip rule' look like this:

   0:      from all lookup local
   1000:   from 63.58.236.234 lookup secnet
   32766:  from all lookup main
   32767:  from all lookup default

Interfaces:

eth0 -- Primary external interface.
     address 88.215.195.178
     netmask 255.255.255.248
     gateway 88.215.195.177

eth0:0 -- Priv. net behind the prim. external interface's ADSL modem.
     address 192.168.1.1
     netmask 255.255.255.0

eth0:1 -- Priv. net behind the sec. external interface's ADSL modem.
     address 10.0.0.100
     netmask 255.0.0.0

eth1 -- Internal segment (UTP)
     address 192.168.13.1
     netmask 255.255.255.0

eth2 -- Internal segment (wireless)
     address 192.168.14.1
     netmask 255.255.255.0

ppp0 -- Secondary external interface.
     address 63.58.236.234
     netmask 255.255.255.255
     P-t-P:  196.190.250.17


A diagram of the network:

                  The
                Internet
               /        \
              /          \
             /            \
            /              \
            |               |
       Sec. |         Prim. |
           ADSL            ADSL    modem
          modem-----------modem   /
                            |    /
                            |   /
                       eth0 |  / ttyS0
            ------------Firewall-------------
           |        eth1        eth2         |
           |                                 |
           |                              Wireless
           |                               Access
           |                               Point
        Ethernet
         switch--------Workstation
           |
           |
        Windows
         Server

The secondary ADSL modem has only one Ethernet interface, while the
firewall (a little Soekris box) has only one interface available for
Internet connectivity. However, the primary ADSL modem has a built-in
Ethernet switch, and since the secondary Internet connection uses PPtP
anyway, I connected it to the primary ADSL modem's switch so that I
could get to it from the firewall.

The secondary ADSL modem is part of a cheap-o consumer subscription
that the client did not want to part with. It was there before I became
responsible for the system, so I thought I might be able to put it to
good use this way -- it gives the remote users an alternative access
route in case the primary external interface goes down.

The primary ADSL modem is part of a business subscription. The ISP
doesn't allow its clients to fiddle with the modem's configuration, so
I wouldn't have recommended it, but this subscription had been
requested by the client before I arrived on the scene. Anyway, I fixed
the situation by asking for a public IP address for the firewall and
was given 87.215.195.176/29, which allows me to bypass their modem.

Did I leave anything out? If so, just ask and I'll post it ASAP.
Thanks very much for your help!

Cheers,

Jaap

Re: Advanced routing problem -- solution

[Jaap Winius]

Hi all,

To recap, my problem was that I have a Windows server on a private net
behind a firewall and a NAT that I want to make accessible to the world
outside through two different Internet connections simultaneously.
Usually, this is not possible because you have to make a choice with
your default gateway. I discovered how to make a second default gateway
by using advanced routing (iproute), which made it possible for me to
ssh to the firewall via the second Internet connection. However, if
requests via this connection were then forwarded on to the server, its
responses were still being routed out the main default gateway, after
which they would be dropped by the ISP.

The first step towards a solution was to define a second network,
192.168.15.0/24, on the UTP segment that the server was attached to.
Luckily, Windows server 2003 allows you to bind additional IP addresses
to its interfaces. In this case, only the server and the firewall (via
eth1) have addresses on this network.

The next step in my plan didn't work out. I wanted to use the firewall
to mark certain packets coming in (into eth1) from this new network
segment. I thought that this would do the trick:

   iptables -t mangle -A PREROUTING -s 192.168.15.2 -i eth1 \
      -p tcp -j MARK --set-mark 0x1
   iptables -t mangle -A PREROUTING -s 192.168.15.2 -i eth1 \
      -p 47 -j MARK --set-mark 0x1

(192.168.15.2 is the server). These marked packets would then be caught
by iproute and routed according to my secondary routing table:

   ip rule add fwmark 0x1 table secnet prio 990

I hoped that this would send them out via my secondary Internet gateway
(ppp0), but it didn't work. In theory, this kind of configuration will
allow you to route your traffic by protocol (http via one gateway, smtp
via another, etc.).

Anyway, the second half of my eventual solution had nothing to do with
the netfilter and everything to do with iproute. It's simple. Since all
of the packets moving from 192.168/.15.0/24 into the firewall are
responses to requests that arrived via the secondary Internet
connection (and should be sent back that way) anyway, I could use this
one routing rule:

   ip rule add from 192.168.15.0/24 lookup secnet prio 990

This may be less precise than marking the packets (perhaps also
matching the protocol and state), but in my case this second approach
will do and is probably more efficient.

Cheers,

Jaap