PREROUTING table and quota

PREROUTING table and quota

[Michael]

Howdy,

I'm having some issues using the quota module in the PREROUTING table.

I'm trying to write a captive portal software, and a requirement of it
is to have the client be automatically redirected to a login page on
HTTP requests where the user has no quota remaining or isn't logged in,
and it is not a request to a "free" site.

The firewall is being rewritten by a daemon which the web frontend
connects to.  On startup, the daemon sets up a basic NAT and all the
standard rules, then when a user logs in the webpage connects to the
daemon and issues commands like "add computer at <ip> <mac> to userid
<n>".  The daemon is fairly dumb and doesn't retain any state data -
it's only there to insulate the web server and application from
requiring root.

As far as I can tell, not all packets are actually hitting my quota
rules.  Instead only new connections seem to be managed correctly -
subsequent and return packets are unaffected, and thus uncounted.

The rules are a bit messy, what I'm trying to achieve and the ordering
is:
     1. p2_unmetered: Unmetered/free locations are sent to the
        p2_allowed chain.
     2. p2_blacklist: Blacklisted locations are sent to the p2_captive
        chain.
     3. p2_users: Maps a host to a user chain (p2u_*).  This allows one
        user to be signed on from many hosts.
     4. p2_captive: All unhandled traffic is sent to this chain.

In p2_allowed:
      * All traffic is allowed through.

In p2_captive:
      * All tcp traffic on port 80 to be redirected to the local port
        998 (which is a small web server that redirects users to the
        captive portal login page)
      * All tcp traffic on other ports are redirected to the local port
        997
      * All icmp traffic is redirected to the local machine
      * All other traffic is dropped

In p2u_*:
      * If there is enough quota for the packet, it is sent to
        p2_allowed, and the size of the packet is subtracted from the
        quota.
      * If there is insufficient quota, it should not end processing
        (and goto p2_captive)

The network setup:
  eth0 (172.20.0.147): The external interface on the gateway.
  eth1 (10.13.37.3): The internal interface on the gateway.
  10.13.37.254: A client PC I'm using for testing.

Here is a dump of my iptables rules:

# Generated by iptables-save v1.3.6 on Thu Nov 13 21:22:03 2008
*filter
:INPUT ACCEPT [2765:294789]
:FORWARD ACCEPT [77631:59364457]
:OUTPUT ACCEPT [3210:423042]
COMMIT
# Completed on Thu Nov 13 21:22:03 2008
# Generated by iptables-save v1.3.6 on Thu Nov 13 21:22:03 2008
*nat
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [750:56421]
:OUTPUT ACCEPT [907:68008]
:p2_allowed - [0:0]
:p2_blacklist - [0:0]
:p2_captive - [0:0]
:p2_hosts - [0:0]
:p2_unmetered - [0:0]
:p2u_12 - [0:0]
-A PREROUTING -j p2_unmetered 
-A PREROUTING -j p2_blacklist 
-A PREROUTING -j p2_hosts 
-A PREROUTING -j p2_captive 
-A POSTROUTING -o eth0 -j MASQUERADE 
-A p2_allowed -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT 
-A p2_allowed -i eth1 -j ACCEPT 
-A p2_blacklist -d 192.168.0.0/255.255.0.0 -j p2_captive 
-A p2_captive -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 998 
-A p2_captive -p tcp -j REDIRECT --to-ports 997 
-A p2_captive -p icmp -j REDIRECT 
-A p2_captive -j DROP 
-A p2_hosts -s 10.13.37.254 -i eth1 -m mac --mac-source
00:02:XX:XX:XX:XX -j p2u_12 
-A p2_hosts -d 10.13.37.254 -o eth1 -j p2u_12
-A p2_unmetered -d 10.13.37.3 -p tcp -m tcp --dport 80 -j ACCEPT 
-A p2_unmetered -d 10.13.37.3 -p tcp -m tcp --dport 443 -j ACCEPT 
-A p2_unmetered -d 10.13.37.3 -p tcp -m tcp --dport 53 -j ACCEPT 
-A p2_unmetered -d 10.13.37.3 -p udp -m udp --dport 53 -j ACCEPT 
-A p2_unmetered -d 172.20.0.147 -j ACCEPT 
-A p2_unmetered -d 10.13.37.3 -p tcp -m tcp --dport 22 -j ACCEPT 
-A p2_unmetered -d 10.13.37.3 -p tcp -m tcp --dport 998 -j ACCEPT
-A p2u_12 -m quota --quota 999705946 -j p2_allowed 
COMMIT
# Completed on Thu Nov 13 21:22:03 2008

Versions of stuff:
      * iptables v1.3.6
      * kernel 2.6.18
      * Debian 4.0 (etch/stable) i386

I've had this previously working correctly when I was doing the work in
the FORWARD chain of the filter rule, however I cannot do automatic
redirection to the login page using this method.

Any advice would be greatly appreciated.

Cheers!


--michael

Re: PREROUTING table and quota

[Pascal Hambourg]

Hello,

Michael a écrit :
> 
> I'm having some issues using the quota module in the PREROUTING table.

There is no such table. Do you mean the PREROUTING _chain_ in the 'nat' 
table ?

[...]
> As far as I can tell, not all packets are actually hitting my quota
> rules.  Instead only new connections seem to be managed correctly -
> subsequent and return packets are unaffected, and thus uncounted.

This looks like the normal behaviour of the 'nat' table.

Re: PREROUTING table and quota

[Michael]

Hi,

On Thu, 2008-11-13 at 18:11 +0100, Pascal Hambourg wrote:
> Michael a écrit :
> > I'm having some issues using the quota module in the PREROUTING table.
> 
> There is no such table. Do you mean the PREROUTING _chain_ in the 'nat' 
> table ?

Yes.

> > As far as I can tell, not all packets are actually hitting my quota
> > rules.  Instead only new connections seem to be managed correctly -
> > subsequent and return packets are unaffected, and thus uncounted.
> 
> This looks like the normal behaviour of the 'nat' table.

Okay.  Thanks for the clarification.

So I will have to implement some rules in the filter table.  As far as I
can tell, though, the only way to share a quota counter between rules in
the nat table and filter table is to implement it in userspace.  Is this
correct?

Thanks,


--michael

Re: PREROUTING table and quota

[Pascal Hambourg]

Michael a écrit :
> 
> So I will have to implement some rules in the filter table.

Or in the mangle table.

> As far as I
> can tell, though, the only way to share a quota counter between rules in
> the nat table and filter table is to implement it in userspace.  Is this
> correct?

If you mean share the quota result with other rules, you can use the 
MARK target to mark packets in the mangle table and test the packet mark 
with the mark match in other tables/chains.

# mark the packet if the quota is not exceeded
iptables -t mangle -A PREROUTING -m quota [...] -j MARK --set-mark 1
# test the mark
iptables -t nat -A PREROUTING -m mark [!] --mark 1 [...]

Alternate routing table and source IP address question.

[bsilva]

At my job, we use iptables commands to mark packets to use alternate
routing tables.  Basically, we're using this to have two default gateways,
some traffic types go out interface A to one gateway, other types of
traffic go out interface B to a second gateway.

The issue is that by the time the packet gets to the routing table, the
source IP address has already been set and we get the packets exiting
interface B, but with the source IP address of interface A.

The only way we've found to correct this is to have the application that
generates the traffic bind to the particular source IP address of the
interface that it's eventually going to use.

Is there a better way to do it?

Interface "A": 123.123.111.2
Interface "B": 123.123.123.2
NTP server: 123.123.222.10

Example:
# Sets mark on NTP packets.
iptables -A OUTPUT -t mangle -p udp  --dport 123 -j MARK --set-mark 1
#Add rule to have marked packets use route table 1
ip rule add fwmark 1 table 1 priority 2000
#Build route table 1
ip route add 123.123.123.0/24 dev bond2 src 123.123.123.2 table 1
ip route add default via 123.123.123.1 src 123.123.123.2 table 1

Using this example, NTP packets will exit interface B, but will have the
source IP address of interface A.

I see this nice box on Jan Engelhardt's Packet Flow graph labeled "reroute
check", but I don't see a way to invoke it.  I was hoping that adding a
"src 123.123.123.2" parameter to the route statement would invoke reroute,
but that doesn't seem to happen.  Or is the "reroute check" only used in
NAT cases?

(http://jengelh.medozas.de/images/nf-packet-flow.png)

Thanks,
Brad

Re: Alternate routing table and source IP address question.

[Pascal Hambourg]

Hello,

bsilva a écrit :
> At my job, we use iptables commands to mark packets to use alternate
> routing tables.  Basically, we're using this to have two default gateways,
> some traffic types go out interface A to one gateway, other types of
> traffic go out interface B to a second gateway.
> 
> The issue is that by the time the packet gets to the routing table, the
> source IP address has already been set and we get the packets exiting
> interface B, but with the source IP address of interface A.

Indeed, this is a common issue.

> The only way we've found to correct this is to have the application that
> generates the traffic bind to the particular source IP address of the
> interface that it's eventually going to use.
> 
> Is there a better way to do it?

You can use SNAT or MASQUERADE, but IMO having the application bind to 
the desired source address is a cleaner way when applicable. NAT does 
not work well with all kinds of traffic. Besides, source-based routing 
is easier than mark-based routing, as it does not need iptables rules.

> I see this nice box on Jan Engelhardt's Packet Flow graph labeled "reroute
> check", but I don't see a way to invoke it.  I was hoping that adding a
> "src 123.123.123.2" parameter to the route statement would invoke reroute,
> but that doesn't seem to happen.  Or is the "reroute check" only used in
> NAT cases?

The reroute check just updates the routing decision due to an address 
translation or a mark set in the OUTPUT chains. The "src" option in a 
route is used only in the first routing decision before the OUTPUT 
chains, when the packet is created without a defined source address. A 
routing decision does not change the defined source address.