raw table and NOTRACK target

raw table and NOTRACK target

[Ramunas Vabolis]

	Hello there,

I'm looking after a pretty busy router. Right now everyting in
PREROUTING chain is joined to NOTRACK.:

/sbin/iptables -t raw -A PREROUTING -j NOTRACK
I need to intercept and redirect tcp http sessions. But right now I'm
stuck while trying to add exceptions to NOTRACK.

I've tried adding one test on local IP (real one) and use RETURN target:
/sbin/iptables -t raw -I PREROUTING -p tcp -s host.ip --dport 80 -j RETURN
/sbin/iptables -t raw -I PREROUTING -p tcp -d host.ip --sport 80 -j RETURN

Does not work. (Tried -j ACCEPT too). Tried creating another chain, jump
from PREROUTING to that chain and RETURN or ACCEPT in there. Same thing.
iptables -vxnL shows increase only in first rule

When I add any of these rules, tcpdump shows that packet arrives to
inner interface, but does not travel to outgoing one. 

I'm using Debian Lenny with stock packets (kernel icluding). 

What am I missing or doing wrong? From what I gather packets are 
discarded at raw table. 

Re: raw table and NOTRACK target

[Покотиленко Костик]

В Чтв, 25/06/2009 в 16:05 +0300, Ramunas Vabolis пишет:
> 	Hello there,
> 
> I'm looking after a pretty busy router. Right now everyting in
> PREROUTING chain is joined to NOTRACK.:
> 
> /sbin/iptables -t raw -A PREROUTING -j NOTRACK
> I need to intercept and redirect tcp http sessions. But right now I'm
> stuck while trying to add exceptions to NOTRACK.
> 
> I've tried adding one test on local IP (real one) and use RETURN target:
> /sbin/iptables -t raw -I PREROUTING -p tcp -s host.ip --dport 80 -j RETURN
> /sbin/iptables -t raw -I PREROUTING -p tcp -d host.ip --sport 80 -j RETURN
> 
> Does not work. (Tried -j ACCEPT too). Tried creating another chain, jump
> from PREROUTING to that chain and RETURN or ACCEPT in there. Same thing.
> iptables -vxnL shows increase only in first rule

Pay attention to order. As far as I understood your rule order is like
this:

-t raw -A PREROUTING -j NOTRACK
-t raw -A PREROUTING -p tcp -s host.ip --dport 80 -j RETURN
-t raw -A PREROUTING -p tcp -d host.ip --sport 80 -j RETURN

first rule matches any packet setting an action of NOTRACK. You should change the order to be like this:

-t raw -A PREROUTING -p tcp -s host.ip --dport 80 -j RETURN
-t raw -A PREROUTING -p tcp -d host.ip --sport 80 -j RETURN
-t raw -A PREROUTING -j NOTRACK

assuming you already have this rule:

/sbin/iptables -t raw -A PREROUTING -j NOTRACK

then you should add rest to the top (at posision 1) by:

/sbin/iptables -t raw -I PREROUTING 1 -p tcp -s host.ip --dport 80 -j RETURN
/sbin/iptables -t raw -I PREROUTING 1 -p tcp -d host.ip --sport 80 -j RETURN

> I add any of these rules, tcpdump shows that packet arrives to
> inner interface, but does not travel to outgoing one. 
> 
> I'm using Debian Lenny with stock packets (kernel icluding). 
> 
> What am I missing or doing wrong? From what I gather packets are 
> discarded at raw table. 
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Покотиленко Костик <casper@meteor.dp.ua>

Re: raw table and NOTRACK target

[Ramunas Vabolis]

> > 
> Pay attention to order. As far as I understood your rule order is like
> this:
> 
> -t raw -A PREROUTING -j NOTRACK
> -t raw -A PREROUTING -p tcp -s host.ip --dport 80 -j RETURN
> -t raw -A PREROUTING -p tcp -d host.ip --sport 80 -j RETURN
<skipped>
Thank you for pointing this out, but I already well aware of that.
That's why I'm using -I to prepend rules to begining of the chain.

The first rule is hit - I can verify that with iptables -t raw -vxnL.
But it seems if packet is RETURNed of ACCEPTed in raw chain it is
removed from further processing (I'm running tcpdump on my router and I
see packet entering in local interface but it does not appear in my
outgoing interface). 

So exact steps to replicate the behaviour:

iptables -t raw -A PREROUTING -p tcp -s real.ip --dport 80 -j RETURN
iptables -t raw -A PREROUTING -p tcp -d real.ip --sport 80 -j RETURN
iptables -t raw -A PREROUTING -j NOTRACK

running lynx http://any.host.com from real.ip

running tcpdump on inner interface:
tcpdump -i ethlocal -n host real.ip and port 80 

does show connection attempts while 
tcpdump -i ethoutside -n host real.ip and port 80 
is silent.

iptables -t raw -vxnL shows that first rule is hit couple times, the
second rule is never hit.

Re: raw table and NOTRACK target

[Philip Craig]

Ramunas Vabolis wrote:
> running lynx http://any.host.com from real.ip
> 
> running tcpdump on inner interface:
> tcpdump -i ethlocal -n host real.ip and port 80 
> 
> does show connection attempts while 
> tcpdump -i ethoutside -n host real.ip and port 80 
> is silent.
> 
> iptables -t raw -vxnL shows that first rule is hit couple times, the
> second rule is never hit.

Then the problem is not in the raw table.  Something else is dropping
the first syn packet after it has been through the raw table.
You should see an outgoing syn packet before you start worrying
about the second rule being hit.

Re: raw table and NOTRACK target

[Ramunas Vabolis]

* Philip Craig <philipc@snapgear.com> [2009-06-26 10:54]:
> Ramunas Vabolis wrote:
> > running lynx http://any.host.com from real.ip
> > 
> > running tcpdump on inner interface:
> > tcpdump -i ethlocal -n host real.ip and port 80 
> > 
> > does show connection attempts while 
> > tcpdump -i ethoutside -n host real.ip and port 80 
> > is silent.
> > 
> > iptables -t raw -vxnL shows that first rule is hit couple times, the
> > second rule is never hit.
> 
> Then the problem is not in the raw table.  Something else is dropping
> the first syn packet after it has been through the raw table.
> You should see an outgoing syn packet before you start worrying
> about the second rule being hit.
	That's why I'm asking for advice :) 
	iptables -n -L |grep DROP -> is empty. I'd paste entire iptables
output, but it is quite biggie (There are quite a lot of chains which do
nothing but ACCEPT packets - for iptables based traffic accounting
solution). I've flushed all FORWARD chain just in case, but it did not
had any influence as I've suspected. 

	What else can be done to pinpoint the problem?

Re: raw table and NOTRACK target

[Покотиленко Костик]

В Птн, 26/06/2009 в 10:43 +0300, Ramunas Vabolis пишет:
> > > 
> > Pay attention to order. As far as I understood your rule order is like
> > this:
> > 
> > -t raw -A PREROUTING -j NOTRACK
> > -t raw -A PREROUTING -p tcp -s host.ip --dport 80 -j RETURN
> > -t raw -A PREROUTING -p tcp -d host.ip --sport 80 -j RETURN
> <skipped>
> Thank you for pointing this out, but I already well aware of that.
> That's why I'm using -I to prepend rules to begining of the chain.
> 
> The first rule is hit - I can verify that with iptables -t raw -vxnL.
> But it seems if packet is RETURNed of ACCEPTed in raw chain it is
> removed from further processing (I'm running tcpdump on my router and I
> see packet entering in local interface but it does not appear in my
> outgoing interface). 
> 
> So exact steps to replicate the behaviour:
> 
> iptables -t raw -A PREROUTING -p tcp -s real.ip --dport 80 -j RETURN
> iptables -t raw -A PREROUTING -p tcp -d real.ip --sport 80 -j RETURN
> iptables -t raw -A PREROUTING -j NOTRACK
> 
> running lynx http://any.host.com from real.ip
> 
> running tcpdump on inner interface:
> tcpdump -i ethlocal -n host real.ip and port 80 
> 
> does show connection attempts while 
> tcpdump -i ethoutside -n host real.ip and port 80 
> is silent.
> 
> iptables -t raw -vxnL shows that first rule is hit couple times, the
> second rule is never hit.

If I've got it right you are trying to do DNAT. The problem was that
everything was not being traced by conntrack, this is what you've
already fixed.

Next, as you've told in further message nothing block that traffic in -t
filter, that's good.

Next, show as the actual DNAT rules and check the counters encrease as
you trying to connect.

You can also check whether the connection is actually being tracked by
analizing output of "conntrack -L" or "conntrack -E".

-- 
Покотиленко Костик <casper@meteor.dp.ua>

Re: raw table and NOTRACK target

[Ramunas Vabolis]

Hello again, 

> If I've got it right you are trying to do DNAT. The problem was that
> everything was not being traced by conntrack, this is what you've
> already fixed.
	Yes, I'm trying to do some DNAT, but still got a long way to go.
 
> Next, as you've told in further message nothing block that traffic in -t
> filter, that's good.
> 
> Next, show as the actual DNAT rules and check the counters encrease as
> you trying to connect.
> 
> You can also check whether the connection is actually being tracked by
> analizing output of "conntrack -L" or "conntrack -E"
	
	The problem is that I'm stuck at step.1 - excluding items from
NOTRACK. As soon as I add rules with RETURN in raw table, packets are
dropped somewhere. To reproduce:

I've got a host, which connects with lynx to any website.

In router there no DROP rules in FORWARD and single line in raw table:
/sbin/iptables -t raw -A PREROUTING -j NOTRACK

Host connects to any remote host port 80.

As soon as I add two lines with -j RETURN in raw table (so the entries 
are tracked) I experience the same effect as I've added -j DROP: 

/sbin/iptables -t raw -I PREROUTING -p tcp -s host.ip --dport 80 -j RETURN
/sbin/iptables -t raw -I PREROUTING -p tcp -d host.ip --sport 80 -j RETURN

are acting as I've made these rules:

/sbin/iptables -t raw -I PREROUTING -p tcp -s host.ip --dport 80 -j DROP
/sbin/iptables -t raw -I PREROUTING -p tcp -d host.ip --sport 80 -j DROP

I've tried replacing RETURN with ACCEPT with same results. As soon as I
remove those 2 rules, host.ip can connect to remote 80 port. 

Re: raw table and NOTRACK target

[Покотиленко Костик]

В Птн, 26/06/2009 в 16:09 +0300, Ramunas Vabolis пишет:
> Hello again, 
> 
> > If I've got it right you are trying to do DNAT. The problem was that
> > everything was not being traced by conntrack, this is what you've
> > already fixed.
> 	Yes, I'm trying to do some DNAT, but still got a long way to go.
>  
> > Next, as you've told in further message nothing block that traffic in -t
> > filter, that's good.
> > 
> > Next, show as the actual DNAT rules and check the counters encrease as
> > you trying to connect.
> > 
> > You can also check whether the connection is actually being tracked by
> > analizing output of "conntrack -L" or "conntrack -E"
> 	
> 	The problem is that I'm stuck at step.1 - excluding items from
> NOTRACK. As soon as I add rules with RETURN in raw table, packets are
> dropped somewhere. To reproduce:
> 
> I've got a host, which connects with lynx to any website.
> 
> In router there no DROP rules in FORWARD and single line in raw table:
> /sbin/iptables -t raw -A PREROUTING -j NOTRACK
> 
> Host connects to any remote host port 80.
> 
> As soon as I add two lines with -j RETURN in raw table (so the entries 
> are tracked) I experience the same effect as I've added -j DROP: 
> 
> /sbin/iptables -t raw -I PREROUTING -p tcp -s host.ip --dport 80 -j RETURN
> /sbin/iptables -t raw -I PREROUTING -p tcp -d host.ip --sport 80 -j RETURN
> 
> are acting as I've made these rules:
> 
> /sbin/iptables -t raw -I PREROUTING -p tcp -s host.ip --dport 80 -j DROP
> /sbin/iptables -t raw -I PREROUTING -p tcp -d host.ip --sport 80 -j DROP
> 
> I've tried replacing RETURN with ACCEPT with same results. As soon as I
> remove those 2 rules, host.ip can connect to remote 80 port. 

Is host.ip the host from which you are trying to run lynx?

This is strange behaviour, but it seems that problems occur when you are
trying to use conntack. Maybe conntrack modules are not loaded or some
rules in -t nat are acting like this. Does the nat table have rules? Can
you show it (iptables-save -t nat)? Show output of "lsmod | grep conn".

-- 
Покотиленко Костик <casper@meteor.dp.ua>

Re: raw table and NOTRACK target

[Ramunas Vabolis]

> Is host.ip the host from which you are trying to run lynx?
> 
> This is strange behaviour, but it seems that problems occur when you are
> trying to use conntack. Maybe conntrack modules are not loaded or some
> rules in -t nat are acting like this. Does the nat table have rules? Can
> you show it (iptables-save -t nat)? Show output of "lsmod | grep conn".
	Thank you for your suggestions. I've left DNAT rule in nat table - 
forgot about it. Then started retracing the steps again. 
	The problem was with SNAT rule. With added additional entry in raw 
table for DNAT target everything started working. 
	Thank you for your time.