conntrack and PREROUTING

conntrack and PREROUTING

[Doug Kehn]

Hi All,

Is the PREROUTING chain bypassed if a connection is ESTABLISHED?  There are hints to this in the documents I've read but I haven't found anything definitive.

I'm using Dansguardian with TinyProxy with the following rule:

iptables -t nat -A PREROUTING -d ! 192.168.2.0/255.255.255.0 -i eth0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3129

Everything is working, from a proxy perspective, as expected.  However, if I play a high bit-rate (>4 Mbps) video stream over HTTP, the playback is very choppy.  The choppiness is due to ACK latency through the proxy.  (Video playback is fine if I remove the proxy.)  

I know I could just create a nat PREROUTING rule to bypass the proxy for the site I'm attempting to stream video from but I'm looking for a more general solution.  Thus, what I'm attempting to do is have ACKs bypass the proxy after the connection is ESTABLISHED.  I tried using the raw table in PREROUTING but the my rule was never hit.  (Thus, the reason for my first question.)  The raw table rules I attempted were:

iptables -t raw -A PREROUTING -d ! 192.168.2.0/255.255.255.0 -i br0 -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK ACK -m tcp --dport 80 -m state --state ESTABLISHED -j NOTRACK

   -and-

iptables -t raw -A PREROUTING -d ! 192.168.2.0/255.255.255.0 -i br0 -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK ACK -m tcp --dport 80 -m conntrack --ctstate ESTABLISHED -j NOTRACK

Is what I'm attempting to do possible with the existing implementation?  Does this even make sense?

I'm attempting to do this on a home router that is running Linux 2.6.18 with iptables v1.3.7-20070509


Thanks,
...doug



      

Re: conntrack and PREROUTING

[Jan Engelhardt]

On Friday 2008-06-20 01:57, Doug Kehn wrote:

>Hi All,
>
>Is the PREROUTING chain bypassed if a connection is ESTABLISHED?

Generally, no. For the nat table yes, which is only consulted on
new connections.

>There are hints to this in the documents I've read but I haven't
>found anything definitive.
>
>I'm using Dansguardian with TinyProxy with the following rule:
>
>iptables -t nat -A PREROUTING -d ! 192.168.2.0/255.255.255.0 -i eth0
>-p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3129
>
>Everything is working, from a proxy perspective, as expected. 
>However, if I play a high bit-rate (>4 Mbps) video stream over HTTP,
>the playback is very choppy.  The choppiness is due to ACK latency
>through the proxy.  (Video playback is fine if I remove the proxy.)
>
>I know I could just create a nat PREROUTING rule to bypass the proxy
>for the site I'm attempting to stream video from but I'm looking for
>a more general solution.  Thus, what I'm attempting to do is have
>ACKs bypass the proxy after the connection is ESTABLISHED.  I tried
>using the raw table in PREROUTING but the my rule was never hit. 
>(Thus, the reason for my first question.) The raw table rules I
>attempted were:
>
>iptables -t raw -A PREROUTING -d ! 192.168.2.0/255.255.255.0 -i br0
>-p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK ACK -m tcp --dport 80 -m
>state --state ESTABLISHED -j NOTRACK
>
>   -and-
>
>iptables -t raw -A PREROUTING -d ! 192.168.2.0/255.255.255.0 -i br0
>-p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK ACK -m tcp --dport 80 -m
>conntrack --ctstate ESTABLISHED -j NOTRACK
>
>Does this even make sense?

Yes, but:

>Is what I'm attempting to do possible with the existing
>implementation?

The connection tracking and NAT subsystems might get confused if
they do not see all ACKs despite the TCP window moving on. You
will know when the connection hangs.

Re: conntrack and PREROUTING

[Patrick McHardy]

Jan Engelhardt wrote:
> On Friday 2008-06-20 01:57, Doug Kehn wrote:
> 
>> iptables -t raw -A PREROUTING -d ! 192.168.2.0/255.255.255.0 -i br0
>> -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK ACK -m tcp --dport 80 -m
>> conntrack --ctstate ESTABLISHED -j NOTRACK
>>
>> Does this even make sense?
> 
> Yes, but:

No. The raw table doesn't have conntrack information.

Re: conntrack and PREROUTING

[Jan Engelhardt]

On Friday 2008-06-20 12:21, Patrick McHardy wrote:

> Jan Engelhardt wrote:
>> On Friday 2008-06-20 01:57, Doug Kehn wrote:
>> 
>> > iptables -t raw -A PREROUTING -d ! 192.168.2.0/24 -i br0
>> > -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK ACK -m tcp --dport 80 -m
>> > conntrack --ctstate ESTABLISHED -j NOTRACK
>> >
>> > Does this even make sense?
>> 
>> Yes, but:
>
> No. The raw table doesn't have conntrack information.
>
Oops ;-)

But you can assume that any ACK belongs to an established connection,
hence omitting -m conntrack --ctstate ESTABLISHED would achieve the
same.

Of course there are scenarios, like probe ACKs sent by nmap that
do not belong to any, or a conntrack router B which got rebooted
(and hence lost the connections that are still active between two
hosts A and C), but that should be real corner cases.

Re: conntrack and PREROUTING

[Doug Kehn]

Hi Jan,


--- On Fri, 6/20/08, Jan Engelhardt wrote:

> 
> >Hi All,
> >
> >Is the PREROUTING chain bypassed if a connection is
> ESTABLISHED?
> 
> Generally, no. For the nat table yes, which is only
> consulted on
> new connections.
> 
> >There are hints to this in the documents I've read
> but I haven't
> >found anything definitive.
> >
> >I'm using Dansguardian with TinyProxy with the
> following rule:
> >
> >iptables -t nat -A PREROUTING -d !
> 192.168.2.0/255.255.255.0 -i eth0
> >-p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3129
> >
> >Everything is working, from a proxy perspective, as
> expected. 
> >However, if I play a high bit-rate (>4 Mbps) video
> stream over HTTP,
> >the playback is very choppy.  The choppiness is due to
> ACK latency
> >through the proxy.  (Video playback is fine if I remove
> the proxy.)
> >
> >I know I could just create a nat PREROUTING rule to
> bypass the proxy
> >for the site I'm attempting to stream video from
> but I'm looking for
> >a more general solution.  Thus, what I'm attempting
> to do is have
> >ACKs bypass the proxy after the connection is
> ESTABLISHED.  I tried
> >using the raw table in PREROUTING but the my rule was
> never hit. 
> >(Thus, the reason for my first question.) The raw table
> rules I
> >attempted were:
> >
> >iptables -t raw -A PREROUTING -d !
> 192.168.2.0/255.255.255.0 -i eth0
> >-p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK ACK -m tcp
> --dport 80 -m
> >state --state ESTABLISHED -j NOTRACK
> >
> >   -and-
> >
> >iptables -t raw -A PREROUTING -d !
> 192.168.2.0/255.255.255.0 -i eth0
> >-p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK ACK -m tcp
> --dport 80 -m
> >conntrack --ctstate ESTABLISHED -j NOTRACK
> >
> >Does this even make sense?
> 
> Yes, but:
> 
> >Is what I'm attempting to do possible with the
> existing
> >implementation?
> 
> The connection tracking and NAT subsystems might get
> confused if
> they do not see all ACKs despite the TCP window moving on.
> You
> will know when the connection hangs.

The connections do hang if I change the rule to:

iptables -t raw -A PREROUTING -d ! 192.168.2.0/255.255.255.0 -i eth0 -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK ACK -m tcp --dport 80 -j NOTRACK

This makes sense, I believe, because the ACK to the SYN-ACK wouldn't be tracked and the connection state would never reach ESTABLISHED.

Regards,
...doug
 


      

Re: conntrack and PREROUTING

[Gáspár Lajos]

Doug Kehn írta:
> The connections do hang if I change the rule to:
>
> iptables -t raw -A PREROUTING -d ! 192.168.2.0/255.255.255.0 -i eth0 -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK ACK -m tcp --dport 80 -j NOTRACK
>
> This makes sense, I believe, because the ACK to the SYN-ACK wouldn't be tracked and the connection state would never reach ESTABLISHED.
>   
What about the UNTRACKED state???
> Regards,
> ...doug
>   

Swifty

Re: conntrack and PREROUTING

[Doug Kehn]

Hi Patrick,


--- On Fri, 6/20/08, Patrick McHardy wrote:

> Jan Engelhardt wrote:
> > On Friday 2008-06-20 01:57, Doug Kehn wrote:
> > 
> >> iptables -t raw -A PREROUTING -d !
> 192.168.2.0/255.255.255.0 -i br0
> >> -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK ACK -m
> tcp --dport 80 -m
> >> conntrack --ctstate ESTABLISHED -j NOTRACK
> >>
> >> Does this even make sense?
> > 
> > Yes, but:
> 
> No. The raw table doesn't have conntrack information.

I assume the same holds for -m state as well?  If so, this would explain why the rules are never matched.

Is there a way to have ACKs bypass the proxy and not break connection tracking?

My theory is that when performing a streaming HTTP download (e.g. streaming video over HTTP) having the ACKs traverse the proxy introduces sufficient delay to degrade video playback.  I'm hoping to find a general solution.  Creating a NOTRACK rule for each site is possible but a little cumbersome.

Thanks,
...doug



      

Re: conntrack and PREROUTING

[Patrick McHardy]

Doug Kehn wrote:
> Hi Patrick,
> 
> 
> --- On Fri, 6/20/08, Patrick McHardy wrote:
> 
>> Jan Engelhardt wrote:
>>> On Friday 2008-06-20 01:57, Doug Kehn wrote:
>>>
>>>> iptables -t raw -A PREROUTING -d !
>> 192.168.2.0/255.255.255.0 -i br0
>>>> -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK ACK -m
>> tcp --dport 80 -m
>>>> conntrack --ctstate ESTABLISHED -j NOTRACK
>>>>
>>>> Does this even make sense?
>>> Yes, but:
>> No. The raw table doesn't have conntrack information.
> 
> I assume the same holds for -m state as well?  If so, this would explain why the rules are never matched.

Correct.

> Is there a way to have ACKs bypass the proxy and not break connection tracking?
> 
> My theory is that when performing a streaming HTTP download (e.g. streaming video over HTTP) having the ACKs traverse the proxy introduces sufficient delay to degrade video playback.  I'm hoping to find a general solution.  Creating a NOTRACK rule for each site is possible but a little cumbersome.

I don't see how that could work, the proxy has two seperate
connections (client<->proxy and proxy<->server), so it
needs to receive all packets.

Re: conntrack and PREROUTING

[Doug Kehn]

Hi Gáspár,


--- On Fri, 6/20/08, Gáspár Lajos wrote:

> Doug Kehn írta:
> > The connections do hang if I change the rule to:
> >
> > iptables -t raw -A PREROUTING -d !
> 192.168.2.0/255.255.255.0 -i eth0 -p tcp -m tcp --tcp-flags
> FIN,SYN,RST,ACK ACK -m tcp --dport 80 -j NOTRACK
> >
> > This makes sense, I believe, because the ACK to the
> SYN-ACK wouldn't be tracked and the connection state
> would never reach ESTABLISHED.
> >   
> What about the UNTRACKED state???

I get the same result with UNTRACKED.  8(

Thanks,
...doug



      

Re: conntrack and PREROUTING

[Doug Kehn]

Hi Patrick,


--- On Fri, 6/20/08, Patrick McHardy wrote:

> Doug Kehn wrote:
> > Hi Patrick,
> > 
> > 
> > --- On Fri, 6/20/08, Patrick McHardy wrote:
> > 
> >> Jan Engelhardt wrote:
> >>> On Friday 2008-06-20 01:57, Doug Kehn wrote:
> >>>
> >>>> iptables -t raw -A PREROUTING -d !
> >> 192.168.2.0/255.255.255.0 -i br0
> >>>> -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK
> ACK -m
> >> tcp --dport 80 -m
> >>>> conntrack --ctstate ESTABLISHED -j NOTRACK
> >>>>
> >>>> Does this even make sense?
> >>> Yes, but:
> >> No. The raw table doesn't have conntrack
> information.
> > 
> > I assume the same holds for -m state as well?  If so,
> this would explain why the rules are never matched.
> 
> Correct.
> 
> > Is there a way to have ACKs bypass the proxy and not
> break connection tracking?
> > 
> > My theory is that when performing a streaming HTTP
> download (e.g. streaming video over HTTP) having the ACKs
> traverse the proxy introduces sufficient delay to degrade
> video playback.  I'm hoping to find a general solution.
>  Creating a NOTRACK rule for each site is possible but a
> little cumbersome.
> 
> I don't see how that could work, the proxy has two
> seperate
> connections (client<->proxy and
> proxy<->server), so it
> needs to receive all packets.

Yep, that's my understanding.  I guess I was hoping that my feeble little brain was missing something obvious/cleaver.

Thanks,
...doug


      

Re: conntrack and PREROUTING

[Jan Engelhardt]

On Friday 2008-06-20 14:49, Doug Kehn wrote:
>
>The connections do hang if I change the rule to:
>
>iptables -t raw -A PREROUTING -d ! 192.168.2.0/255.255.255.0 -i eth0
>-p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK ACK -m tcp --dport 80 -j
>NOTRACK
>
>This makes sense, I believe, because the ACK to the SYN-ACK wouldn't
>be tracked and the connection state would never reach ESTABLISHED.

The connection is established once conntrack sees the SYNACK.