netfilter/dansguardian/polipo slow

netfilter/dansguardian/polipo slow

[Doug Kehn]

Hi All,

I have an AMCC PPC-440 based board running Monta Vista Linux 2.6.18, dansguardian 2.9.8.5, and polipo 1.0.4.  dansguardian is used for URL and content filtering.  polipo is serving as the http proxy.  The iptables rule to redirect port 80 packets to dansguardian is:

iptables -t nat -A PREROUTING -d ! 192.168.2.0/255.255.255.0 -i br0 \
-p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3129

All components are functioning properly.  However, web page load times are 3 to 4 times slower with this rule in place than without.  I suspected the delay was with dansguardian/polipo.  Then, I left the rule in place and configured the browser's http proxy settings for 192.168.2.1/3129.  Page load times decreased dramatically.  Packets were still traversing dansguardian/polipo as URL/content filtering rules still worked as expected.  I also changed the PREROUTING policy from ACCEPT to DROP.  After doing this, I could no longer browse the internet (or communicate with the board).  I'm pretty sure the PREROUTING chain is being traversed; the rule is just not matching.

It appears (???) the delay is only observed when the rule matches.  I tried different variants of the rule to see if writing the rule in different ways produced different results.  For example,

iptables -t nat -A PREROUTING -i br0 -p tcp -m tcp --dport 80 \
-j REDIRECT --to-ports 3129

All rule variations resulted in the same increased page load times.

Unfortunately, updating the kernel and/or configuring the browser's http proxy settings aren't allowable options. 8(  Does anyone have any information, comments suggestions, tips, or tricks?  

Thanks,
...doug



      

Re: netfilter/dansguardian/polipo slow

[Grant Taylor]

On 09/14/08 09:49, Doug Kehn wrote:
> I have an AMCC PPC-440 based board running Monta Vista Linux 2.6.18, 
> dansguardian 2.9.8.5, and polipo 1.0.4.  dansguardian is used for URL 
> and content filtering.  polipo is serving as the http proxy.  The 
> iptables rule to redirect port 80 packets to dansguardian is:

I take it that DansGuardian is talking to Polipo and clients are talking 
(via redirect) to DansGuardian?

> All components are functioning properly.  However, web page load 
> times are 3 to 4 times slower with this rule in place than without. 
> I suspected the delay was with dansguardian/polipo.  Then, I left the 
> rule in place and configured the browser's http proxy settings for 
> 192.168.2.1/3129.  Page load times decreased dramatically.  Packets 
> were still traversing dansguardian/polipo as URL/content filtering 
> rules still worked as expected.  I also changed the PREROUTING policy 
> from ACCEPT to DROP.  After doing this, I could no longer browse the 
> internet (or communicate with the board).  I'm pretty sure the 
> PREROUTING chain is being traversed; the rule is just not matching.
 >
> It appears (???) the delay is only observed when the rule matches.  I 
> tried different variants of the rule to see if writing the rule in 
> different ways produced different results.  For example,
>
> All rule variations resulted in the same increased page load times.

This is as I would expect.  If you write the rules differently and 
compare the output of iptables-save you will see the rules are 
translated to the same thing in kernel.

> Unfortunately, updating the kernel and/or configuring the browser's 
> http proxy settings aren't allowable options. 8(  Does anyone have 
> any information, comments suggestions, tips, or tricks?

Try using a different caching proxy behind DansGuardian to cache the 
filtered content rather than having DansGuardian filter all content each 
and every time someone requests it.

If memory recalls, DansGuardian has to talk to an upstream proxy so, you 
will most likely end up with a proxy on both sides of DansGuardian, with 
at least the one behind it being a caching proxy.



Grant. . . .

Re: netfilter/dansguardian/polipo slow

[Brian Austin - Standard Universal]

Grant Taylor wrote:
> On 09/14/08 09:49, Doug Kehn wrote:
>> I have an AMCC PPC-440 based board running Monta Vista Linux 2.6.18, 
>> dansguardian 2.9.8.5, and polipo 1.0.4.  dansguardian is used for URL 
>> and content filtering.  polipo is serving as the http proxy.  The 
>> iptables rule to redirect port 80 packets to dansguardian is:
>
> I take it that DansGuardian is talking to Polipo and clients are 
> talking (via redirect) to DansGuardian?
>
>> All components are functioning properly.  However, web page load 
>> times are 3 to 4 times slower with this rule in place than without. I 
>> suspected the delay was with dansguardian/polipo.  Then, I left the 
>> rule in place and configured the browser's http proxy settings for 
>> 192.168.2.1/3129.  Page load times decreased dramatically.  Packets 
>> were still traversing dansguardian/polipo as URL/content filtering 
>> rules still worked as expected.  I also changed the PREROUTING policy 
>> from ACCEPT to DROP.  After doing this, I could no longer browse the 
>> internet (or communicate with the board).  I'm pretty sure the 
>> PREROUTING chain is being traversed; the rule is just not matching.
> >
>> It appears (???) the delay is only observed when the rule matches.  I 
>> tried different variants of the rule to see if writing the rule in 
>> different ways produced different results.  For example,
>>
>> All rule variations resulted in the same increased page load times.
>
> This is as I would expect.  If you write the rules differently and 
> compare the output of iptables-save you will see the rules are 
> translated to the same thing in kernel.
>
>> Unfortunately, updating the kernel and/or configuring the browser's 
>> http proxy settings aren't allowable options. 8(  Does anyone have 
>> any information, comments suggestions, tips, or tricks?
>
> Try using a different caching proxy behind DansGuardian to cache the 
> filtered content rather than having DansGuardian filter all content 
> each and every time someone requests it.
>
> If memory recalls, DansGuardian has to talk to an upstream proxy so, 
> you will most likely end up with a proxy on both sides of 
> DansGuardian, with at least the one behind it being a caching proxy.
>
>
>
> Grant. . . .
> -- 
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

one subtle performance hit can be dns.

client requests dns, asks proxy, proxy has to look up dns.. I fixed this 
in my site by installing bind on the proxy machine, and having it as the 
forwarder for the rest of the network.

probably unrelated...