[LARTC] Squid + shaping question

[LARTC] Squid + shaping question

[Teodor Yantchev]

Hi folks,

So, I have a pretty simple setup - a linux router machine running as a
firewall/router for a small neighborhood LAN (approx 20 machines). I also
have squid running on the box in non-transparent mode, and also I have set
up NAT for TCP/UDP ports above 1024 for all clients and SSH/POP/SMTP/CVS
NAT'd for selected ones based on MAC filtering. No hosts whatsoever can
access ports 80 and 443 without going through squid. The uplink to the
internet is 512kbit/s downstream and 64kbit/s upstream cable modem connected
on eth1 (LAN on eth0, no DMZ).
When the LAN started to grow from a few well known friends of mine to more
people I didn't know so well 'social shaping' stopped working for us - bulk
downloaders started to saturate the link so badly that I even couldn't use
acceptably ssh from outside. So - the usual solution - www.lartc.org.
I did a lot of reading on the topic (This really got me interested in) and
finally ended up installing a self-modified version of wondershaper on the
external interface. This did solve the problem of me having usable ssh from
my office to the router machine, and the ingress qdisc partially solved the
problem of the downlink being fairly distributed between all incoming
connections - but as most of you know this is a half-baked bread. What I
think should be done is shaping the internal interface - BUT - the squid
in-between causes trouble.
So the question is - How to differentiate between traffic served from
squid's cache and traffic squid got directly from the internet ?
Shaping/policing all web traffic negates the benefits of having a caching
proxy pretty much.
After lots of googling and reading(at one point I was ready to completely
forget squid) a came up with the following alternatives, both found on the
FAQ section of www.docum.org - 'SQUID zero penalty for HIT traffic patch' by
a fellow bulgarian Marin Stavrev, and a patch giving you the ability to 'use
ACL lists to put packets in classes' by a guy named Patrick.
I'd like to ask you for your experiences with those, which one is better,
any other alternatives you know of and of course general
recipes/recommendations for solving my problem.

Well, That's it put shortly in an over-sized mail. Thanks in advance for
your advice.

Regards,
Teddy


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Squid + shaping question

[Evgeni Gechev]

Short: you need zph patch.
Detailed: you could use both, if you need. They just do different jobs.
With the first patch you could control outgoing connections, i.e. 
communication between squid and web servers/peers. With the second patch 
(zph), you could control communication between squid and clients, and as 
I understand, this is what you are interested in.

Teodor Yantchev wrote:

>Hi folks,
>
>So, I have a pretty simple setup - a linux router machine running as a
>firewall/router for a small neighborhood LAN (approx 20 machines). I also
>have squid running on the box in non-transparent mode, and also I have set
>up NAT for TCP/UDP ports above 1024 for all clients and SSH/POP/SMTP/CVS
>NAT'd for selected ones based on MAC filtering. No hosts whatsoever can
>access ports 80 and 443 without going through squid. The uplink to the
>internet is 512kbit/s downstream and 64kbit/s upstream cable modem connected
>on eth1 (LAN on eth0, no DMZ).
>When the LAN started to grow from a few well known friends of mine to more
>people I didn't know so well 'social shaping' stopped working for us - bulk
>downloaders started to saturate the link so badly that I even couldn't use
>acceptably ssh from outside. So - the usual solution - www.lartc.org.
>I did a lot of reading on the topic (This really got me interested in) and
>finally ended up installing a self-modified version of wondershaper on the
>external interface. This did solve the problem of me having usable ssh from
>my office to the router machine, and the ingress qdisc partially solved the
>problem of the downlink being fairly distributed between all incoming
>connections - but as most of you know this is a half-baked bread. What I
>think should be done is shaping the internal interface - BUT - the squid
>in-between causes trouble.
>So the question is - How to differentiate between traffic served from
>squid's cache and traffic squid got directly from the internet ?
>Shaping/policing all web traffic negates the benefits of having a caching
>proxy pretty much.
>After lots of googling and reading(at one point I was ready to completely
>forget squid) a came up with the following alternatives, both found on the
>FAQ section of www.docum.org - 'SQUID zero penalty for HIT traffic patch' by
>a fellow bulgarian Marin Stavrev, and a patch giving you the ability to 'use
>ACL lists to put packets in classes' by a guy named Patrick.
>I'd like to ask you for your experiences with those, which one is better,
>any other alternatives you know of and of course general
>recipes/recommendations for solving my problem.
>
>Well, That's it put shortly in an over-sized mail. Thanks in advance for
>your advice.
>
>Regards,
>Teddy
>
>
>_______________________________________________
>LARTC mailing list / LARTC@mailman.ds9a.nl
>http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
>
>
>  
>

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Squid + shaping question

[Radoslav Kolev]

Hi, Teodor!
Integrating squid with traffic control has been a big problem for all of 
us.
Besides the options listed at the docum.org faq, there's a patch at 
http://sed.pl/~mrk/qos/, which is very similar to ZPH, unfortunately the 
page is available only in Polish, so it didn't become very popular. You 
can just download the patch and figure out how to use it.
There's also the wipl/wrr proxy remap package at 
http://wipl-wrr.sourceforge.net/proxyremap.html
As a last resort, if you have a small number of clients (<256) you can 
put IP aliases on the outer intefrace of the Squid machine, then use acl 
to select different source IP for each client machine.
To me the ZPH + Patrick  McHardy's acl classify patch combination seems 
the best solutions available now, but I don't have any experience to share.
It would be interesting to hear from someone using it.

Greetings,
RAdo
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Re: [LARTC] Squid + shaping question

[Andy Furniss]

Teodor Yantchev wrote:
> Hi folks,
> 
> So, I have a pretty simple setup - a linux router machine running as a
> firewall/router for a small neighborhood LAN (approx 20 machines). I also
> have squid running on the box in non-transparent mode, and also I have set
> up NAT for TCP/UDP ports above 1024 for all clients and SSH/POP/SMTP/CVS
> NAT'd for selected ones based on MAC filtering. No hosts whatsoever can
> access ports 80 and 443 without going through squid. The uplink to the
> internet is 512kbit/s downstream and 64kbit/s upstream cable modem connected
> on eth1 (LAN on eth0, no DMZ).
> When the LAN started to grow from a few well known friends of mine to more
> people I didn't know so well 'social shaping' stopped working for us - bulk
> downloaders started to saturate the link so badly that I even couldn't use
> acceptably ssh from outside. So - the usual solution - www.lartc.org.
> I did a lot of reading on the topic (This really got me interested in) and
> finally ended up installing a self-modified version of wondershaper on the
> external interface. This did solve the problem of me having usable ssh from
> my office to the router machine, and the ingress qdisc partially solved the
> problem of the downlink being fairly distributed between all incoming
> connections - but as most of you know this is a half-baked bread. What I
> think should be done is shaping the internal interface - BUT - the squid
> in-between causes trouble.
> So the question is - How to differentiate between traffic served from
> squid's cache and traffic squid got directly from the internet ?
> Shaping/policing all web traffic negates the benefits of having a caching
> proxy pretty much.
> After lots of googling and reading(at one point I was ready to completely
> forget squid) a came up with the following alternatives, both found on the
> FAQ section of www.docum.org - 'SQUID zero penalty for HIT traffic patch' by
> a fellow bulgarian Marin Stavrev, and a patch giving you the ability to 'use
> ACL lists to put packets in classes' by a guy named Patrick.
> I'd like to ask you for your experiences with those, which one is better,
> any other alternatives you know of and of course general
> recipes/recommendations for solving my problem.

You could shape on just the internet link using IMQ with the NAT patch 
to control traffic from the inet to squid.

You can already shape up traffic - 64K for 20 machines isn't nice, but 
you can still do it if interactive traffic is less.

Given the other answers - I may be missing something, I've never used 
squid, but can shape local destined bittorrent OK.

Andy.

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/