Managing traffic on an internal Squid box

Managing traffic on an internal Squid box

[Lonney]

Hello all,

I am new to the list, but I have spent some time searching the
archives and the internet at large for an answer to my question.

Background:

I run Squid with SquidClamAV on an internal machine with one Ethernet
interface – all traffic to and from squid flows over this. This system
is not a gateway, and it cannot be the gateway.

Using this example
http://www.lartc.org/howto/lartc.cookbook.ultimate-tc.html#AEN2233 and
adjusting the settings to 75% of our down and upstream bandwidth, and
to use eth0 works very well for managing web traffic – as expected
downloads get equal bandwidth, and web browsing can continue without
significant slowdowns.

However this also manages everything else on eth0 with these limits
which is not ideal.
What I want to achieve is to have the traffic control applied to
traffic to/from the internet (ports 80 and 443 for example), but not
locally between squid and other machines on the local network. I
assume this could be done with some iptables rules, but I'm not very
experienced with combining iptables and tc together.

Every example I can find assumes you are running squid on the gateway
with two interfaces and using iptables to do “transparent” caching.
This is not what I'm doing.


Thanks,

Lonney.

Re: Managing traffic on an internal Squid box

[Benjamin Kiessling]

[-- Attachment #1: Type: text/plain, Size: 1837 bytes --]

Hi,

On 04/17, Lonney wrote:
> Using this example
> http://www.lartc.org/howto/lartc.cookbook.ultimate-tc.html#AEN2233 and
> adjusting the settings to 75% of our down and upstream bandwidth, and
> to use eth0 works very well for managing web traffic – as expected
> downloads get equal bandwidth, and web browsing can continue without
> significant slowdowns.
> 
> However this also manages everything else on eth0 with these limits
> which is not ideal.
> What I want to achieve is to have the traffic control applied to
> traffic to/from the internet (ports 80 and 443 for example), but not
> locally between squid and other machines on the local network. I
> assume this could be done with some iptables rules, but I'm not very
> experienced with combining iptables and tc together.

The easiest way to achieve this behavior would be to either set no
default class for HTB as traffic which is not classified will traverse
the interface unshaped, e.g.:

	tc qdisc add dev eth0 root handle 1: htb 
	tc class add dev eth0 parent 1: classid 1:1 htb rate 3Mbit
	tc qdisc add dev eth0 parent 1:1 handle 10: sfq

and then matching on the locally generated traffic from squid using
iptables (look for the CLASSIFY target). This works if you can be fairly
sure that other traffic will not swamp out traffic going through HTB.
Otherwise, setting a default class with line speed and another squid
class beneath it will fix this flaw.

Another (significantly more complex) matter is ingress shaping.
Utilizing the ifb device to redirect ingress traffic is the technology
of choice here, but you will have to use tc-filter here as ifb does not
possess the necessary netfilter hooks. Take a look at [0] to get an
example on how to employ ifbs.

Regards,
Ben


[0] http://github.com/westnetz/qos-script 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Managing traffic on an internal Squid box

[Lonney]

Hi Ben,

Thanks for the reply.

I have just managed to get this working how I wanted, based on an
example I found here
http://forums.opensuse.org/english/get-technical-help-here/network-internet/454307-wondershaper-modification-exclude-lan-should-included.html

This is the script I came up with..

http://pastebin.com/6wJ4eVnd

With some quick tests it appears to work as expected.


Lonney.

On 17 April 2013 14:11, Benjamin Kiessling <mittagessen@l.unchti.me> wrote:
> Hi,
>
> On 04/17, Lonney wrote:
>> Using this example
>> http://www.lartc.org/howto/lartc.cookbook.ultimate-tc.html#AEN2233 and
>> adjusting the settings to 75% of our down and upstream bandwidth, and
>> to use eth0 works very well for managing web traffic – as expected
>> downloads get equal bandwidth, and web browsing can continue without
>> significant slowdowns.
>>
>> However this also manages everything else on eth0 with these limits
>> which is not ideal.
>> What I want to achieve is to have the traffic control applied to
>> traffic to/from the internet (ports 80 and 443 for example), but not
>> locally between squid and other machines on the local network. I
>> assume this could be done with some iptables rules, but I'm not very
>> experienced with combining iptables and tc together.
>
> The easiest way to achieve this behavior would be to either set no
> default class for HTB as traffic which is not classified will traverse
> the interface unshaped, e.g.:
>
>         tc qdisc add dev eth0 root handle 1: htb
>         tc class add dev eth0 parent 1: classid 1:1 htb rate 3Mbit
>         tc qdisc add dev eth0 parent 1:1 handle 10: sfq
>
> and then matching on the locally generated traffic from squid using
> iptables (look for the CLASSIFY target). This works if you can be fairly
> sure that other traffic will not swamp out traffic going through HTB.
> Otherwise, setting a default class with line speed and another squid
> class beneath it will fix this flaw.
>
> Another (significantly more complex) matter is ingress shaping.
> Utilizing the ifb device to redirect ingress traffic is the technology
> of choice here, but you will have to use tc-filter here as ifb does not
> possess the necessary netfilter hooks. Take a look at [0] to get an
> example on how to employ ifbs.
>
> Regards,
> Ben
>
>
> [0] http://github.com/westnetz/qos-script

Re: Managing traffic on an internal Squid box

[Andy Furniss]

Lonney wrote:
> Hi Ben,
>
> Thanks for the reply.
>
> I have just managed to get this working how I wanted, based on an
> example I found here
> http://forums.opensuse.org/english/get-technical-help-here/network-internet/454307-wondershaper-modification-exclude-lan-should-included.html
>
> This is the script I came up with..
>
> http://pastebin.com/6wJ4eVnd
>
> With some quick tests it appears to work as expected.

HTB wondershaper is ancient and was written before htb even got into 
kernel IIRC.

It is actually flawed, but the author was never contactable to correct it.

In practice with the filter rules given and sfq on the leafs it will 
usually work, but anything derived/modified from it has the potential to 
fail.

The main issue is -

tc class add dev $DEV parent 1: classid 1:1 htb rate ${UPLINK}kbit burst 6k

# High prio class 1:10:

tc class add dev $DEV parent 1:1 classid 1:10 htb rate ${UPLINK}kbit \
    burst 6k prio 1

# Bulk & default class 1:20 - gets slightly less traffic,
# and a lower priority:

tc class add dev $DEV parent 1:1 classid 1:20 htb rate $[9*$UPLINK/10]kbit \
    burst 6k prio 2

With htb the rate on the parent class does not cap the rates on the 
leafs. so potentially it will go over.

I would change this to (untested so may fail due to my rustyness)

tc class add dev $DEV parent 1:1 classid 1:10 htb rate 
${9*UPLINK/10}kbit ceil ${UPLINK}kbit burst 6k prio 0

tc class add dev $DEV parent 1:1 classid 1:20 htb rate ${UPLINK/10}kbit 
ceil ${UPLINK}kbit burst 6k prio 1

This changes rate for bulk slightly so UPLINK may need reducing.

If you know the exact details of your wan it's possible to set overheads 
so for egress you can push to near 100%. This doesn't wirk for ingress 
as you have to sacrifice bandwidth to have any effect.


IIRC prio 0 is top for htb - it isn't for filters 1 is.


Issue 2 - the use of htb default on eth.

wondershaper was tested on ppp so it didn't matter then, also the sfq on 
the bulk class will save the day.

The issue is that arp will be sent to a low prio class which if sfq were 
omitted/changed to [b]fifo maybe even red there would be potential for 
it to be dropped/delayed too long, which is not good :-)

more minor issues -

quantum 1500 on eth tc sees 1500 byte ip packets as 1514

LAN_SPEED if you really wanted to limit with this set to 100/1000 you 
would have find/set what the overheads are or just back off a bit.

filters -

If dns doesn't get caught by the tos filter giving prio would be nice, 
but again sfq will help it.

The filter that matches acks always seemed a bit odd to me given almost 
every tcp packet has the ack bit set anyway matching it seems only to 
exclude syns etc that should also be prio, 64 may also be smaller than 
some sacks.

The policer -

# Filter *everything* to it (0.0.0.0/0), drop everything that's
# coming in too fast:

tc filter add dev $DEV parent ffff: protocol ip prio 50 u32 match ip src \
    0.0.0.0/0 police rate ${DOWNLINK}kbit burst 10k drop flowid :1

Well luckily the comment is wrong as protocol ip means that you only 
filter ipv4 - which is good.

policers are crude, can be aggressive and won't do any fairness - as 
Benjamin has said ifb can let you shape properly, but you don't get the 
help of iptables.

There have been recent posts regarding policers and gso (generic 
segmentation offload) ethtool -k will show offloads enabled for your 
nic. If you get poor throughput you may need to turn it off or add

mtu 3100

to the policer line after burst 10k.

Re: Managing traffic on an internal Squid box

[Lonney]

Hi Andy,

Thanks for your reply,

Surprising that there are so many problems with the example from
http://www.lartc.org/howto/lartc.cookbook.ultimate-tc.html

I'll try and use your suggestions when I have some more time to put
in-to this project.

Lonney.

On 18 April 2013 03:27, Andy Furniss <andyqos@ukfsn.org> wrote:
> Lonney wrote:
>>
>> Hi Ben,
>>
>> Thanks for the reply.
>>
>> I have just managed to get this working how I wanted, based on an
>> example I found here
>>
>> http://forums.opensuse.org/english/get-technical-help-here/network-internet/454307-wondershaper-modification-exclude-lan-should-included.html
>>
>> This is the script I came up with..
>>
>> http://pastebin.com/6wJ4eVnd
>>
>> With some quick tests it appears to work as expected.
>
>
> HTB wondershaper is ancient and was written before htb even got into kernel
> IIRC.
>
> It is actually flawed, but the author was never contactable to correct it.
>
> In practice with the filter rules given and sfq on the leafs it will usually
> work, but anything derived/modified from it has the potential to fail.
>
> The main issue is -
>
> tc class add dev $DEV parent 1: classid 1:1 htb rate ${UPLINK}kbit burst 6k
>
> # High prio class 1:10:
>
> tc class add dev $DEV parent 1:1 classid 1:10 htb rate ${UPLINK}kbit \
>    burst 6k prio 1
>
> # Bulk & default class 1:20 - gets slightly less traffic,
> # and a lower priority:
>
> tc class add dev $DEV parent 1:1 classid 1:20 htb rate $[9*$UPLINK/10]kbit \
>    burst 6k prio 2
>
> With htb the rate on the parent class does not cap the rates on the leafs.
> so potentially it will go over.
>
> I would change this to (untested so may fail due to my rustyness)
>
> tc class add dev $DEV parent 1:1 classid 1:10 htb rate ${9*UPLINK/10}kbit
> ceil ${UPLINK}kbit burst 6k prio 0
>
> tc class add dev $DEV parent 1:1 classid 1:20 htb rate ${UPLINK/10}kbit ceil
> ${UPLINK}kbit burst 6k prio 1
>
> This changes rate for bulk slightly so UPLINK may need reducing.
>
> If you know the exact details of your wan it's possible to set overheads so
> for egress you can push to near 100%. This doesn't wirk for ingress as you
> have to sacrifice bandwidth to have any effect.
>
>
> IIRC prio 0 is top for htb - it isn't for filters 1 is.
>
>
> Issue 2 - the use of htb default on eth.
>
> wondershaper was tested on ppp so it didn't matter then, also the sfq on the
> bulk class will save the day.
>
> The issue is that arp will be sent to a low prio class which if sfq were
> omitted/changed to [b]fifo maybe even red there would be potential for it to
> be dropped/delayed too long, which is not good :-)
>
> more minor issues -
>
> quantum 1500 on eth tc sees 1500 byte ip packets as 1514
>
> LAN_SPEED if you really wanted to limit with this set to 100/1000 you would
> have find/set what the overheads are or just back off a bit.
>
> filters -
>
> If dns doesn't get caught by the tos filter giving prio would be nice, but
> again sfq will help it.
>
> The filter that matches acks always seemed a bit odd to me given almost
> every tcp packet has the ack bit set anyway matching it seems only to
> exclude syns etc that should also be prio, 64 may also be smaller than some
> sacks.
>
> The policer -
>
> # Filter *everything* to it (0.0.0.0/0), drop everything that's
> # coming in too fast:
>
> tc filter add dev $DEV parent ffff: protocol ip prio 50 u32 match ip src \
>    0.0.0.0/0 police rate ${DOWNLINK}kbit burst 10k drop flowid :1
>
> Well luckily the comment is wrong as protocol ip means that you only filter
> ipv4 - which is good.
>
> policers are crude, can be aggressive and won't do any fairness - as
> Benjamin has said ifb can let you shape properly, but you don't get the help
> of iptables.
>
> There have been recent posts regarding policers and gso (generic
> segmentation offload) ethtool -k will show offloads enabled for your nic. If
> you get poor throughput you may need to turn it off or add
>
> mtu 3100
>
> to the policer line after burst 10k.
>
>
>
>