[LARTC] Prioritizing based on HTTP Content-Type header

[LARTC] Prioritizing based on HTTP Content-Type header

[Michael Alger]

I'm setting up a reverse-proxy on a limited-bandwidth pipe. The
system is Debian "etch" on Linux 2.6, using squid as the proxy.

As we've only got 5mbit to play with, what I'd really like to do is
set up priority levels based on the Content-Type of the (outgoing)
response:

  1. text/* gets highest priority (along with
     application/x-javascript).
  2. image/* gets middle priority.
  3. */* gets lowest priority.

Today I tried just using tc, with netfilter's "string" match module
to select matching packets, with limited success: while it does
match the packet containing the response header, additional packets
in the same stream don't retain the fwmark (unsurprisingly).

Does anyone have any ideas of -- or even better, experience with --
a stack which can achieve this? squid's built-in rate limiting
doesn't have the concept of borrowing bandwidth, so that's out.

I'm open to pretty much anything: userspace proxies (either in front
of or replacing squid) are fine.

Another option is simply to "punish" bandwidth hogs: the primary
goal is to ensure downloads of large files don't slow down users
that are browing webpages. Possibly just using SFQ will work for
this, but I'm not sure.

Any suggestions would be appreciated. I'm even open to changing
platform (e.g. FreeBSD), but I'd prefer to stick with Debian as it's
what I'm most comfortable with.
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Prioritizing based on HTTP Content-Type header

[Alexandre J. Correa - Onda Internet]

You can use STRING + CONSAVE modules !!

mark packets...

because string match only "starter packet" ... the others packets from 
the same connection isn´t marked.. consave can track this..

-j CONNMARK --restore-mark
-m string --string 'string' --algo bm -j MARK --set-mark 1
-m string --string 'string2' --algo bm -j MARK --set-mark 2
-m mark --mark 1 -j CONNMARK --save-mark
-m mark --mark 2 -j CONNMARK --save-mark


Michael Alger wrote:
> I'm setting up a reverse-proxy on a limited-bandwidth pipe. The
> system is Debian "etch" on Linux 2.6, using squid as the proxy.
>
> As we've only got 5mbit to play with, what I'd really like to do is
> set up priority levels based on the Content-Type of the (outgoing)
> response:
>
>   1. text/* gets highest priority (along with
>      application/x-javascript).
>   2. image/* gets middle priority.
>   3. */* gets lowest priority.
>
> Today I tried just using tc, with netfilter's "string" match module
> to select matching packets, with limited success: while it does
> match the packet containing the response header, additional packets
> in the same stream don't retain the fwmark (unsurprisingly).
>
> Does anyone have any ideas of -- or even better, experience with --
> a stack which can achieve this? squid's built-in rate limiting
> doesn't have the concept of borrowing bandwidth, so that's out.
>
> I'm open to pretty much anything: userspace proxies (either in front
> of or replacing squid) are fine.
>
> Another option is simply to "punish" bandwidth hogs: the primary
> goal is to ensure downloads of large files don't slow down users
> that are browing webpages. Possibly just using SFQ will work for
> this, but I'm not sure.
>
> Any suggestions would be appreciated. I'm even open to changing
> platform (e.g. FreeBSD), but I'd prefer to stick with Debian as it's
> what I'm most comfortable with.
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
>   


-- 
Sds.

Alexandre J. Correa
Onda Internet
www.ondainternet.com.br
Linux User ID #142329

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Re: [LARTC] Prioritizing based on HTTP Content-Type header

[Michael Alger]

On Tue, Apr 24, 2007 at 06:13:37AM -0300, Alexandre J. Correa - Onda Internet wrote:
> You can use STRING + CONSAVE modules !!
> 
> mark packets...
> 
> because string match only "starter packet" ... the others packets from 
> the same connection isn´t marked.. consave can track this..
> 
> -j CONNMARK --restore-mark
> -m string --string 'string' --algo bm -j MARK --set-mark 1
> -m string --string 'string2' --algo bm -j MARK --set-mark 2
> -m mark --mark 1 -j CONNMARK --save-mark
> -m mark --mark 2 -j CONNMARK --save-mark

I haven't fully tested the shaping setup, but it appears to be
classifying packets correctly.

One limitation is that it can't cope with SSL; fortunately that's
not a current requirement for us, but I probably will need to find a
solution for that at some point.

Anyway, thanks again.
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc