Creating an equivalent to ESFQ using flow classifier

Creating an equivalent to ESFQ using flow classifier

[Andrew Beverley]

I have a server which I am using to share an ADSL line between a number of
users. I am successfully shaping traffic based on type using 4 different
classes, but I also want to share traffic within those classes evenly
between computers (rather than between connections). I used to use ESFQ for
this, but am trying to move to the new flow classifier as described in
http://www.mail-archive.com/netdev@vger.kernel.org/msg60634.html

I have set up the following rules. However, if I start 4 downloads on one
PC and 1 download on another PC, then the first PC will get 4/5 of the
available bandwidth and the second PC one 1/5. I would expect them to get
half each. I am using HTB to shape between each class.

# Add a HTB qdisc to the root device
tc qdisc add dev eth0 root handle 1: htb

# Rate limit it
tc class add dev eth0 parent 1: classid 1:1 htb rate 2200kbit

# Add 4 classes to it for different traffic
tc class add dev eth0 parent 1:1  classid 1:10 htb \
        rate 100kbit ceil 100kbit prio 0
tc class add dev eth0 parent 1:1  classid 1:30 htb \
        rate 1000kbit ceil 1000kbit prio 1
tc class add dev eth0 parent 1:1  classid 1:40 htb \
        rate 1000kbit ceil 1000kbit prio 2
tc class add dev eth0 parent 1:1  classid 1:60 htb \
        rate 100kbit ceil 100kbit prio 3

# Add SFQ qdisc to each HTB class
tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev eth0 parent 1:30 handle 30: sfq perturb 10
tc qdisc add dev eth0 parent 1:40 handle 40: sfq perturb 10
tc qdisc add dev eth0 parent 1:60 handle 60: sfq perturb 10

# Filter the traffic to each class based on MARK
tc filter add dev eth0 parent 1:0 protocol ip handle 10 fw flowid 1:10
tc filter add dev eth0 parent 1:0 protocol ip handle 30 fw flowid 1:30
tc filter add dev eth0 parent 1:0 protocol ip handle 40 fw flowid 1:40
tc filter add dev eth0 parent 1:0 protocol ip handle 60 fw flowid 1:60

# Share traffic between each PC evenly
tc filter add dev eth0 parent 10: protocol ip handle 10 flow hash keys \
        nfct-src,nfct-dst,proto,nfct-proto-src,nfct-proto-dst divisor 1024
tc filter add dev eth0 parent 30: protocol ip handle 30 flow hash keys \
        nfct-src,nfct-dst,proto,nfct-proto-src,nfct-proto-dst divisor 1024
tc filter add dev eth0 parent 40: protocol ip handle 40 flow hash keys \
        nfct-src,nfct-dst,proto,nfct-proto-src,nfct-proto-dst divisor 1024
tc filter add dev eth0 parent 60: protocol ip handle 60 flow hash keys \
        nfct-src,nfct-dst,proto,nfct-proto-src,nfct-proto-dst divisor 1024


It is the last part that I am unsure about. I do not get any errors with
the commands, but they do not seem to make any difference to the shaping.

I am using linux-2.6.26 on Debian Lenny but have also tried linux-2.6.28

Thanks in advance

Andy Beverley

Re: Creating an equivalent to ESFQ using flow classifier

[Nikolay S. Rybaloff]

Hi Andrew,

That's right, you have configured hashing based on src/dst address +
src/dst port + protocol, which is almost classical SFQ hash. In your
case you should try only nfct-src, which will create queues per source
IP and service these queues fairly within htb class.

В Вск, 15/03/2009 в 16:29 +0000, Andrew Beverley пишет:
> I have a server which I am using to share an ADSL line between a number of
> users. I am successfully shaping traffic based on type using 4 different
> classes, but I also want to share traffic within those classes evenly
> between computers (rather than between connections). I used to use ESFQ for
> this, but am trying to move to the new flow classifier as described in
> http://www.mail-archive.com/netdev@vger.kernel.org/msg60634.html
> 
> I have set up the following rules. However, if I start 4 downloads on one
> PC and 1 download on another PC, then the first PC will get 4/5 of the
> available bandwidth and the second PC one 1/5. I would expect them to get
> half each. I am using HTB to shape between each class.
> 
> # Add a HTB qdisc to the root device
> tc qdisc add dev eth0 root handle 1: htb
> 
> # Rate limit it
> tc class add dev eth0 parent 1: classid 1:1 htb rate 2200kbit
> 
> # Add 4 classes to it for different traffic
> tc class add dev eth0 parent 1:1  classid 1:10 htb \
>         rate 100kbit ceil 100kbit prio 0
> tc class add dev eth0 parent 1:1  classid 1:30 htb \
>         rate 1000kbit ceil 1000kbit prio 1
> tc class add dev eth0 parent 1:1  classid 1:40 htb \
>         rate 1000kbit ceil 1000kbit prio 2
> tc class add dev eth0 parent 1:1  classid 1:60 htb \
>         rate 100kbit ceil 100kbit prio 3
> 
> # Add SFQ qdisc to each HTB class
> tc qdisc add dev eth0 parent 1:10 handle 10: sfq perturb 10
> tc qdisc add dev eth0 parent 1:30 handle 30: sfq perturb 10
> tc qdisc add dev eth0 parent 1:40 handle 40: sfq perturb 10
> tc qdisc add dev eth0 parent 1:60 handle 60: sfq perturb 10
> 
> # Filter the traffic to each class based on MARK
> tc filter add dev eth0 parent 1:0 protocol ip handle 10 fw flowid 1:10
> tc filter add dev eth0 parent 1:0 protocol ip handle 30 fw flowid 1:30
> tc filter add dev eth0 parent 1:0 protocol ip handle 40 fw flowid 1:40
> tc filter add dev eth0 parent 1:0 protocol ip handle 60 fw flowid 1:60
> 
> # Share traffic between each PC evenly
> tc filter add dev eth0 parent 10: protocol ip handle 10 flow hash keys \
>         nfct-src,nfct-dst,proto,nfct-proto-src,nfct-proto-dst divisor 1024
> tc filter add dev eth0 parent 30: protocol ip handle 30 flow hash keys \
>         nfct-src,nfct-dst,proto,nfct-proto-src,nfct-proto-dst divisor 1024
> tc filter add dev eth0 parent 40: protocol ip handle 40 flow hash keys \
>         nfct-src,nfct-dst,proto,nfct-proto-src,nfct-proto-dst divisor 1024
> tc filter add dev eth0 parent 60: protocol ip handle 60 flow hash keys \
>         nfct-src,nfct-dst,proto,nfct-proto-src,nfct-proto-dst divisor 1024
> 
> 
> It is the last part that I am unsure about. I do not get any errors with
> the commands, but they do not seem to make any difference to the shaping.
> 
> I am using linux-2.6.26 on Debian Lenny but have also tried linux-2.6.28
> 
> Thanks in advance
> 
> Andy Beverley
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Creating an equivalent to ESFQ using flow classifier

[Tom Eastep]

[-- Attachment #1: Type: text/plain, Size: 1244 bytes --]

Nikolay S. Rybaloff wrote:

>>
>> It is the last part that I am unsure about. I do not get any errors with
>> the commands, but they do not seem to make any difference to the shaping.

You are fortunate. Every time that I've tried to use the 'flow'
classifier with SFQ, the SFQ qdisc eventually drops 100% of the packets
presented to it.

Example:

tc qdisc add dev wlan0 root handle 1: sfq quantum 1514 limit 127
tc filter add dev wlan0 protocol ip pref 1 parent 1: handle 1 flow \
   hash keys src,dst,proto-src divisor 1024

ursa:~ # tc -s  qdisc ls dev wlan0
qdisc sfq 1: root limit 127p quantum 1514b
 Sent 161724244 bytes 107284 pkt (dropped 200, overlimits 0 requeues 21313)
 rate 0bit 0pps backlog 0b 0p requeues 21313
ursa:~ # tc -s  qdisc ls dev wlan0
qdisc sfq 1: root limit 127p quantum 1514b
 Sent 161724320 bytes 107285 pkt (dropped 205, overlimits 0 requeues 21313)
 rate 0bit 0pps backlog 0b 0p requeues 21313
ursa:~ #

-Tom
-- 
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: Creating an equivalent to ESFQ using flow classifier

[Andrew Beverley]

> That's right, you have configured hashing based on src/dst address +
> src/dst port + protocol, which is almost classical SFQ hash. In your
> case you should try only nfct-src, which will create queues per source
> IP and service these queues fairly within htb class.

Thanks, that makes perfect sense now. I have to admit I didn't really
understand the rules I was writing!

I have tried it with just nfct-src for the ppp0 device going from the
server to the internet, and nfct-dst for the eth0 device going to the
individual machines. This seems to work a lot more as I would expect.

Thanks,

Andy Beverley