All of lore.kernel.org
 help / color / mirror / Atom feed
* Auditing a broken and basic traffic shaping setup - PRIO
@ 2014-12-06 19:32 OmegaPhil
  2014-12-07  4:27 ` Dave Taht
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: OmegaPhil @ 2014-12-06 19:32 UTC (permalink / raw)
  To: lartc

[-- Attachment #1: Type: text/plain, Size: 3980 bytes --]

Disclaimer: I don't do this very often so there is probably a retard
error in here somewhere. I'm not expecting people to do my work for me,
I'm just after a better understanding of the problem so I can get more
control of the situation.

tldr: Custom priomap + iptables TOS isn't sorting packets correctly,
Wireshark won't even filter on TOS...

----

I'm currently attempting to implement a 4 band prio shaper with fq_codel
queues on a 100Mbit connection (Debian Testing server):

======================================================================

tc qdisc add dev eth0 root handle 1: htb default 1
tc class add dev eth0 parent 1:0 classid 1:1 htb rate 12800kibps ceil
12800kibps
tc qdisc add dev eth0 parent 1:1 handle 100: prio bands 4 priomap  1 3 1
3 2 3 2 3 0 3 0 3 1 3 1 3
tc qdisc add dev eth0 parent 100:1 handle 1001: fq_codel
tc qdisc add dev eth0 parent 100:2 handle 1002: fq_codel
tc qdisc add dev eth0 parent 100:3 handle 1003: fq_codel
tc qdisc add dev eth0 parent 100:4 handle 1004: fq_codel

======================================================================

Packets are tagged for the various prio queues via iptables:

======================================================================

# ICMP
$IPTABLES -t mangle -A POSTROUTING -o eth0 -p icmp -j TOS --set-tos
Minimize-Delay

# TCP control packets
$IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags
FIN,SYN,RST,ACK FIN,ACK -j TOS --set-tos Minimize-Delay
$IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags
FIN,SYN,RST,ACK SYN,ACK -j TOS --set-tos Minimize-Delay
$IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags
FIN,SYN,RST,ACK RST,ACK -j TOS --set-tos Minimize-Delay
$IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags
FIN,SYN,RST,ACK RST -j TOS --set-tos Minimize-Delay
$IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --syn -j TOS --set-tos
Minimize-Delay

# TCP ACK packets with no or very little data payload (p2p traffic sets
all packets to ACK packets otherwise, source of size: http://phix.me/dm/)
$IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags
FIN,SYN,RST,ACK ACK -m length --length 40:89 -j TOS --set-tos Minimize-Delay

# Band 2 prioritisation
# Torrenting
$IPTABLES -t mangle -A POSTROUTING -o eth0 -m owner --uid-owner deluge
-j TOS --set-tos Maximize-Throughput

# Band 3 prioritisation
#$IPTABLES -t mangle -A POSTROUTING -o eth0 -m owner --uid-owner user1
-j TOS --set-tos Minimize-Cost
#$IPTABLES -t mangle -A POSTROUTING -o eth0 -m owner --uid-owner user2
-j TOS --set-tos Minimize-Cost

======================================================================

This is based on an otherwise-successful configuration on a local Ubuntu
server that admittedly doesn't originate traffic itself, without a
custom priomap.

The general idea is:

Band 0: High priority TCP packets, Minimize Delay,
Band 1: Normal (nothing targetted here)
Band 2: Torrenting, Maximize Throughput
Band 3: Special programs, Minimize Monetary Cost

When I let the above run, virtually all packets get dumped into band 1,
whereas by far the bulk of the traffic is torrenting - the shaping code
is behaving like iptables isn't tagging the packets properly, however
'iptables -v -L -t mangle' is showing a lot of packets going through the
TOS rules.

I next captured packets and opened up with Wireshark to see what was
going on (it would be nice if I could just capture from the queues
directly but I've found no evidence this is possible), however the
following expressions fail to return anything:

ip.tos
ip.tos==8
ip.tos==0x8

etc with other values - I then moved to ip.dsfield.dscp, this failed in
a different way - ip.dsfield.dscp==2 returned packets with
'Differentiated Services Field: 0x08', ip.dsfield.dscp==2 returned 0x10
- why?

At this point I stopped as I clearly didn't know what I was doing. Any
pointers?

Thanks for any help.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Auditing a broken and basic traffic shaping setup - PRIO
  2014-12-06 19:32 Auditing a broken and basic traffic shaping setup - PRIO OmegaPhil
@ 2014-12-07  4:27 ` Dave Taht
  2014-12-08 18:52 ` OmegaPhil
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: Dave Taht @ 2014-12-07  4:27 UTC (permalink / raw)
  To: lartc

On Sat, Dec 6, 2014 at 11:32 AM, OmegaPhil <OmegaPhil@startmail.com> wrote:
> Disclaimer: I don't do this very often so there is probably a retard
> error in here somewhere. I'm not expecting people to do my work for me,
> I'm just after a better understanding of the problem so I can get more
> control of the situation.

A couple quick notes:

1) strict priority queuing as you do here is generally a hugely bad
idea, as the higher classes can completely starve the rest.

DRR with weights or QFQ with weights are better alternatives, or htb
if you want to strictly rate limit each class. (and been working on
something easier to setup than all that called cake... aint done yet,
if you want patches to test, contact me off list).

Here for example, I ran a netperf-wrapper rrul test, and the EF class
was completely starved.

http://pastebin.com/WaKRDATx

2) ToS as used here, was obsoleted in *1998* by the ietf and replaced
with Diffserv and ECN.

http://en.wikipedia.org/wiki/Type_of_service

CS1 would have been the right thing for minimize-cost in particular.

> tldr: Custom priomap + iptables TOS isn't sorting packets correctly,
> Wireshark won't even filter on TOS...
>
> ----
>
> I'm currently attempting to implement a 4 band prio shaper with fq_codel
> queues on a 100Mbit connection (Debian Testing server):
>
> ===================================
>
> tc qdisc add dev eth0 root handle 1: htb default 1
> tc class add dev eth0 parent 1:0 classid 1:1 htb rate 12800kibps ceil
> 12800kibps
> tc qdisc add dev eth0 parent 1:1 handle 100: prio bands 4 priomap  1 3 1
> 3 2 3 2 3 0 3 0 3 1 3 1 3
> tc qdisc add dev eth0 parent 100:1 handle 1001: fq_codel
> tc qdisc add dev eth0 parent 100:2 handle 1002: fq_codel
> tc qdisc add dev eth0 parent 100:3 handle 1003: fq_codel
> tc qdisc add dev eth0 parent 100:4 handle 1004: fq_codel
>
> ===================================
>
> Packets are tagged for the various prio queues via iptables:
>
> ===================================
>
> # ICMP
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -p icmp -j TOS --set-tos
> Minimize-Delay
>
> # TCP control packets
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags
> FIN,SYN,RST,ACK FIN,ACK -j TOS --set-tos Minimize-Delay
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags
> FIN,SYN,RST,ACK SYN,ACK -j TOS --set-tos Minimize-Delay
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags
> FIN,SYN,RST,ACK RST,ACK -j TOS --set-tos Minimize-Delay
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags
> FIN,SYN,RST,ACK RST -j TOS --set-tos Minimize-Delay
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --syn -j TOS --set-tos
> Minimize-Delay
>
> # TCP ACK packets with no or very little data payload (p2p traffic sets
> all packets to ACK packets otherwise, source of size: http://phix.me/dm/)
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags
> FIN,SYN,RST,ACK ACK -m length --length 40:89 -j TOS --set-tos Minimize-Delay
>
> # Band 2 prioritisation
> # Torrenting
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -m owner --uid-owner deluge
> -j TOS --set-tos Maximize-Throughput
>
> # Band 3 prioritisation
> #$IPTABLES -t mangle -A POSTROUTING -o eth0 -m owner --uid-owner user1
> -j TOS --set-tos Minimize-Cost
> #$IPTABLES -t mangle -A POSTROUTING -o eth0 -m owner --uid-owner user2
> -j TOS --set-tos Minimize-Cost


> ===================================
>
> This is based on an otherwise-successful configuration on a local Ubuntu
> server that admittedly doesn't originate traffic itself, without a
> custom priomap.
>
> The general idea is:
>
> Band 0: High priority TCP packets, Minimize Delay,

By peeing on the markings here you are messing with the intent of the sender.

and there is no need to fiddle with the lower level tcp flags here at all.

fq_codel automagically recognises sparse flows (like tcp syns) and
does the right thing already. IF you are on an asymmetric network you
might want to use fq_codel with a lower quantum so to give acks a
little more priority that way.

> Band 1: Normal (nothing targetted here)
> Band 2: Torrenting, Maximize Throughput

No, this should be Background, CS1. IF you have control over your
torrent clients, most support setting the CS1 bit in their
configuration....

> Band 3: Special programs, Minimize Monetary Cost

totally obsolete bit. dont do that. see ecn.

>
> When I let the above run, virtually all packets get dumped into band 1,
> whereas by far the bulk of the traffic is torrenting - the shaping code
> is behaving like iptables isn't tagging the packets properly, however
> 'iptables -v -L -t mangle' is showing a lot of packets going through the
> TOS rules.
>
> I next captured packets and opened up with Wireshark to see what was
> going on (it would be nice if I could just capture from the queues
> directly but I've found no evidence this is possible), however the
> following expressions fail to return anything:
>
> ip.tos
> ip.tos=8
> ip.tos=0x8
>
> etc with other values - I then moved to ip.dsfield.dscp, this failed in
> a different way - ip.dsfield.dscp=2 returned packets with
> 'Differentiated Services Field: 0x08', ip.dsfield.dscp=2 returned 0x10
> - why?
>
> At this point I stopped as I clearly didn't know what I was doing. Any
> pointers?
>
> Thanks for any help.
>



-- 
Dave Täht

thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Auditing a broken and basic traffic shaping setup - PRIO
  2014-12-06 19:32 Auditing a broken and basic traffic shaping setup - PRIO OmegaPhil
  2014-12-07  4:27 ` Dave Taht
@ 2014-12-08 18:52 ` OmegaPhil
  2014-12-08 19:25 ` Dave Taht
  2015-08-23 19:45 ` OmegaPhil
  3 siblings, 0 replies; 5+ messages in thread
From: OmegaPhil @ 2014-12-08 18:52 UTC (permalink / raw)
  To: lartc

[-- Attachment #1: Type: text/plain, Size: 3538 bytes --]

On 07/12/14 04:27, Dave Taht wrote:
> On Sat, Dec 6, 2014 at 11:32 AM, OmegaPhil <OmegaPhil@startmail.com> wrote:
>> Disclaimer: I don't do this very often so there is probably a retard
>> error in here somewhere. I'm not expecting people to do my work for me,
>> I'm just after a better understanding of the problem so I can get more
>> control of the situation.
> 
> A couple quick notes:
> 
> 1) strict priority queuing as you do here is generally a hugely bad
> idea, as the higher classes can completely starve the rest.
> 
> DRR with weights or QFQ with weights are better alternatives, or htb
> if you want to strictly rate limit each class. (and been working on
> something easier to setup than all that called cake... aint done yet,
> if you want patches to test, contact me off list).
> 
> Here for example, I ran a netperf-wrapper rrul test, and the EF class
> was completely starved.
> 
> http://pastebin.com/WaKRDATx


Thanks for the reply :) For reference all traffic on this server is
mine, and therefore I can do what I want with it.

I do know about strict priority stuff - that is the aim, to make sure
that important packets are not affected by those of less importance
(e.g. so I don't care that there are 100 torrent UDP streams hammering a
connection, my SSH connection always wins and the lag impact of any
other traffic is minimal).

I will read into DRR and QFQ - I originally settled in PRIO because of
the KISS principle, it sounded exactly what I wanted and should be easy
to set up and maintain.

Will email off list - while the remote server wouldn't be a good idea
for testing I do have the local Ubuntu server running the default
priomap that I can test on - cheers.


> 2) ToS as used here, was obsoleted in *1998* by the ietf and replaced
> with Diffserv and ECN.
> 
> http://en.wikipedia.org/wiki/Type_of_service
> 
> CS1 would have been the right thing for minimize-cost in particular.


I know that ToS is old, I thought it was 1:1 with the new Diffserv
stuff, but since you said that I guess not. I will read into it again
(if it isn't then I'll need to look into how iptables is supposed to tag
the packets without --set-tos).


> By peeing on the markings here you are messing with the intent of the sender.
> 
> and there is no need to fiddle with the lower level tcp flags here at all.
> 
> fq_codel automagically recognises sparse flows (like tcp syns) and
> does the right thing already. IF you are on an asymmetric network you
> might want to use fq_codel with a lower quantum so to give acks a
> little more priority that way.

I am the sender, so the ToS stuff is my intent? Its just relevant for
the local prioritisation, while deluge can flag ToS I can't properly
audit it (the Wireshark issue), and the other programs don't have the
functionality.

Right, I see with fq_codel - that was a recent advancement for me,
originally the PRIO children were just normal queues. Good, I'll get rid
of that complexity.


>> Band 1: Normal (nothing targetted here)
>> Band 2: Torrenting, Maximize Throughput
> 
> No, this should be Background, CS1. IF you have control over your
> torrent clients, most support setting the CS1 bit in their
> configuration....
> 
>> Band 3: Special programs, Minimize Monetary Cost
> 
> totally obsolete bit. dont do that. see ecn.

Will read into Diffserv and ECN.

Thanks for your feedback.


-- 
Libre software on Github: https://github.com/OmegaPhil
FSF member #9442


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Auditing a broken and basic traffic shaping setup - PRIO
  2014-12-06 19:32 Auditing a broken and basic traffic shaping setup - PRIO OmegaPhil
  2014-12-07  4:27 ` Dave Taht
  2014-12-08 18:52 ` OmegaPhil
@ 2014-12-08 19:25 ` Dave Taht
  2015-08-23 19:45 ` OmegaPhil
  3 siblings, 0 replies; 5+ messages in thread
From: Dave Taht @ 2014-12-08 19:25 UTC (permalink / raw)
  To: lartc

On Mon, Dec 8, 2014 at 10:52 AM, OmegaPhil <OmegaPhil@startmail.com> wrote:
> On 07/12/14 04:27, Dave Taht wrote:
>> On Sat, Dec 6, 2014 at 11:32 AM, OmegaPhil <OmegaPhil@startmail.com> wrote:
>>> Disclaimer: I don't do this very often so there is probably a retard
>>> error in here somewhere. I'm not expecting people to do my work for me,
>>> I'm just after a better understanding of the problem so I can get more
>>> control of the situation.
>>
>> A couple quick notes:
>>
>> 1) strict priority queuing as you do here is generally a hugely bad
>> idea, as the higher classes can completely starve the rest.
>>
>> DRR with weights or QFQ with weights are better alternatives, or htb
>> if you want to strictly rate limit each class. (and been working on
>> something easier to setup than all that called cake... aint done yet,
>> if you want patches to test, contact me off list).
>>
>> Here for example, I ran a netperf-wrapper rrul test, and the EF class
>> was completely starved.
>>
>> http://pastebin.com/WaKRDATx
>
>
> Thanks for the reply :) For reference all traffic on this server is
> mine, and therefore I can do what I want with it.
>
> I do know about strict priority stuff - that is the aim, to make sure
> that important packets are not affected by those of less importance
> (e.g. so I don't care that there are 100 torrent UDP streams hammering a
> connection, my SSH connection always wins and the lag impact of any
> other traffic is minimal).

I am all in favor of making sure ssh works. Having other forms of traffic
fail completely (instead of just getting slow), is not something I care for.

5% of traffic for other stuff and 95% for ssh is ok (well, I would prefer
ssh tunnels did more of the right thing, and ssh does try to use
Tos markings to do more of the right thing)

/me has switched to mosh entirely for interactive traffic.

>
> I will read into DRR and QFQ - I originally settled in PRIO because of
> the KISS principle, it sounded exactly what I wanted and should be easy
> to set up and maintain.
>
> Will email off list - while the remote server wouldn't be a good idea
> for testing I do have the local Ubuntu server running the default
> priomap that I can test on - cheers.
>
>
>> 2) ToS as used here, was obsoleted in *1998* by the ietf and replaced
>> with Diffserv and ECN.
>>
>> http://en.wikipedia.org/wiki/Type_of_service
>>
>> CS1 would have been the right thing for minimize-cost in particular.
>
>
> I know that ToS is old, I thought it was 1:1 with the new Diffserv
> stuff, but since you said that I guess not. I will read into it again
> (if it isn't then I'll need to look into how iptables is supposed to tag
> the packets without --set-tos).

I would have liked it if there were multiple classes for background
traffic, not just CS1. The other markings (EF, AFxx) all fail the
game theory test. But if the sender wants to be background-ish,
it would be nice to be able to signal that end to end.

CS1 is the only marking (besides ect(0) and ect(1) (ecn)
that comcast preserves e2e, for example.

>
>> By peeing on the markings here you are messing with the intent of the sender.
>>
>> and there is no need to fiddle with the lower level tcp flags here at all.
>>
>> fq_codel automagically recognises sparse flows (like tcp syns) and
>> does the right thing already. IF you are on an asymmetric network you
>> might want to use fq_codel with a lower quantum so to give acks a
>> little more priority that way.


>
> I am the sender, so the ToS stuff is my intent? Its just relevant for
> the local prioritisation, while deluge can flag ToS I can't properly
> audit it (the Wireshark issue), and the other programs don't have the
> functionality.

you might be able to do it by using fw markings rather than messing with
tos.

if you are going to alter tos, try to preserve ecn marks.

> Right, I see with fq_codel - that was a recent advancement for me,
> originally the PRIO children were just normal queues. Good, I'll get rid
> of that complexity.

Yea....

I realize that how SFQ works has confused the issues here, and that is
why we used DRR instead in fq_codel.

I ranted a bit more here....

http://svn.dd-wrt.com/ticket/3366

>
>
>>> Band 1: Normal (nothing targetted here)
>>> Band 2: Torrenting, Maximize Throughput
>>
>> No, this should be Background, CS1. IF you have control over your
>> torrent clients, most support setting the CS1 bit in their
>> configuration....
>>
>>> Band 3: Special programs, Minimize Monetary Cost
>>
>> totally obsolete bit. dont do that. see ecn.
>
> Will read into Diffserv and ECN.
>
> Thanks for your feedback.
>
>
> --
> Libre software on Github: https://github.com/OmegaPhil
> FSF member #9442
>



-- 
Dave Täht

thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Auditing a broken and basic traffic shaping setup - PRIO
  2014-12-06 19:32 Auditing a broken and basic traffic shaping setup - PRIO OmegaPhil
                   ` (2 preceding siblings ...)
  2014-12-08 19:25 ` Dave Taht
@ 2015-08-23 19:45 ` OmegaPhil
  3 siblings, 0 replies; 5+ messages in thread
From: OmegaPhil @ 2015-08-23 19:45 UTC (permalink / raw)
  To: lartc

[-- Attachment #1: Type: text/plain, Size: 7318 bytes --]

On 06/12/14 19:32, OmegaPhil wrote:
> Disclaimer: I don't do this very often so there is probably a retard
> error in here somewhere. I'm not expecting people to do my work for me,
> I'm just after a better understanding of the problem so I can get more
> control of the situation.
> 
> tldr: Custom priomap + iptables TOS isn't sorting packets correctly,
> Wireshark won't even filter on TOS...
> 
> ----
> 
> I'm currently attempting to implement a 4 band prio shaper with fq_codel
> queues on a 100Mbit connection (Debian Testing server):
> 
> ======================================================================
> 
> tc qdisc add dev eth0 root handle 1: htb default 1
> tc class add dev eth0 parent 1:0 classid 1:1 htb rate 12800kibps ceil
> 12800kibps
> tc qdisc add dev eth0 parent 1:1 handle 100: prio bands 4 priomap  1 3 1
> 3 2 3 2 3 0 3 0 3 1 3 1 3
> tc qdisc add dev eth0 parent 100:1 handle 1001: fq_codel
> tc qdisc add dev eth0 parent 100:2 handle 1002: fq_codel
> tc qdisc add dev eth0 parent 100:3 handle 1003: fq_codel
> tc qdisc add dev eth0 parent 100:4 handle 1004: fq_codel
> 
> ======================================================================
> 
> Packets are tagged for the various prio queues via iptables:
> 
> ======================================================================
> 
> # ICMP
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -p icmp -j TOS --set-tos
> Minimize-Delay
> 
> # TCP control packets
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags
> FIN,SYN,RST,ACK FIN,ACK -j TOS --set-tos Minimize-Delay
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags
> FIN,SYN,RST,ACK SYN,ACK -j TOS --set-tos Minimize-Delay
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags
> FIN,SYN,RST,ACK RST,ACK -j TOS --set-tos Minimize-Delay
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags
> FIN,SYN,RST,ACK RST -j TOS --set-tos Minimize-Delay
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --syn -j TOS --set-tos
> Minimize-Delay
> 
> # TCP ACK packets with no or very little data payload (p2p traffic sets
> all packets to ACK packets otherwise, source of size: http://phix.me/dm/)
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags
> FIN,SYN,RST,ACK ACK -m length --length 40:89 -j TOS --set-tos Minimize-Delay
> 
> # Band 2 prioritisation
> # Torrenting
> $IPTABLES -t mangle -A POSTROUTING -o eth0 -m owner --uid-owner deluge
> -j TOS --set-tos Maximize-Throughput
> 
> # Band 3 prioritisation
> #$IPTABLES -t mangle -A POSTROUTING -o eth0 -m owner --uid-owner user1
> -j TOS --set-tos Minimize-Cost
> #$IPTABLES -t mangle -A POSTROUTING -o eth0 -m owner --uid-owner user2
> -j TOS --set-tos Minimize-Cost
> 
> ======================================================================
> 
> This is based on an otherwise-successful configuration on a local Ubuntu
> server that admittedly doesn't originate traffic itself, without a
> custom priomap.
> 
> The general idea is:
> 
> Band 0: High priority TCP packets, Minimize Delay,
> Band 1: Normal (nothing targetted here)
> Band 2: Torrenting, Maximize Throughput
> Band 3: Special programs, Minimize Monetary Cost
> 
> When I let the above run, virtually all packets get dumped into band 1,
> whereas by far the bulk of the traffic is torrenting - the shaping code
> is behaving like iptables isn't tagging the packets properly, however
> 'iptables -v -L -t mangle' is showing a lot of packets going through the
> TOS rules.
> 
> I next captured packets and opened up with Wireshark to see what was
> going on (it would be nice if I could just capture from the queues
> directly but I've found no evidence this is possible), however the
> following expressions fail to return anything:
> 
> ip.tos
> ip.tos==8
> ip.tos==0x8
> 
> etc with other values - I then moved to ip.dsfield.dscp, this failed in
> a different way - ip.dsfield.dscp==2 returned packets with
> 'Differentiated Services Field: 0x08', ip.dsfield.dscp==2 returned 0x10
> - why?
> 
> At this point I stopped as I clearly didn't know what I was doing. Any
> pointers?
> 
> Thanks for any help.


This answering my own question for others that want a simple strict
priority hierarchy with a customisable band count:

I've finally managed to get a custom number of bands PRIO queue on my
server working now (no need to maintain a custom kernel, tc etc) - the
key was to drop the broken TOS classification and just the iptables
CLASSIFY target directly (no need to get involved in complicated tc
filter stuff either):

Aim:

Band 0: SSH traffic
Band 1: 'Normal' traffic, anything unclassified including iroffer
Band 2: Torrent traffic
Band 3: Darknet traffic

Setup 4 band PRIO qdisc:

=======================================================================

tc qdisc add dev eth0 parent root handle 1: prio bands 4 priomap 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1

=======================================================================

Handle must be 1+, it doesn't like 0, you end up with a 8000+ number
that naturally breaks any later references in iptables. Note that the
band number in priomap counts from 0, so the bands are 0, 1, 2 and 3 -
the actual qdisc IDs start from 1 (...). Dumping in band 1 (band 2 qdisc
ID) across the board acts as the default classification.

Setup usual fq_codel qdiscs:

=======================================================================

tc qdisc add dev eth0 parent 1:1 handle 101: fq_codel
tc qdisc add dev eth0 parent 1:2 handle 102: fq_codel
tc qdisc add dev eth0 parent 1:3 handle 103: fq_codel
tc qdisc add dev eth0 parent 1:4 handle 104: fq_codel

=======================================================================

The child PRIO qdiscs associated with your bands have been created for
you already and their ID starts from 1.

Now get iptables to do the classification:

SSH (port 22222 here):

=======================================================================

iptables -t mangle -A POSTROUTING -o eth0 -p tcp -s "$PUBLIC_IP" --sport
22222 -j CLASSIFY --set-class 1:1

=======================================================================

Torrenting:

=======================================================================

iptables -t mangle -A POSTROUTING -o eth0 -m owner --uid-owner deluge -j
CLASSIFY --set-class 1:3

=======================================================================

Darknets:

=======================================================================

iptables -t mangle -A POSTROUTING -o eth0 -m owner --uid-owner
debian-tor -j CLASSIFY --set-class 1:4
iptables -t mangle -A POSTROUTING -o eth0 -m owner --uid-owner i2p -j
CLASSIFY --set-class 1:4

=======================================================================

Everything else ends up in 1:2 as mentioned previously due to the
initial priomap.

For a nice realtime view of how packets are flowing through the qdiscs
to prove things are actually doing what you told them to do, use bmon
(https://github.com/tgraf/bmon) - literally the 'bmon' command, then
move the left white cursor thing up and down to select the interface or
qdisc/class you are interested in.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2015-08-23 19:45 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-12-06 19:32 Auditing a broken and basic traffic shaping setup - PRIO OmegaPhil
2014-12-07  4:27 ` Dave Taht
2014-12-08 18:52 ` OmegaPhil
2014-12-08 19:25 ` Dave Taht
2015-08-23 19:45 ` OmegaPhil

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.