From mboxrd@z Thu Jan 1 00:00:00 1970 From: OmegaPhil Date: Sat, 06 Dec 2014 19:32:38 +0000 Subject: Auditing a broken and basic traffic shaping setup - PRIO Message-Id: <548359D6.7030505@startmail.com> MIME-Version: 1 Content-Type: multipart/mixed; boundary="IUN88x5r9bQIu4n2cl6vsnTBuRDKOD2K1" List-Id: To: lartc@vger.kernel.org This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --IUN88x5r9bQIu4n2cl6vsnTBuRDKOD2K1 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Disclaimer: I don't do this very often so there is probably a retard error in here somewhere. I'm not expecting people to do my work for me, I'm just after a better understanding of the problem so I can get more control of the situation. tldr: Custom priomap + iptables TOS isn't sorting packets correctly, Wireshark won't even filter on TOS... ---- I'm currently attempting to implement a 4 band prio shaper with fq_codel queues on a 100Mbit connection (Debian Testing server): =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D tc qdisc add dev eth0 root handle 1: htb default 1 tc class add dev eth0 parent 1:0 classid 1:1 htb rate 12800kibps ceil 12800kibps tc qdisc add dev eth0 parent 1:1 handle 100: prio bands 4 priomap 1 3 1 3 2 3 2 3 0 3 0 3 1 3 1 3 tc qdisc add dev eth0 parent 100:1 handle 1001: fq_codel tc qdisc add dev eth0 parent 100:2 handle 1002: fq_codel tc qdisc add dev eth0 parent 100:3 handle 1003: fq_codel tc qdisc add dev eth0 parent 100:4 handle 1004: fq_codel =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Packets are tagged for the various prio queues via iptables: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D # ICMP $IPTABLES -t mangle -A POSTROUTING -o eth0 -p icmp -j TOS --set-tos Minimize-Delay # TCP control packets $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags FIN,SYN,RST,ACK FIN,ACK -j TOS --set-tos Minimize-Delay $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags FIN,SYN,RST,ACK SYN,ACK -j TOS --set-tos Minimize-Delay $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags FIN,SYN,RST,ACK RST,ACK -j TOS --set-tos Minimize-Delay $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags FIN,SYN,RST,ACK RST -j TOS --set-tos Minimize-Delay $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --syn -j TOS --set-tos Minimize-Delay # TCP ACK packets with no or very little data payload (p2p traffic sets all packets to ACK packets otherwise, source of size: http://phix.me/dm/)= $IPTABLES -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags FIN,SYN,RST,ACK ACK -m length --length 40:89 -j TOS --set-tos Minimize-De= lay # Band 2 prioritisation # Torrenting $IPTABLES -t mangle -A POSTROUTING -o eth0 -m owner --uid-owner deluge -j TOS --set-tos Maximize-Throughput # Band 3 prioritisation #$IPTABLES -t mangle -A POSTROUTING -o eth0 -m owner --uid-owner user1 -j TOS --set-tos Minimize-Cost #$IPTABLES -t mangle -A POSTROUTING -o eth0 -m owner --uid-owner user2 -j TOS --set-tos Minimize-Cost =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D This is based on an otherwise-successful configuration on a local Ubuntu server that admittedly doesn't originate traffic itself, without a custom priomap. The general idea is: Band 0: High priority TCP packets, Minimize Delay, Band 1: Normal (nothing targetted here) Band 2: Torrenting, Maximize Throughput Band 3: Special programs, Minimize Monetary Cost When I let the above run, virtually all packets get dumped into band 1, whereas by far the bulk of the traffic is torrenting - the shaping code is behaving like iptables isn't tagging the packets properly, however 'iptables -v -L -t mangle' is showing a lot of packets going through the TOS rules. I next captured packets and opened up with Wireshark to see what was going on (it would be nice if I could just capture from the queues directly but I've found no evidence this is possible), however the following expressions fail to return anything: ip.tos ip.tos=3D=3D8 ip.tos=3D=3D0x8 etc with other values - I then moved to ip.dsfield.dscp, this failed in a different way - ip.dsfield.dscp=3D=3D2 returned packets with 'Differentiated Services Field: 0x08', ip.dsfield.dscp=3D=3D2 returned 0x= 10 - why? At this point I stopped as I clearly didn't know what I was doing. Any pointers? Thanks for any help. --IUN88x5r9bQIu4n2cl6vsnTBuRDKOD2K1 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBAgAGBQJUg1nXAAoJEBfSPH39wvOPzGMP/RaMFzqGON31pd0JM9NOtTYL eswC9FK0HN2CluAsmcT4B4pOVkN/yMfpZhuDtxOk/7ilmOKxSF3o6Hrh62zv8kcE iknGW8fzT42uB3sOxwGPfOHEZzxKixSI55hhqE7YDmIJ4ZjxvLIV+9Y45JRux4Xv c75DFHfnDg1zHAabzu8EU6sutAbvsI4awoXURTzv7kRTwTnXWqQV0pFOxf4CrSyE 5ew/4/9Koae9im1BYae8hXADq9pmPmTfdCOuhtuSFKj4PsFEc6i2VVPYAIM37uej vqGlwXQGkcbVsRJivN+Uce6O6MMM0Pok6yTgUI4S+K14M6NfnpqDW98kdfwReYre Hrdi1D48EcJwV1GiIsgVmxNc34Tj/luT/VjCUhM1JG8lhATLZScYOzQovWIL2YvB 7irDCtYCWKHyCKfDDcd8fcLczdLKtQjpTqCytnh4hmdVeIitfzXSqr0x9rDK+9ss UZg3Fqtb96DdwAr3ClEEHdZRw8NL7JOeTq8Zd5k4TlDC+o4srXh2oIDGxlLyDdrV cb8mY5P3m84Z8r1t6iR/Q/cdfizok3SZZVEoJKkhQ9oXXprTQ989y/PGi+SNsLc8 9yTsSomZKmhzgD5kZsB7jq6IIdCLN5WePFtZKDPZbAKncLwz0FBfI3RBHBAMc9n2 N4iu0i8LE+xYpCVfPfnQ =1Z8p -----END PGP SIGNATURE----- --IUN88x5r9bQIu4n2cl6vsnTBuRDKOD2K1--