From: Eric Dumazet <eric.dumazet@gmail.com>
To: Dave Taht <dave.taht@gmail.com>
Cc: jg@freedesktop.org, Stephen Hemminger <shemminger@vyatta.com>,
Juliusz Chroboczek <jch@pps.jussieu.fr>,
Kathleen Nichols <nichols@pollere.com>,
netdev <netdev@vger.kernel.org>
Subject: Re: BQL + Basic Latency under load results - 100Mbit, GSO/TSO off, pfifo_fast vs SFQ vs QFQ
Date: Mon, 02 Jan 2012 06:27:09 +0100 [thread overview]
Message-ID: <1325482029.2526.26.camel@edumazet-laptop> (raw)
In-Reply-To: <1325480861.2526.15.camel@edumazet-laptop>
Le lundi 02 janvier 2012 à 06:07 +0100, Eric Dumazet a écrit :
> Le lundi 02 janvier 2012 à 05:55 +0100, Eric Dumazet a écrit :
>
> > I tested this patch with a 50 concurrent netperf workload, and indeed
> > this fixes the problem for me.
> >
> > # ping 192.168.20.108
> > PING 192.168.20.108 (192.168.20.108) 56(84) bytes of data.
> > 64 bytes from 192.168.20.108: icmp_req=1 ttl=64 time=0.021 ms
> > 64 bytes from 192.168.20.108: icmp_req=2 ttl=64 time=0.011 ms
> > 64 bytes from 192.168.20.108: icmp_req=3 ttl=64 time=0.011 ms
> > 64 bytes from 192.168.20.108: icmp_req=4 ttl=64 time=0.010 ms
> > 64 bytes from 192.168.20.108: icmp_req=5 ttl=64 time=0.010 ms
> > 64 bytes from 192.168.20.108: icmp_req=6 ttl=64 time=0.010 ms
> >
>
> Oops, pinging a real machine, not myself I get more realistic numbers :)
>
> # ping -c 20 192.168.20.112
> PING 192.168.20.112 (192.168.20.112) 56(84) bytes of data.
> 64 bytes from 192.168.20.112: icmp_req=1 ttl=64 time=0.488 ms
> 64 bytes from 192.168.20.112: icmp_req=2 ttl=64 time=0.214 ms
> 64 bytes from 192.168.20.112: icmp_req=3 ttl=64 time=0.696 ms
> 64 bytes from 192.168.20.112: icmp_req=4 ttl=64 time=0.135 ms
> 64 bytes from 192.168.20.112: icmp_req=5 ttl=64 time=0.110 ms
> 64 bytes from 192.168.20.112: icmp_req=6 ttl=64 time=0.401 ms
> 64 bytes from 192.168.20.112: icmp_req=7 ttl=64 time=0.378 ms
> 64 bytes from 192.168.20.112: icmp_req=8 ttl=64 time=0.384 ms
> 64 bytes from 192.168.20.112: icmp_req=9 ttl=64 time=1.03 ms
> 64 bytes from 192.168.20.112: icmp_req=10 ttl=64 time=0.439 ms
> 64 bytes from 192.168.20.112: icmp_req=11 ttl=64 time=0.126 ms
> 64 bytes from 192.168.20.112: icmp_req=12 ttl=64 time=0.093 ms
> 64 bytes from 192.168.20.112: icmp_req=13 ttl=64 time=0.834 ms
> 64 bytes from 192.168.20.112: icmp_req=14 ttl=64 time=0.696 ms
> 64 bytes from 192.168.20.112: icmp_req=15 ttl=64 time=0.776 ms
> 64 bytes from 192.168.20.112: icmp_req=16 ttl=64 time=0.215 ms
> 64 bytes from 192.168.20.112: icmp_req=17 ttl=64 time=0.262 ms
> 64 bytes from 192.168.20.112: icmp_req=18 ttl=64 time=0.554 ms
> 64 bytes from 192.168.20.112: icmp_req=19 ttl=64 time=0.373 ms
> 64 bytes from 192.168.20.112: icmp_req=20 ttl=64 time=0.666 ms
>
> --- 192.168.20.112 ping statistics ---
> 20 packets transmitted, 20 received, 0% packet loss, time 19000ms
> rtt min/avg/max/mdev = 0.093/0.443/1.035/0.264 ms
>
And after disabling TSO (as you did in your tests) I get this :
(Note my link is Gigabit, so I had to install a HTB shaper to 100Mbit to
mimic your workload)
# ping -c 20 192.168.20.112
PING 192.168.20.112 (192.168.20.112) 56(84) bytes of data.
64 bytes from 192.168.20.112: icmp_req=1 ttl=64 time=0.113 ms
64 bytes from 192.168.20.112: icmp_req=2 ttl=64 time=0.153 ms
64 bytes from 192.168.20.112: icmp_req=3 ttl=64 time=0.092 ms
64 bytes from 192.168.20.112: icmp_req=4 ttl=64 time=0.095 ms
64 bytes from 192.168.20.112: icmp_req=5 ttl=64 time=0.176 ms
64 bytes from 192.168.20.112: icmp_req=6 ttl=64 time=0.159 ms
64 bytes from 192.168.20.112: icmp_req=7 ttl=64 time=0.169 ms
64 bytes from 192.168.20.112: icmp_req=8 ttl=64 time=0.122 ms
64 bytes from 192.168.20.112: icmp_req=9 ttl=64 time=0.148 ms
64 bytes from 192.168.20.112: icmp_req=10 ttl=64 time=0.123 ms
64 bytes from 192.168.20.112: icmp_req=11 ttl=64 time=0.186 ms
64 bytes from 192.168.20.112: icmp_req=12 ttl=64 time=0.210 ms
64 bytes from 192.168.20.112: icmp_req=13 ttl=64 time=0.142 ms
64 bytes from 192.168.20.112: icmp_req=14 ttl=64 time=0.134 ms
64 bytes from 192.168.20.112: icmp_req=15 ttl=64 time=0.092 ms
64 bytes from 192.168.20.112: icmp_req=16 ttl=64 time=0.187 ms
64 bytes from 192.168.20.112: icmp_req=17 ttl=64 time=0.123 ms
64 bytes from 192.168.20.112: icmp_req=18 ttl=64 time=0.159 ms
64 bytes from 192.168.20.112: icmp_req=19 ttl=64 time=0.142 ms
64 bytes from 192.168.20.112: icmp_req=20 ttl=64 time=0.207 ms
--- 192.168.20.112 ping statistics ---
20 packets transmitted, 20 received, 0% packet loss, time 18999ms
rtt min/avg/max/mdev = 0.092/0.146/0.210/0.037 ms
# tc -s -d cl show dev eth3
class htb 1:1 root leaf 10: prio 0 quantum 80000 rate 100000Kbit ceil 100000Kbit
burst 40000b/256 mpu 0b overhead 0b cburst 40000b/256 mpu 0b overhead 0b level 0
Sent 4569893772 bytes 3107171 pkt (dropped 43457, overlimits 0 requeues 0)
rate 75836Kbit 6315pps backlog 0b 127p requeues 0
lended: 1204316 borrowed: 0 giants: 0
tokens: -1892 ctokens: -1892
class sfq 10:1b parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 3028b 2p requeues 0
allot 1520
class sfq 10:25 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:34 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:48 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:4b parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 3028b 2p requeues 0
allot -448
class sfq 10:4c parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:68 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:96 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:a9 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 3028b 2p requeues 0
allot 1520
class sfq 10:c8 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:ca parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:112 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:11a parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:149 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:150 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 3028b 2p requeues 0
allot 1520
class sfq 10:16b parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:171 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:185 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:19f parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:1ac parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:1bb parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:1bf parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 3028b 2p requeues 0
allot 1520
class sfq 10:1dc parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:1e7 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 3028b 2p requeues 0
allot 1520
class sfq 10:1f1 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:201 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:20f parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:216 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:21d parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 3028b 2p requeues 0
allot 1520
class sfq 10:22a parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 3028b 2p requeues 0
allot 1520
class sfq 10:22e parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:263 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:2d4 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:306 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:307 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:344 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:346 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 3028b 2p requeues 0
allot 1520
class sfq 10:34f parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:359 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:36f parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:385 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:3a0 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:3a4 parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 1514b 1p requeues 0
allot 0
class sfq 10:3ac parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 3028b 2p requeues 0
allot 1520
class sfq 10:3ee parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
class sfq 10:3fe parent 10:
(dropped 0, overlimits 0 requeues 0)
backlog 4542b 3p requeues 0
allot 1520
next prev parent reply other threads:[~2012-01-02 5:27 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CAA93jw6r45zBn4p76R9-2UNdp=OM+uiYqPKcKnjqBReR_oh1BA@mail.gmail.com>
[not found] ` <CAA93jw5JjVBQOFL=td55rOpXbUYc=uT_rjeY04O137g_-tBJHw@mail.gmail.com>
[not found] ` <CAA93jw7bLnHtVnZGN5wVPQZc46XT_B--cY1ocUbnm1AS=J_YHw@mail.gmail.com>
2012-01-02 4:33 ` BQL + Basic Latency under load results - 100Mbit, GSO/TSO off, pfifo_fast vs SFQ vs QFQ Eric Dumazet
2012-01-02 4:55 ` Eric Dumazet
2012-01-02 5:07 ` Eric Dumazet
2012-01-02 5:27 ` Eric Dumazet [this message]
2012-01-03 17:52 ` David Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1325482029.2526.26.camel@edumazet-laptop \
--to=eric.dumazet@gmail.com \
--cc=dave.taht@gmail.com \
--cc=jch@pps.jussieu.fr \
--cc=jg@freedesktop.org \
--cc=netdev@vger.kernel.org \
--cc=nichols@pollere.com \
--cc=shemminger@vyatta.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox