From: Jesper Dangaard Brouer <brouer@redhat.com>
To: Eric Dumazet <eric.dumazet@gmail.com>
Cc: "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
"Stephen Hemminger" <stephen@networkplumber.org>,
"Tom Herbert" <therbert@google.com>,
"David Miller" <davem@davemloft.net>,
"Hannes Frederic Sowa" <hannes@stressinduktion.org>,
"Daniel Borkmann" <dborkman@redhat.com>,
"Florian Westphal" <fw@strlen.de>,
"Toke Høiland-Jørgensen" <toke@toke.dk>,
"Dave Taht" <dave.taht@gmail.com>,
brouer@redhat.com
Subject: Re: Qdisc: Measuring Head-of-Line blocking with netperf-wrapper
Date: Tue, 16 Sep 2014 17:56:00 +0200 [thread overview]
Message-ID: <20140916175600.018350fe@redhat.com> (raw)
In-Reply-To: <1410875959.7106.200.camel@edumazet-glaptop2.roam.corp.google.com>
On Tue, 16 Sep 2014 06:59:19 -0700
Eric Dumazet <eric.dumazet@gmail.com> wrote:
> With the TCP usec rtt work I did lately, you'll get more precise results
> from a TCP_RR flow, as Tom and I explained.
Here you go, developed a new test:
http://people.netfilter.org/hawk/qdisc/experiment01/README.txt
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol.conf
https://github.com/netoptimizer/netperf-wrapper/commit/7d0241a78e5
The test includes both a TCP_RR and UDP_RR test that derive the
latency, also kept the ping tests for comparison.
One problem: The NoneXSO test is basically invalid, because I cannot
make it exhaust the bandwidth, see "Total Upload":
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__totals--NoneXSO_net_next.png
Looking at the ping test, there is a clear difference between priority
bands, this just shows that the priority band are working as expected
and the qdisc is backlogged.
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__ping--GSO_net_next.png
E.g. ping test for NoneXSO show it is not backlogged, a broken test:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__ping--NoneXSO_net_next.png
Zooming in on the high priority band, we see how the different
high-prio band measurements are working.
Here for GSO:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__ping_hiprio--GSO_net_next.png
Here for TSO:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__ping_hiprio--TSO_net_next.png
I've created a new graph called "rr_latency" that further zooms in on
the difference between TCP_RR and UDP_RR measurements:
Here for GSO:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__rr_latency--GSO_net_next.png
Here for TSO:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__rr_latency--TSO_net_next.png
A compare graph:
http://people.netfilter.org/hawk/qdisc/experiment01/compare_TSO_vs_GSO__rr_latency.png
I found the interactions a little strange in the above graphs.
Even more strange, I started to play with the ixgbe cleanup interval,
adjusting via cmdline:
sudo ethtool -C eth4 rx-usecs 30
Then the "rr_latency" graph change, significantly lowering the latetency.
Here for GSO:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__rr_latency--rxusecs30_GSO_net_next.png
Here for TSO:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__totals--rxusecs30_TSO_net_next.png
Compare graph for GSO:
http://people.netfilter.org/hawk/qdisc/experiment01/compare_GSO_vs_GSO_with_rxusec30__rr_latency.png
Compare graph for TSO:
http://people.netfilter.org/hawk/qdisc/experiment01/compare_TSO_vs_TSO_with_rxusec30__rr_latency.png
Comparing TSO vs GSO both with rx-usecs 30, which is almost equal.
http://people.netfilter.org/hawk/qdisc/experiment01/compare_TSO_vs_GSO_both_with_rxusec30__rr_latency.png
Checking ping, still follow TCP_RR and UDP_RR, with rx-usecs 30:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__ping_hiprio--rxusecs30_GSO_net_next.png
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__ping_hiprio--rxusecs30_TSO_net_next.png
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Sr. Network Kernel Developer at Red Hat
Author of http://www.iptv-analyzer.org
LinkedIn: http://www.linkedin.com/in/brouer
next prev parent reply other threads:[~2014-09-16 15:56 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-15 16:45 Qdisc: Measuring Head-of-Line blocking with netperf-wrapper Jesper Dangaard Brouer
2014-09-15 17:10 ` Tom Herbert
2014-09-15 17:24 ` Eric Dumazet
2014-09-15 18:55 ` Dave Taht
2014-09-15 19:12 ` Rick Jones
2014-09-16 6:30 ` Jesper Dangaard Brouer
2014-09-16 15:52 ` Tom Herbert
2014-09-16 13:22 ` Jesper Dangaard Brouer
2014-09-16 13:59 ` Eric Dumazet
2014-09-16 15:56 ` Jesper Dangaard Brouer [this message]
2014-09-16 16:08 ` Dave Taht
2014-09-16 16:30 ` Eric Dumazet
2014-09-17 7:39 ` Jesper Dangaard Brouer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140916175600.018350fe@redhat.com \
--to=brouer@redhat.com \
--cc=dave.taht@gmail.com \
--cc=davem@davemloft.net \
--cc=dborkman@redhat.com \
--cc=eric.dumazet@gmail.com \
--cc=fw@strlen.de \
--cc=hannes@stressinduktion.org \
--cc=netdev@vger.kernel.org \
--cc=stephen@networkplumber.org \
--cc=therbert@google.com \
--cc=toke@toke.dk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).