From: Jesper Dangaard Brouer <brouer@redhat.com>
To: Eric Dumazet <eric.dumazet@gmail.com>
Cc: "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
"Stephen Hemminger" <stephen@networkplumber.org>,
"Tom Herbert" <therbert@google.com>,
"David Miller" <davem@davemloft.net>,
"Hannes Frederic Sowa" <hannes@stressinduktion.org>,
"Daniel Borkmann" <dborkman@redhat.com>,
"Florian Westphal" <fw@strlen.de>,
"Toke Høiland-Jørgensen" <toke@toke.dk>,
"Dave Taht" <dave.taht@gmail.com>,
brouer@redhat.com
Subject: Re: Qdisc: Measuring Head-of-Line blocking with netperf-wrapper
Date: Tue, 16 Sep 2014 17:56:00 +0200 [thread overview]
Message-ID: <20140916175600.018350fe@redhat.com> (raw)
In-Reply-To: <1410875959.7106.200.camel@edumazet-glaptop2.roam.corp.google.com>
On Tue, 16 Sep 2014 06:59:19 -0700
Eric Dumazet <eric.dumazet@gmail.com> wrote:
> With the TCP usec rtt work I did lately, you'll get more precise results
> from a TCP_RR flow, as Tom and I explained.
Here you go, developed a new test:
http://people.netfilter.org/hawk/qdisc/experiment01/README.txt
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol.conf
https://github.com/netoptimizer/netperf-wrapper/commit/7d0241a78e5
The test includes both a TCP_RR and UDP_RR test that derive the
latency, also kept the ping tests for comparison.
One problem: The NoneXSO test is basically invalid, because I cannot
make it exhaust the bandwidth, see "Total Upload":
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__totals--NoneXSO_net_next.png
Looking at the ping test, there is a clear difference between priority
bands, this just shows that the priority band are working as expected
and the qdisc is backlogged.
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__ping--GSO_net_next.png
E.g. ping test for NoneXSO show it is not backlogged, a broken test:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__ping--NoneXSO_net_next.png
Zooming in on the high priority band, we see how the different
high-prio band measurements are working.
Here for GSO:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__ping_hiprio--GSO_net_next.png
Here for TSO:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__ping_hiprio--TSO_net_next.png
I've created a new graph called "rr_latency" that further zooms in on
the difference between TCP_RR and UDP_RR measurements:
Here for GSO:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__rr_latency--GSO_net_next.png
Here for TSO:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__rr_latency--TSO_net_next.png
A compare graph:
http://people.netfilter.org/hawk/qdisc/experiment01/compare_TSO_vs_GSO__rr_latency.png
I found the interactions a little strange in the above graphs.
Even more strange, I started to play with the ixgbe cleanup interval,
adjusting via cmdline:
sudo ethtool -C eth4 rx-usecs 30
Then the "rr_latency" graph change, significantly lowering the latetency.
Here for GSO:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__rr_latency--rxusecs30_GSO_net_next.png
Here for TSO:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__totals--rxusecs30_TSO_net_next.png
Compare graph for GSO:
http://people.netfilter.org/hawk/qdisc/experiment01/compare_GSO_vs_GSO_with_rxusec30__rr_latency.png
Compare graph for TSO:
http://people.netfilter.org/hawk/qdisc/experiment01/compare_TSO_vs_TSO_with_rxusec30__rr_latency.png
Comparing TSO vs GSO both with rx-usecs 30, which is almost equal.
http://people.netfilter.org/hawk/qdisc/experiment01/compare_TSO_vs_GSO_both_with_rxusec30__rr_latency.png
Checking ping, still follow TCP_RR and UDP_RR, with rx-usecs 30:
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__ping_hiprio--rxusecs30_GSO_net_next.png
http://people.netfilter.org/hawk/qdisc/experiment01/qdisc_prio_hol__ping_hiprio--rxusecs30_TSO_net_next.png
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Sr. Network Kernel Developer at Red Hat
Author of http://www.iptv-analyzer.org
LinkedIn: http://www.linkedin.com/in/brouer
next prev parent reply other threads:[~2014-09-16 15:56 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-15 16:45 Qdisc: Measuring Head-of-Line blocking with netperf-wrapper Jesper Dangaard Brouer
2014-09-15 17:10 ` Tom Herbert
2014-09-15 17:24 ` Eric Dumazet
2014-09-15 18:55 ` Dave Taht
2014-09-15 19:12 ` Rick Jones
2014-09-16 6:30 ` Jesper Dangaard Brouer
2014-09-16 15:52 ` Tom Herbert
2014-09-16 13:22 ` Jesper Dangaard Brouer
2014-09-16 13:59 ` Eric Dumazet
2014-09-16 15:56 ` Jesper Dangaard Brouer [this message]
2014-09-16 16:08 ` Dave Taht
2014-09-16 16:30 ` Eric Dumazet
2014-09-17 7:39 ` Jesper Dangaard Brouer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140916175600.018350fe@redhat.com \
--to=brouer@redhat.com \
--cc=dave.taht@gmail.com \
--cc=davem@davemloft.net \
--cc=dborkman@redhat.com \
--cc=eric.dumazet@gmail.com \
--cc=fw@strlen.de \
--cc=hannes@stressinduktion.org \
--cc=netdev@vger.kernel.org \
--cc=stephen@networkplumber.org \
--cc=therbert@google.com \
--cc=toke@toke.dk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.