From: Niklas Cassel <niklas.cassel@axis.com>
To: Eric Dumazet <eric.dumazet@gmail.com>
Cc: "edumazet@google.com" <edumazet@google.com>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>
Subject: Re: network stream fairness
Date: Mon, 9 Nov 2015 17:50:29 +0100 [thread overview]
Message-ID: <5640CED5.6040103@axis.com> (raw)
In-Reply-To: <1447086271.17135.45.camel@edumazet-glaptop2.roam.corp.google.com>
On 11/09/2015 05:24 PM, Eric Dumazet wrote:
> On Mon, 2015-11-09 at 08:07 -0800, Eric Dumazet wrote:
>
>> Your numbers suggest a cwnd growth then, which might show a CC bug.
>>
>> Please run the following when your iper3 runs on regular 4.3 kernel
>>
>> for i in `seq 1 10`
>> do
>> ss -temoi dst 192.168.0.141
>> sleep 1
>> done
>>
See output at the end of the mail.
iperf3 output sample:
[ 4] 24.00-24.53 sec 1.92 MBytes 30.2 Mbits/sec
[ 6] 24.00-24.53 sec 3.84 MBytes 60.5 Mbits/sec
[SUM] 24.00-24.53 sec 5.76 MBytes 90.7 Mbits/sec
>
> Another thing to try is to change tcp_limit_output_bytes back to 131072
I tried both
sysctl -w net.ipv4.tcp_limit_output_bytes=131072
sysctl -w net.ipv4.tcp_limit_output_bytes=13107
Neither appears to solve the problem.
I don't know why but
echo 3000 > /sys/class/net/eth0/queues/tx-0/byte_queue_limits/limit_max
solves it. Feels weird that setting tcp_limit_output_bytes doesn't fix it.
>
> Of course, your driver be the problem as well, but I am guessing it is
> not upstream ?
It's not upstreamed, but it is GPL, so I could probably upload it somewhere
if that helps.
for i in `seq 1 20`; do ss -temoi dst 192.168.0.141; sleep 1; done
State Recv-Q Send-Q Local Address:Port Peer Address:Port
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 106 192.168.0.1:54574 192.168.0.141:5201 timer:(on,190ms,0) rto:0.2 ato:0.04 cwnd:10 qack:14 bidir uid:20283 ino:7902591 sk:ffff880223311800
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 1448 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 1448 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 1448 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 1448 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.0.1:54576 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901837 sk:ffff880143735800
ESTAB 0 0 192.168.0.1:54574 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 qack:10 uid:20283 ino:7902591 sk:ffff880223311800
ESTAB 0 0 192.168.0.1:54578 192.168.0.141:5201 rto:0.2 ato:0.04 cwnd:10 uid:20283 ino:7901838 sk:ffff880143735040
next prev parent reply other threads:[~2015-11-09 16:50 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-09 15:41 network stream fairness Niklas Cassel
2015-11-09 15:50 ` Eric Dumazet
2015-11-09 15:53 ` Niklas Cassel
2015-11-09 16:07 ` Eric Dumazet
2015-11-09 16:24 ` Eric Dumazet
2015-11-09 16:50 ` Niklas Cassel [this message]
2015-11-09 17:23 ` Eric Dumazet
2015-11-10 9:38 ` Niklas Cassel
2015-11-20 15:33 ` Niklas Cassel
2015-11-20 18:16 ` Eric Dumazet
2015-11-25 12:47 ` Niklas Cassel
2015-11-25 13:49 ` Eric Dumazet
2015-11-29 2:41 ` Niklas Cassel
2015-11-29 17:40 ` Eric Dumazet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5640CED5.6040103@axis.com \
--to=niklas.cassel@axis.com \
--cc=edumazet@google.com \
--cc=eric.dumazet@gmail.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).