From: Ben Greear <greearb@candelatech.com>
To: Stephen Hemminger <shemminger@vyatta.com>
Cc: netdev <netdev@vger.kernel.org>
Subject: Re: TCP funny-ness when over-driving a 1Gbps link.
Date: Thu, 19 May 2011 16:20:45 -0700 [thread overview]
Message-ID: <4DD5A5CD.7040303@candelatech.com> (raw)
In-Reply-To: <20110519161827.2ba4b40e@nehalam>
On 05/19/2011 04:18 PM, Stephen Hemminger wrote:
> On Thu, 19 May 2011 15:47:14 -0700
> Ben Greear<greearb@candelatech.com> wrote:
>
>> I noticed something that struck me as a bit weird today,
>> but perhaps it's normal.
>>
>> I was using our application to create 3 TCP streams from one port to
>> another (1Gbps, igb driver), running through a network emulator.
>> Traffic is flowing bi-directional in each connection.
>>
>> I am doing 24k byte writes per system call. I tried 100ms, 10ms, and 1ms
>> latency (one-way) in the emulator, but behaviour is similar in each case.
>> The rest of this info was gathered with 1ms delay in the emulator.
>>
>> If I ask all 3 connections to run 1Gbps, netstat shows 30+GB in the
>> sending queues and 1+ second latency (user-space to user-space). Aggregate
>> throughput is around 700Mbps in each direction.
>>
>> But, if I ask each of the connections to run at 300Mbps, latency averages
>> 2ms and each connection runs right at 300Mbps (950Mbps or so on the wire).
>>
>> It seems that when you over-drive the link, things back up and perform
>> quite badly over-all.
>>
>> This is a core-i7 3.2Ghz with 12GB RAM, Fedora 14, 2.6.38.6 kernel
>> (with some hacks), 64-bit OS and user-space app. Quick testing on 2.6.36.3
>> showed similar results, so I don't think it's a regression.
>>
>> I am curious if others see similar results?
>>
>> Thanks,
>> Ben
>>
>
> If you overdrive, TCP expects your network emulator to have
> a some but limited queueing (like a real router).
The emulator is fine, it's not being over-driven (and has limited queueing if it was
being over-driven). The queues that are backing up are in the tcp sockets on the
sending machine.
But, just to make sure, I'll re-run the test with a looped back cable...
Ben
>
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
next prev parent reply other threads:[~2011-05-19 23:20 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-05-19 22:47 TCP funny-ness when over-driving a 1Gbps link Ben Greear
2011-05-19 23:18 ` Stephen Hemminger
2011-05-19 23:20 ` Ben Greear [this message]
2011-05-19 23:42 ` Ben Greear
2011-05-20 0:05 ` Rick Jones
2011-05-20 0:12 ` Ben Greear
2011-05-20 0:24 ` Rick Jones
2011-05-20 0:37 ` Ben Greear
2011-05-20 0:46 ` Rick Jones
2011-05-20 3:39 ` Ben Greear
2011-05-20 21:33 ` TCP funny-ness when over-driving a 1Gbps link (and wifi) Ben Greear
2011-05-26 15:28 ` Chris Friesen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4DD5A5CD.7040303@candelatech.com \
--to=greearb@candelatech.com \
--cc=netdev@vger.kernel.org \
--cc=shemminger@vyatta.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).