From: Andi Kleen <ak@suse.de>
To: "David S. Miller" <davem@davemloft.net>
Cc: jheffner@psc.edu, netdev@oss.sgi.com
Subject: Re: bad TSO performance in 2.6.9-rc2-BK
Date: Wed, 29 Sep 2004 22:56:35 +0200 [thread overview]
Message-ID: <20040929225635.061f2405.ak@suse.de> (raw)
In-Reply-To: <20040929125644.40358b42.davem@davemloft.net>
On Wed, 29 Sep 2004 12:56:44 -0700
"David S. Miller" <davem@davemloft.net> wrote:
> On Wed, 29 Sep 2004 11:01:03 +0200
> Andi Kleen <ak@suse.de> wrote:
>
> > > Does this help?
> >
> > Possible. I don't have any plans to change the receiver because
> > it works well for most cases.
> >
> > The problem must be still in the sender.
>
> "Must"? So you've proven that the ack-every-22 packets behavior
> is due entirely to the sender?
Just saying the the kernel was tested in a very wide range
of very wide range of workloads and environments and there are no
known TCP issues.
Also the test that John wanted to diable has been in Linux practically
forever (irc dates back to Eric Schenk's work in 2.0)
And other kernels talking to it get ok performance, all the problems
only started with the TSO changes in 2.6.9rc.
>
> If you put 2.6.9-current on both ends and this makes things go smoothly
> that is an important data point and nobody else is seeing the behavior
> you are at the moment.
>
> Personally, all of my bets are on the 2.6.5 frankenstein kernel
> as the culprit :-)
Umm, the TCP stack is pretty vanilla 2.6.5.
Ok, I ran it talking to a 2.6.9rc2bk11+instanto-oops-patchkit
bandaided to avoid oops kernel with a tg3.
Actually I tried to, with TSO on on the sender it doesn't finish the normal
10s netperf standard test even in 30s. Then the sender eventually crashed
even with unit-at-a-time turned off.
And the crash starts looking a bit less like a compiler issue. Does it really
work for you?
Part of the trace. It looks like the new kernel has really
bad problems with acking.
22:45:06.065031 10.23.202.15.32777 > 10.23.202.31.32775: . ack 11585 win 7240 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065042 10.23.202.15.32777 > 10.23.202.31.32775: . ack 13033 win 7964 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065047 10.23.202.15.32777 > 10.23.202.31.32775: . ack 14481 win 8688 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065052 10.23.202.15.32777 > 10.23.202.31.32775: . ack 15929 win 9412 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065061 10.23.202.15.32777 > 10.23.202.31.32775: . ack 17377 win 10136 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065066 10.23.202.15.32777 > 10.23.202.31.32775: . ack 18825 win 10860 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065091 10.23.202.15.32777 > 10.23.202.31.32775: . ack 20273 win 11584 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065254 10.23.202.31.32775 > 10.23.202.15.32777: . 20273:21721(1448) ack 1 win 1460 <nop,nop,timestamp 325947 352118> (DF)
22:45:06.065317 10.23.202.15.32777 > 10.23.202.31.32775: . ack 21721 win 12308 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065322 10.23.202.31.32775 > 10.23.202.15.32777: . 21721:23169(1448) ack 1 win 1460 <nop,nop,timestamp 325947 352118> (DF)
22:45:06.065327 10.23.202.31.32775 > 10.23.202.15.32777: . 23169:24617(1448) ack 1 win 1460 <nop,nop,timestamp 325947 352118> (DF)
22:45:06.065331 10.23.202.31.32775 > 10.23.202.15.32777: . 24617:26065(1448) ack 1 win 1460 <nop,nop,timestamp 325947 352118> (DF)
22:45:06.065335 10.23.202.31.32775 > 10.23.202.15.32777: . 26065:27513(1448) ack 1 win 1460 <nop,nop,timestamp 325947 352118> (DF)
22:45:06.065339 10.23.202.31.32775 > 10.23.202.15.32777: P 27513:28961(1448) ack 1 win 1460 <nop,nop,timestamp 325947 352118> (DF)
22:45:06.065348 10.23.202.15.32777 > 10.23.202.31.32775: . ack 23169 win 13032 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065353 10.23.202.15.32777 > 10.23.202.31.32775: . ack 24617 win 13756 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065358 10.23.202.15.32777 > 10.23.202.31.32775: . ack 26065 win 14480 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065368 10.23.202.15.32777 > 10.23.202.31.32775: . ack 27513 win 15204 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065372 10.23.202.15.32777 > 10.23.202.31.32775: . ack 28961 win 15928 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065529 10.23.202.31.32775 > 10.23.202.15.32777: . 28961:30409(1448) ack 1 win 1460 <nop,nop,timestamp 325947 352118> (DF)
22:45:06.065534 10.23.202.31.32775 > 10.23.202.15.32777: . 30409:31857(1448) ack 1 win 1460 <nop,nop,timestamp 325947 352118> (DF)
22:45:06.065540 10.23.202.31.32775 > 10.23.202.15.32777: . 31857:33305(1448) ack 1 win 1460 <nop,nop,timestamp 325947 352118> (DF)
22:45:06.065577 10.23.202.31.32775 > 10.23.202.15.32777: . 33305:34753(1448) ack 1 win 1460 <nop,nop,timestamp 325947 352118> (DF)
22:45:06.065581 10.23.202.31.32775 > 10.23.202.15.32777: . 34753:36201(1448) ack 1 win 1460 <nop,nop,timestamp 325947 352118> (DF)
22:45:06.065585 10.23.202.31.32775 > 10.23.202.15.32777: P 36201:37649(1448) ack 1 win 1460 <nop,nop,timestamp 325947 352118> (DF)
22:45:06.065603 10.23.202.15.32777 > 10.23.202.31.32775: . ack 30409 win 16022 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065608 10.23.202.15.32777 > 10.23.202.31.32775: . ack 31857 win 16022 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065613 10.23.202.15.32777 > 10.23.202.31.32775: . ack 33305 win 16022 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065622 10.23.202.15.32777 > 10.23.202.31.32775: . ack 37649 win 16022 <nop,nop,timestamp 352118 325947> (DF)
22:45:06.065771 10.23.202.31.32775 > 10.23.202.15.32777: . 37649:39097(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352118> (DF)
22:45:06.065778 10.23.202.31.32775 > 10.23.202.15.32777: . 39097:40545(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352118> (DF)
22:45:06.065784 10.23.202.31.32775 > 10.23.202.15.32777: . 40545:41993(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352118> (DF)
22:45:06.065847 10.23.202.31.32775 > 10.23.202.15.32777: . 41993:43441(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352118> (DF)
22:45:06.065851 10.23.202.31.32775 > 10.23.202.15.32777: . 43441:44889(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352118> (DF)
22:45:06.065855 10.23.202.31.32775 > 10.23.202.15.32777: . 44889:46337(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352118> (DF)
22:45:06.065858 10.23.202.31.32775 > 10.23.202.15.32777: . 46337:47785(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352118> (DF)
22:45:06.065862 10.23.202.31.32775 > 10.23.202.15.32777: . 47785:49233(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352118> (DF)
22:45:06.065900 10.23.202.15.32777 > 10.23.202.31.32775: . ack 49233 win 16022 <nop,nop,timestamp 352119 325948> (DF)
22:45:06.066152 10.23.202.31.32775 > 10.23.202.15.32777: . 49233:50681(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352119> (DF)
22:45:06.066159 10.23.202.31.32775 > 10.23.202.15.32777: . 50681:52129(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352119> (DF)
22:45:06.066165 10.23.202.31.32775 > 10.23.202.15.32777: . 52129:53577(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352119> (DF)
22:45:06.066172 10.23.202.31.32775 > 10.23.202.15.32777: . 53577:55025(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352119> (DF)
22:45:06.066208 10.23.202.31.32775 > 10.23.202.15.32777: . 55025:56473(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352119> (DF)
22:45:06.066212 10.23.202.31.32775 > 10.23.202.15.32777: . 56473:57921(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352119> (DF)
22:45:06.066216 10.23.202.31.32775 > 10.23.202.15.32777: . 57921:59369(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352119> (DF)
22:45:06.066219 10.23.202.31.32775 > 10.23.202.15.32777: P 59369:60817(1448) ack 1 win 1460 <nop,nop,timestamp 325948 352119> (DF)
22:45:06.066263 10.23.202.15.32777 > 10.23.202.31.32775: . ack 60817 win 16022 <nop,nop,timestamp 352119 325948> (DF)
-Andi
next prev parent reply other threads:[~2004-09-29 20:56 UTC|newest]
Thread overview: 97+ messages / expand[flat|nested] mbox.gz Atom feed top
2004-09-20 6:30 bad TSO performance in 2.6.9-rc2-BK Anton Blanchard
2004-09-20 15:54 ` Nivedita Singhvi
2004-09-21 15:55 ` Anton Blanchard
2004-09-20 20:30 ` Andi Kleen
2004-09-21 22:58 ` David S. Miller
2004-09-22 14:00 ` Andi Kleen
2004-09-22 18:12 ` David S. Miller
2004-09-22 19:55 ` Andi Kleen
2004-09-22 20:07 ` Nivedita Singhvi
2004-09-22 20:30 ` David S. Miller
2004-09-22 20:56 ` Nivedita Singhvi
2004-09-22 21:56 ` Andi Kleen
2004-09-22 22:04 ` David S. Miller
2004-09-22 20:12 ` Andrew Grover
2004-09-22 20:39 ` David S. Miller
2004-09-22 22:06 ` Andi Kleen
2004-09-22 22:25 ` David S. Miller
2004-09-22 22:47 ` Andi Kleen
2004-09-22 22:50 ` David S. Miller
2004-09-23 23:11 ` David S. Miller
2004-09-23 23:41 ` Herbert Xu
2004-09-23 23:41 ` David S. Miller
2004-09-24 0:12 ` Herbert Xu
2004-09-24 0:40 ` Herbert Xu
2004-09-24 1:07 ` Herbert Xu
2004-09-24 1:17 ` David S. Miller
2004-09-27 1:27 ` Herbert Xu
2004-09-27 2:50 ` Herbert Xu
2004-09-27 4:00 ` David S. Miller
2004-09-27 5:45 ` Herbert Xu
2004-09-27 19:01 ` David S. Miller
2004-09-27 21:32 ` Herbert Xu
2004-09-28 21:10 ` David S. Miller
2004-09-28 21:34 ` Andi Kleen
2004-09-28 21:53 ` David S. Miller
2004-09-28 22:33 ` Andi Kleen
2004-09-28 22:57 ` David S. Miller
2004-09-28 23:27 ` Andi Kleen
2004-09-28 23:35 ` David S. Miller
2004-09-28 23:55 ` Andi Kleen
2004-09-29 0:04 ` David S. Miller
2004-09-29 20:58 ` John Heffner
2004-09-29 21:10 ` Nivedita Singhvi
2004-09-29 21:50 ` David S. Miller
2004-09-29 21:56 ` Andi Kleen
2004-09-29 23:29 ` David S. Miller
2004-09-29 23:51 ` John Heffner
2004-09-30 0:03 ` David S. Miller
2004-09-30 0:10 ` Herbert Xu
2004-10-01 0:34 ` David S. Miller
2004-10-01 1:12 ` David S. Miller
2004-10-01 3:40 ` David S. Miller
2004-10-01 10:35 ` Andi Kleen
2004-10-01 10:23 ` Andi Kleen
2004-09-30 0:10 ` John Heffner
2004-09-30 17:25 ` John Heffner
2004-09-30 20:23 ` David S. Miller
2004-09-30 0:05 ` Herbert Xu
2004-09-30 4:33 ` David S. Miller
2004-09-30 5:47 ` Herbert Xu
2004-09-30 7:39 ` David S. Miller
2004-09-30 8:09 ` Herbert Xu
2004-09-30 9:29 ` Andi Kleen
2004-09-30 20:20 ` David S. Miller
2004-09-29 3:27 ` John Heffner
2004-09-29 9:01 ` Andi Kleen
2004-09-29 19:56 ` David S. Miller
2004-09-29 20:56 ` Andi Kleen [this message]
2004-09-29 21:17 ` David S. Miller
2004-09-29 21:00 ` David S. Miller
2004-09-29 21:16 ` Nivedita Singhvi
2004-09-29 21:22 ` David S. Miller
2004-09-29 21:43 ` Andi Kleen
2004-09-29 21:51 ` John Heffner
2004-09-29 21:52 ` David S. Miller
2004-09-24 8:30 ` Andi Kleen
2004-09-27 22:38 ` John Heffner
2004-09-27 23:04 ` David S. Miller
2004-09-27 23:25 ` Andi Kleen
2004-09-27 23:37 ` David S. Miller
2004-09-27 23:51 ` Andi Kleen
2004-09-28 0:15 ` David S. Miller
2004-09-27 23:36 ` Herbert Xu
2004-09-28 0:13 ` David S. Miller
2004-09-28 0:34 ` Herbert Xu
2004-09-28 4:59 ` David S. Miller
2004-09-28 5:15 ` Herbert Xu
2004-09-28 5:58 ` David S. Miller
2004-09-28 6:45 ` Nivedita Singhvi
2004-09-28 7:20 ` Nivedita Singhvi
2004-09-28 20:38 ` David S. Miller
2004-09-28 7:23 ` Nivedita Singhvi
2004-09-28 8:23 ` Herbert Xu
2004-09-28 12:53 ` John Heffner
2004-09-22 20:28 ` David S. Miller
[not found] <Pine.NEB.4.33.0409301625560.13549-100000@dexter.psc.edu>
2004-10-02 1:32 ` John Heffner
2004-10-04 20:07 ` David S. Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20040929225635.061f2405.ak@suse.de \
--to=ak@suse.de \
--cc=davem@davemloft.net \
--cc=jheffner@psc.edu \
--cc=netdev@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).