From: Bart Alewijnse <scarfboy@gmail.com>
To: netdev@oss.sgi.com, linux-kernel@vger.kernel.org
Subject: Re: gigabit trouble
Date: Sun, 1 Aug 2004 21:03:32 +0200 [thread overview]
Message-ID: <b71082d804080112031621e041@mail.gmail.com> (raw)
In-Reply-To: <20040731231836.A31121@electric-eye.fr.zoreil.com>
[-- Attachment #1: Type: text/plain, Size: 3253 bytes --]
On Sat, 31 Jul 2004 23:18:36 +0200, Francois Romieu
<romieu@fr.zoreil.com> wrote:
> > Right, both computers now run 2.6.8-rc2-mm1. It's better. Roughly
> > speaking, the top *benchmark* speeds, rouding slightly up, are:
> > udp: 33MB/s (using ~26Kints) one way, 12MB/s (9Kints/s) the other,
> > tcp: 22MB/s (16Kints)one way, 12MB/s (9Kints/s) the other.
> >
> > (The 9Kints ca be 12 when the packet size is 1K or 2K)
>
> Can you give a try to a napi version of the module (it is a compile-time
> option) ?
> You may want higher {r/w}mem_max as well as renicing ksoftirqd with
> a strongly < 0 value on the celeron client.
That's with NAPI on both. I notice r/wmem has an effect, but it's not much.
Although on my new computer it seems moot as there's something weird
going on with interrupts with my botherboard.
> > Oddly enough, the slow direction is sending from my new computer
> > (XP1700, KT333, 1GB ram) to my old (both running linux and the
> > mentioned kernel). I'm fairly sure this is due to the fact that my
>
> The r8169 driver has been reported to be faster on Rx than on Tx so your
> observation makes sense.
How does that make sense? When one side receives, the other sends, hrm?
They're two identical cards, it should be entirely symmetrical assuming
equal hardware - not faster on the years older hardware.
> [...]
> > I guess I'll have to settle for this. I'm just annoyed that the 'giga'
> > is basically a joke, especially on my setup.
>
> It should be possible to make things slightly better for the celeron box
> as a client. I'll cook up a patch.
The celeron box is the server, that's the entire point. Anyhow, it's
much faster in transmission than my new computer right now due to said
mobo problem, so I *want* it as the server.
> Would you be kind enough to do some ftp transfer test where the celeron
> box retrieves data and send the 'vmstat 1' output of the client during
> the test ?
Sure, but I can tell you right now with reasonable certaintly that my
new computer
won't top 9000interrupts/sec, i.e. 9000 packets per second, and therefore do
the 12MB/s at most, and probably less; interruptwise, my new computer
is the bottleneck, and I'm guessing UDP is faster because TCP is
limited by the amount of packets, or in the other direction ACKs, it
can send per second.
Transfers to the celeron are a relatively pointless measure because
of my new computer being horribly interrupt limited at the moment.
That may have started when I installed the gbit card, but mostly because it
disturbs the mobo's precious and unguessable hardware balance. I'll try figuring
out if I can solve it, but basically it just involves swapping around
cards until it
works better, so that'ld take a while.
Attached are the vmstats for an ftp and nfs transfer, as measured from
my old computer (the nfs and ftp server). Both were going at a pretty
low speed, sub-9000 packets; this may have to do with drive
fragmentation, my hard drives are rather full at the moment.
Also, the logs on the client (my new computer) while sending data to
my old one. These were made at a different time, and I believe the
transfer rate was better (6MB for ftp, 9MB for nfs) than for the
-to-old- logs, which were worse, and varied more.
--Bart
[-- Attachment #2: ftp-put-to-old-computer__lin-mm-to-lin-mm_larger --]
[-- Type: application/octet-stream, Size: 4268 bytes --]
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 0 48532 29648 175464 0 0 101 31 273 169 15 2 80 2
0 0 0 48532 29648 175468 0 0 0 0 1007 9 0 1 99 0
1 0 0 45796 29648 178212 0 0 0 0 2873 35 2 33 65 0
1 0 0 39216 29648 184532 0 0 0 0 5131 36 7 76 17 0
1 0 0 32820 29648 191348 0 0 0 0 4938 52 7 77 16 0
0 1 0 28788 29908 195028 0 0 0 4628 3276 41 4 47 26 23
1 1 0 24308 29908 199020 0 0 0 8656 4211 40 8 90 0 2
1 0 0 20596 29908 202724 0 0 0 6500 3964 257 6 94 0 0
1 0 0 13940 29908 209404 0 0 0 0 5994 14 8 92 0 0
1 0 0 7216 29908 216116 0 0 0 8 5996 12 6 94 0 0
1 0 0 2408 29840 221084 0 0 0 8 5931 28 8 92 0 0
1 0 0 2856 29740 221320 0 0 0 1184 5574 51 5 95 0 0
1 1 0 2536 30400 221400 0 0 0 6524 4886 90 6 94 0 0
1 2 0 2856 30276 221580 0 0 0 7780 4279 73 7 93 0 0
1 2 0 2408 30248 222368 0 0 0 6368 4251 73 5 95 0 0
1 2 0 2244 29904 223260 0 0 0 6344 4952 68 6 94 0 0
1 2 0 3116 29636 222960 0 0 0 5244 4163 81 8 92 0 0
1 1 0 2472 29528 223968 0 0 0 5664 4731 101 7 93 0 0
1 0 0 2920 29416 224020 0 0 0 1268 5124 194 6 94 0 0
1 0 0 2984 29132 224628 0 0 0 636 5825 40 8 92 0 0
1 0 0 2280 28944 225796 0 0 0 6668 4807 73 6 94 0 0
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
2 0 0 2260 28668 226352 0 0 0 3976 4973 77 7 93 0 0
1 0 0 2408 28420 226664 0 0 0 5320 4514 96 6 94 0 0
1 1 0 2664 28368 226480 0 0 0 19264 3261 44 6 94 0 0
1 1 0 2476 28288 226732 0 0 0 16184 2124 252 7 93 0 0
1 0 0 2920 28056 226968 0 0 0 536 3898 67 7 93 0 0
1 0 0 2804 27696 227700 0 0 0 8 5863 53 6 94 0 0
1 0 0 2984 27504 227928 0 0 0 8 5929 70 7 93 0 0
1 0 0 2920 27008 228648 0 0 0 4 5949 53 9 91 0 0
1 0 0 2408 26840 229468 0 0 0 1436 5543 50 6 94 0 0
2 0 0 2792 26808 228560 0 0 0 35532 3685 46 8 92 0 0
1 1 0 3064 26716 228916 0 0 0 0 2085 217 8 92 0 0
1 0 0 2984 26520 229572 0 0 0 160 3751 69 6 94 0 0
1 0 0 2536 26352 230324 0 0 0 8 5953 50 8 92 0 0
1 0 0 2920 24116 232332 0 0 0 8 5894 53 7 93 0 0
1 0 0 3048 22420 234180 0 0 0 4 5945 48 6 94 0 0
1 0 0 3112 21064 235396 0 0 0 1348 5476 55 7 91 2 0
1 1 0 1944 20572 236508 0 0 0 35652 3628 52 6 94 0 0
0 2 0 2132 20056 237048 0 0 1064 0 1125 150 7 22 0 70
1 1 0 3188 18984 236408 0 0 224 0 1077 108 61 10 0 29
0 0 0 4616 18984 236408 0 0 0 144 1034 34 2 2 83 13
0 0 0 4616 18984 236408 0 0 0 0 1006 10 0 0 100 0
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 0 178516 18984 63676 0 0 0 0 1009 12 0 33 67 0
0 0 0 177572 20180 63680 0 0 48 1192 1033 73 1 4 70 25
0 0 0 177572 20180 63680 0 0 0 0 1008 10 0 0 100 0
0 0 0 177892 20216 63680 0 0 0 1124 1224 39 0 3 40 57
0 0 0 177892 20216 63684 0 0 0 0 1067 13 0 1 85 14
0 0 0 177892 20220 63688 0 0 8 0 1011 16 0 0 99 1
[-- Attachment #3: nfs-to-old-computer__lin-mm-to-lin-mm --]
[-- Type: application/octet-stream, Size: 4139 bytes --]
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 0 178240 20360 63688 0 0 101 33 276 169 15 2 80 2
0 0 0 178240 20400 63696 0 0 8 68 1019 27 0 1 96 3
0 0 0 178240 20400 63696 0 0 0 0 1006 8 0 0 100 0
0 0 0 178240 20400 63696 0 0 0 0 1008 14 0 0 100 0
0 0 0 178240 20400 63696 0 0 0 0 1004 12 0 0 100 0
0 0 0 178240 20400 63696 0 0 0 0 1005 14 0 0 100 0
0 0 0 178240 20412 63696 0 0 0 20 1011 20 0 0 100 0
0 0 0 178176 20460 63696 0 0 4 72 1024 37 0 0 99 1
1 0 0 171012 20468 70512 0 0 0 0 2589 222 0 75 25 0
1 0 0 162244 20476 79280 0 0 0 0 5497 289 0 100 0 0
1 0 0 153732 20484 88048 0 0 0 0 5618 1214 0 100 0 0
1 1 0 145540 21016 95280 0 0 0 5848 4887 528 1 99 0 0
1 1 0 139268 21020 101552 0 0 0 5276 4414 1260 0 100 0 0
1 2 0 132228 21028 108656 0 0 0 2828 4992 5500 0 100 0 0
1 2 0 125380 21036 115408 0 0 0 4320 4885 6024 0 100 0 0
1 2 0 119044 21040 121840 0 0 0 5760 5352 5829 0 100 0 0
1 2 0 113988 21048 126832 0 0 0 6196 4268 4731 1 99 0 0
2 1 0 109764 21052 130888 0 0 0 5224 4011 3766 0 64 0 36
1 0 0 103876 21056 136812 0 0 0 7876 4831 5618 0 100 0 0
1 0 0 96516 21064 143916 0 0 0 0 5529 6671 0 100 0 0
1 0 0 91332 21068 149068 0 0 0 9348 3804 4864 0 100 0 0
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
2 1 0 82948 21076 157064 0 0 0 6724 6287 7503 0 100 0 0
1 1 0 79364 21356 159848 0 0 0 36220 1967 2719 0 100 0 0
1 1 0 76228 21360 163336 0 0 0 0 2064 3335 0 100 0 0
1 0 0 71620 21364 168224 0 0 0 280 3422 4576 1 99 0 0
1 0 0 67716 21424 172092 0 0 0 10808 3681 3775 0 56 0 44
1 0 0 59396 21432 180284 0 0 0 0 6356 7676 0 100 0 0
1 0 0 51012 21440 188536 0 0 0 0 6379 7739 0 100 0 0
1 0 0 42628 21448 196836 0 0 0 0 6465 7772 0 100 0 0
1 1 0 35460 21532 203256 0 0 0 33336 4836 5986 0 100 0 0
1 1 0 32196 21536 206852 0 0 0 0 2180 3430 0 100 0 0
1 0 0 27908 21540 211252 0 0 0 376 3021 4165 0 100 0 0
1 0 0 19652 21548 219428 0 0 0 4 6154 7661 0 100 0 0
1 0 0 11524 21556 227504 0 0 0 4 6416 7563 0 100 0 0
1 0 0 3128 21564 235772 0 0 0 0 6432 7746 1 99 0 0
3 0 0 2352 17428 240988 0 0 0 3604 5411 5982 0 100 0 0
0 2 0 2816 16080 241064 0 0 0 32860 4643 3364 0 85 0 15
2 1 0 2832 15604 241944 0 0 0 292 2091 2591 0 84 0 16
2 1 0 2600 14836 243504 0 0 0 3640 3358 4167 0 100 0 0
2 0 0 2856 13836 244664 0 0 0 5796 4396 1931 0 100 0 0
0 1 0 2224 13532 245740 0 0 0 4224 2970 2312 0 35 0 65
0 1 0 3060 13404 244828 0 0 0 8028 1660 52 0 5 0 95
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
2 0 0 2736 13380 245708 0 0 0 3148 3310 2317 0 42 0 58
1 0 0 2856 12616 246500 0 0 0 0 6892 5792 0 100 0 0
1 1 0 2732 12396 247544 0 0 0 2624 6287 1131 0 100 0 0
1 1 0 2536 12100 248196 0 0 0 11404 3940 251 0 100 0 0
[-- Attachment #4: clientlog-ftpput --]
[-- Type: application/octet-stream, Size: 3635 bytes --]
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 0 198312 70336 258240 0 0 299 85 1536 767 8 6 78 8
0 0 0 198228 70336 258308 0 0 68 0 1437 463 3 1 94 2
0 0 0 198228 70336 258308 0 0 0 0 2013 1589 11 6 83 0
0 1 0 196652 70388 259820 0 0 1506 92 2329 540 2 3 87 8
0 0 0 190240 70392 266208 0 0 6414 0 6567 808 0 11 84 5
0 0 0 183604 70400 272864 0 0 6662 0 6618 814 0 10 84 6
0 0 0 179236 70404 277212 0 0 4357 0 5303 1305 6 10 81 3
0 0 0 175024 70408 281424 0 0 4228 0 4545 712 1 7 89 3
0 0 0 167832 70428 288136 0 0 6718 20 7369 1026 8 13 73 6
0 0 0 160492 70432 294796 0 0 6663 0 6648 906 0 9 85 5
0 0 0 153236 70440 301384 0 0 6534 0 6612 905 1 11 83 5
0 0 0 145748 70448 308108 0 0 6791 0 6696 926 1 11 81 7
0 0 0 142612 70448 310964 0 0 2818 0 3477 648 1 5 92 2
0 0 0 142612 70460 310952 0 0 0 20 1112 500 0 0 100 0
0 0 0 135956 70468 316996 0 0 6022 0 6129 856 1 10 84 5
0 0 0 128724 70472 323520 0 0 6535 0 6670 888 1 13 80 6
0 0 0 121364 70480 330176 0 0 6662 0 6606 894 0 11 83 6
0 0 0 114004 70488 336832 0 0 6663 0 6678 925 1 12 81 6
0 0 0 107156 70504 342936 0 0 6150 20 6360 867 1 10 82 6
0 0 0 104340 70508 345516 0 0 2562 0 3200 685 1 5 91 3
0 0 0 101396 70508 348168 0 0 2691 0 3338 645 0 3 95 2
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
2 0 0 94164 70516 354756 0 0 6534 0 6698 904 1 11 82 6
0 0 0 86740 70524 361412 0 0 6663 0 6668 935 4 12 77 6
0 0 0 79380 70540 368060 0 0 6662 20 6651 897 1 11 83 5
0 0 0 72020 70548 374716 0 0 6663 0 6685 903 1 11 82 6
0 0 0 66720 70552 379676 0 0 4997 0 5777 871 2 8 85 4
1 0 0 64148 70556 381984 0 0 2306 0 3309 1039 5 6 87 2
0 0 0 60764 70556 385044 0 0 3075 0 4013 631 1 6 91 2
0 0 0 53404 70576 391688 0 0 6662 20 7302 1390 5 15 74 6
1 0 0 46412 70580 398008 0 0 6278 0 6824 1459 6 15 75 4
0 0 0 46412 70580 398008 0 0 0 0 1191 400 1 1 98 0
0 0 0 46412 70580 398008 0 0 0 0 1091 396 0 1 99 0
0 0 0 46412 70580 398008 0 0 0 0 1113 436 1 0 99 0
0 0 0 46412 70592 397996 0 0 0 20 1127 563 2 0 98 0
0 0 0 46412 70592 397996 0 0 0 0 1171 617 3 1 96 0
0 0 0 46428 70592 397996 0 0 0 0 1300 580 3 1 96 0
0 0 0 46428 70592 397996 0 0 0 0 1097 409 1 0 99 0
0 0 0 46428 70592 397996 0 0 0 0 1203 804 5 1 94 0
0 0 0 46300 70640 398016 0 0 0 112 1236 1273 11 3 86 0
0 0 0 46300 70640 398016 0 0 0 0 1568 1574 13 4 83 0
0 0 0 46620 70640 398016 0 0 0 0 2298 1439 12 6 82 0
0 0 0 46648 70640 398016 0 0 0 0 1408 908 5 2 93 0
[-- Attachment #5: clientlog-nfsput --]
[-- Type: application/octet-stream, Size: 4266 bytes --]
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 0 16776 292012 166988 0 0 276 87 1521 770 8 6 78 8
0 0 0 16748 292012 166988 0 0 0 0 1476 416 3 1 96 0
0 0 0 16748 292012 166988 0 0 0 0 2640 1564 12 6 82 0
0 0 0 16748 292012 166988 0 0 0 0 1104 426 1 0 99 0
0 0 0 16748 292012 166988 0 0 0 0 1219 435 1 1 98 0
0 1 0 15356 292052 168308 0 0 744 80 1259 488 1 2 93 4
0 1 0 5036 286580 184524 0 0 9476 0 1284 622 3 14 0 83
0 1 0 4588 275636 198868 0 0 9348 0 1301 664 4 17 0 79
0 1 0 4972 269376 206352 0 0 4625 0 1263 564 1 8 0 91
1 1 0 5248 264268 212208 0 0 3725 0 1495 687 5 8 0 87
1 1 0 5292 238764 243764 0 0 19358 16 2129 1886 16 38 0 47
0 1 0 5164 212332 274888 0 0 19613 8 2160 1023 7 36 0 57
0 1 0 5008 199712 289140 0 0 14739 0 3938 921 6 32 0 62
0 1 0 5344 180980 312020 0 0 18578 0 8116 1149 6 44 0 49
0 1 0 4960 173048 322876 0 0 8852 0 7988 1086 4 25 0 71
0 1 0 5360 163640 334188 0 0 8474 0 6578 1012 5 22 0 73
0 2 0 5288 154676 346280 0 0 9730 4 4463 915 4 23 0 72
0 2 0 5348 145140 358332 0 0 7812 12 7056 1168 4 25 0 71
0 1 0 5368 139152 366564 0 0 5065 0 7798 1062 3 18 0 79
0 1 0 5012 133780 372820 0 0 3855 0 7854 995 3 18 0 79
0 1 0 5012 130208 377004 0 0 2571 0 7841 967 3 14 0 83
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 1 0 5136 128292 379056 0 0 1161 0 7270 2039 12 16 0 72
0 1 0 4760 120656 388392 0 0 5904 0 4059 805 4 15 0 81
0 2 0 5536 111688 397496 0 0 5762 12 4195 899 3 15 0 82
1 1 0 5528 108032 401764 0 0 2721 4 3605 851 3 11 0 86
0 1 0 5276 100952 410204 0 0 5243 0 7641 1045 2 20 0 78
0 1 0 4636 95532 417188 0 0 4289 0 7621 1042 3 15 0 81
0 1 0 5612 90808 421436 0 0 2781 0 7609 1053 2 14 0 84
0 1 0 5276 80724 433696 0 0 7256 4 7748 1093 3 22 0 74
0 0 0 4908 69872 446044 0 0 7533 20 5074 948 4 17 61 17
0 0 0 4908 69872 446044 0 0 0 0 3803 610 0 3 97 0
0 0 0 4908 69872 446044 0 0 0 0 5064 713 1 3 96 0
0 0 0 4972 69872 446044 0 0 0 0 6978 864 0 8 92 0
0 0 0 5164 69872 446044 0 0 0 0 7493 914 0 6 94 0
0 0 0 5164 69884 446032 0 0 0 20 1207 419 1 1 98 0
0 0 0 6380 69884 446032 0 0 0 0 4992 712 0 6 94 0
0 0 0 6508 69884 446032 0 0 0 0 7394 915 0 5 95 0
0 0 0 6572 69884 446032 0 0 0 0 3858 619 1 5 94 0
0 0 0 6636 69884 446100 0 0 12 0 5700 820 1 4 94 1
0 0 0 7600 69928 446056 0 0 0 104 5860 783 1 7 92 0
0 0 0 8116 69928 446056 0 0 0 0 5489 762 3 6 91 0
0 0 0 8440 69928 446056 0 0 0 0 6020 852 1 6 93 0
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 0 8568 69928 446056 0 0 0 0 5433 822 1 5 94 0
0 0 0 10488 69944 446040 0 0 7 0 2099 728 4 6 86 4
0 0 0 10480 69956 446096 0 0 0 20 1224 952 4 1 95 0
0 0 0 10480 69956 446096 0 0 0 0 1283 465 2 1 97 0
0 0 0 199252 69956 257600 0 0 0 0 1779 1300 9 7 84 0
0 0 0 199252 69956 257600 0 0 0 0 1445 1205 19 4 77 0
next prev parent reply other threads:[~2004-08-01 19:03 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <b71082d8040729094537e59a11@mail.gmail.com>
[not found] ` <20040729210401.A32456@electric-eye.fr.zoreil.com>
2004-07-29 22:41 ` gigabit trouble Bart Alewijnse
2004-07-30 15:15 ` Bart Alewijnse
2004-07-30 18:54 ` Francois Romieu
2004-07-30 21:03 ` Bart Alewijnse
2004-07-30 21:41 ` Francois Romieu
2004-07-31 19:51 ` Bart Alewijnse
2004-07-31 21:18 ` Francois Romieu
2004-08-01 19:03 ` Bart Alewijnse [this message]
2004-08-03 2:47 ` Bart Alewijnse
2004-08-03 7:48 ` Francois Romieu
2004-08-03 12:32 ` Bart Alewijnse
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b71082d804080112031621e041@mail.gmail.com \
--to=scarfboy@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).