Re: 2.2.18Pre Lan Performance Rocks!

Jeff V. Merkey
2000-10-31T22:19:47Z

Larry,

The quality of the networking code in Linux is quite excellent. There's
some scaling problems relative to NetWare. We are firmly committed to
getting something out with a Linux code base and NetWare metrics. Love
to have your help.

Jeff

Larry McVoy wrote:
>
> {lots of perf stuff deleted}
>
> I'm posting this to point out that Linux networking is getting better at
> a substantial pace.
>
> I've already sent this to Davem and Linus a while back, but I have a
> pretty nice lab here at BitMover, 4 100Mbit switched networks, servers
> with 4 cards, and enough clients to generate load. I actually have
> two servers both of which have a NIC on each network; one server has
> .2.15pre9 on it and the other has 2.4.0-test5 on it.
>
> I don't have a lot of spare time, but if you are one of the kernel
> developers and you have tests you want run, contact me privately.
>
> I ran some tests to see how things have changed. What follows are the
> details, the short summary is that 2.4 looks to me to be about 2x better
> in both latency and bandwidth, no mean feat. I'm very impressed with
> this, and I'm especially tickled to see the hand that Dave has had in
> this, he's really come into his own as a senior kernel hacker. I'm sure
> he doesn't need me to stroke his ego, but I'm doing it anyways because
> I'm proud of him (with no disrespect to the many other people who have
> worked on this intended).
>
> So here's what I did. I fired up the lat_tcp and bw_tcp servers from
> lmbench on the server and then generated load from all the clients.
> I noodled around until I found the right mix which gave the best numbers
> and that's roughly what is reported below. I don't have the 2.2 numbers
> handy but I can get them if you care, it was very close to 2x worse,
> like about 1.9x or so.
>
> The server is running Linux 2.4 test9, I believe. It has 3 Intel EEpro's
> and one 3c905B. It's a Ghz K7.
>
> Ethernet controller: Intel Corporation 82557 [Ethernet Pro 100] (rev 8).
> Ethernet controller: Intel Corporation 82557 [Ethernet Pro 100] (#2) (rev 8).
> Ethernet controller: Intel Corporation 82557 [Ethernet Pro 100] (#3) (rev 8).
> Ethernet controller: 3Com Corporation 3c905B 100BaseTX [Cyclone] (rev 48).
>
> All are going into Netgear Fs308 8 port switches. There are 13 clients,
> mostly Intel Linux boxes, but various others as well, let me know if
> you care. A couple of the clients were behind two levels of switches
> (I have 6 here).
>
> Run a single copy of lat_tcp on each client against the server, we see:
> load free cach swap pgin pgou dk0 dk1 dk2 dk3 ipkt opkt int ctx usr sys idl
> 4.68 443M 21M 0 0 0 0 0 0 0 42K 39K 55K 46K 4 96 0
> 4.68 443M 21M 0 0 2.0K 0 0 0 0 40K 38K 55K 44K 2 98 0
> 4.68 443M 21M 0 0 0 0 0 0 0 40K 38K 55K 44K 3 97 0
> 4.55 443M 21M 0 0 0 0 0 0 0 42K 40K 54K 48K 4 96 0
> 4.55 443M 21M 0 0 0 0 0 0 0 41K 39K 54K 45K 3 97 0
> 4.50 443M 21M 0 0 0 0 0 0 0 40K 38K 54K 44K 2 98 0
> 4.50 443M 21M 0 0 0 0 0 0 0 41K 38K 55K 44K 3 97 0
> 4.50 443M 21M 0 0 0 0 0 0 0 41K 41K 54K 45K 7 93 0
> 4.86 443M 21M 0 0 0 0 0 0 0 38K 38K 54K 44K 3 97 0
>
> OK, now bandwidth. Each client is capable of getting at least 11MB/sec from
> the server when run one at a time. I ran just 4 clients, one per network.
>
> load free cach swap pgin pgou dk0 dk1 dk2 dk3 ipkt opkt int ctx usr sys idl
> 0.28 444M 22M 0 0 0 0 0 0 0 14K 27K 15K 2.9K 2 55 43
> 0.28 444M 22M 0 0 0 0 0 0 0 14K 29K 16K 3.1K 2 66 32
> 0.26 444M 22M 0 0 0 0 0 0 0 14K 29K 16K 3.0K 1 67 32
> 0.26 444M 22M 0 0 0 0 0 0 0 15K 29K 16K 3.0K 1 65 34
> 0.24 444M 22M 0 0 0 0 0 0 0 15K 29K 16K 3.0K 0 70 30
> 0.24 444M 22M 0 0 0 0 0 0 0 15K 29K 16K 3.0K 0 63 37
> 0.24 444M 22M 0 0 0 0 0 0 0 14K 28K 16K 3.0K 1 62 37
> 0.22 444M 22M 0 2.0K 0 0 0 0 0 14K 28K 16K 2.9K 1 65 34
>
> It works out to an average of 10.4MB/sec per client or 41.6MB/sec on the
> server on a PCI/32 @ 33Mhz bus. Same Ghz server. Note the idle cycles,
> bandwidth is a lot easier than latency.
>
> Hope this is useful to someone.
> --
> ---
> Larry McVoy lm at bitmover.com http://www.bitmover.com/lm