From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bin Ren Subject: Re: [PATCH] Network Checksum Removal Date: Tue, 24 May 2005 01:38:39 +0100 Message-ID: <8ae7802505052317384cefd041@mail.gmail.com> References: <8ae7802505052314485e898622@mail.gmail.com> Reply-To: bin.ren@cl.cam.ac.uk Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: Content-Disposition: inline List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Rolf Neugebauer Cc: Ian Pratt , xen-devel@lists.xensource.com, Andrew Theurer , Jon Mason List-Id: xen-devel@lists.xenproject.org On 5/24/05, Rolf Neugebauer wrote: > These results are pretty bad. >=20 > What do you get for dom0->external? That definitely should be close or eq= ual > to native. with default BVT, dom->external gets 643Mbps. native gets 744Mbps. > Have you tweaked /proc/sys/net/core/rmem_max? No. I once did Linux tcp tuning on native linux and increased the throughput to around 810Mbps. But it's not very stable and occasionally produced weird behaviors so I turned off tuning on both server and client. > Is the socket buffer set to some large value? Both sender and receiver buffers are 32K. > Are you transmitting/receiving enough data? Each tests last 50 seconds, transmitting around 3g data. >=20 > I don't know netperf but for ttcp I would normally do: >=20 > echo 1048576 > /proc/sys/net/core/rmem_max > ttcp -b 65536 (or similar) ... > And then transmit a few gigabytes >=20 > What's the interrupt rate etc. Haven't noticed yet. I'll get you the number tomorrow. What currently I'm really really obssessed is (1) dom1->external with default BVT gives only ~400Mbps (2) dom1->external with my EEVDF scheduler (everything else is exactly the same) gives 610Mbps, very close to dom0->external. With scheduler latency histograms, it seems to be caused by *far too frequent* context switches in BVT. I'm still digging. Thanks a lot, Bin >=20 > Rolf >=20 >=20 > On 23/5/05 10:48 pm, "Bin Ren" wrote: >=20 > > On 5/23/05, Nivedita Singhvi wrote: > >> Bin Ren wrote: > >>> I've added the support for ethtools. By turning on and off netfront > >>> checksum offloading, I'm getting the following throughput numbers, > >>> using iperf. Each test was run three times. CPU usages are quite > >>> similar in two cases ('top' output). Looks like checksum computation > >>> is not a major overhead in domU networking. > >>> > >>> dom0/1/2 all have 128M memory. dom0 has e1000 tx checksum offloading = turned > >>> on. > >> > >> Yeah, if you want to do anything network intensive, 128MB is just > >> not enough - you really need more memory in your system. > > > > I've given all the domains 256M memory and switched to netperf > > TCP_STREAM (netperf -H server). almost no change. Details: > > > > dom1->external: 420Mbps > > dom1->dom0: 437Mbps > > dom0->dom1: 200Mbps (!!!) > > dom1->dom2: 327Mbps > > > >> > >>> With Tx checksum on: > >>> > >>> dom1->dom2: 300Mb/s (dom0 cpu maxed out by software interrupts) > >>> dom1->dom0: 459Mb/s (dom0 cpu 80% in SI, dom1 cpu 20% in SI) > >>> dom1->external: 439Mb/s (over 1Gb/s ethernet) (dom0 cpu 50% in SI, > >>> dom1 60% in SI) > >>> > >>> With Tx checksum off: > >>> > >>> dom1->dom2: 301Mb/s > >>> dom1->dom0: 454Mb/s > >>> dom1->externel: 437Mb/s (over 1Gb/s ethernet) > >> > >> > >> iperf is a directional send test, correct? > >> i.e. is dom1 -> dom0 perf the same as dom0 -> dom1 for you? > > > > Please see above. > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel >=20 >