From mboxrd@z Thu Jan 1 00:00:00 1970 From: Cherie Cheung Subject: Network performance - sending from VM to VM using TCP Date: Thu, 26 May 2005 06:24:48 +0800 Message-ID: <4713f859050525152448a0f609@mail.gmail.com> Reply-To: Cherie Cheung Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Return-path: Content-Disposition: inline List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-users-bounces@lists.xensource.com Errors-To: xen-users-bounces@lists.xensource.com To: xen-devel@lists.xensource.com, xen-users@lists.xensource.com List-Id: xen-devel@lists.xenproject.org Hi, I have been simulating a network using dummynet and evaluating it using netperf. Xen3.0-unstable is used and the VMs are vmlinuz-2.6.11-xenU. The simulated link is 300Mbps with 80ms RTT. Using netperf, I sent data using TCP from domain-0 of machine 1 to domain-0 of machine 2. Then I repeat the experiment, but this time from VM-1 of machine 1 to VM-1 of machine 2. However, the performance across the two VMs is substantially worse than that across domain-0. Here's the result: FROM VM to VM: TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to dw10.ucsd.edu (172.19.222.210) port 0 AF_INET Recv Send Send =20 Socket Socket Message Elapsed =20 Size Size Size Time Throughput =20 bytes bytes bytes secs. 10^6bits/sec =20 87380 65536 65536 80.28 24.83=20 FROM domain-0 to domain-0: TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to damp.ucsd.edu (137.110.222.236) port 0 AF_INET Recv Send Send =20 Socket Socket Message Elapsed =20 Size Size Size Time Throughput =20 bytes bytes bytes secs. 10^6bits/sec =20 87380 65536 65536 80.11 280.62=20 Here's the setting of the network buffer: net.core.wmem_max =3D 8388608 net.core.rmem_max =3D 8388608 net.ipv4.tcp_bic =3D 1 net.ipv4.tcp_rmem =3D 4096 87380 8388608 net.ipv4.tcp_wmem =3D 4096 65536 8388608 Does anyone know why the performance across two VMs is so bad? Any fix to it? Thank you. Cherie