From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Peter Huang" Subject: RE: VM outperforming host Date: Thu, 2 Jan 2014 23:27:48 +0800 Message-ID: <52c5857d.e2a1440a.2ee8.4e55@mx.google.com> References: <4265B0A2-6FFD-4FA5-B670-1D6409806331@isi.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: QUOTED-PRINTABLE To: "'John Paul Walters'" , Return-path: Received: from mail-pd0-f169.google.com ([209.85.192.169]:51742 "EHLO mail-pd0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751380AbaABP17 convert rfc822-to-8bit (ORCPT ); Thu, 2 Jan 2014 10:27:59 -0500 Received: by mail-pd0-f169.google.com with SMTP id v10so14317625pde.0 for ; Thu, 02 Jan 2014 07:27:58 -0800 (PST) In-Reply-To: <4265B0A2-6FFD-4FA5-B670-1D6409806331@isi.edu> Content-Language: zh-cn Sender: kvm-owner@vger.kernel.org List-ID: I have experimented a similar problem because of HT, you can try HT dis= abled in BIOS setting. -----Original Message----- =46rom: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On= Behalf Of John Paul Walters Sent: Monday, December 30, 2013 11:59 AM To: kvm@vger.kernel.org Subject: VM outperforming host HI, I=E2=80=99ve been benchmarking of several GPU-enabled applications on b= oth physical hardware and within KVM. To my surprise, I=E2=80=99ve fou= nd a small subset of benchmarks that are able to outperform the host sy= stem by as much as 15% in some cases, and I=E2=80=99m hoping that someo= ne may be able to offer some insight into what might be the cause, or w= here to start looking. The system in question: Host: * Arch Linux with 3.12 kernel * qemu 1.7 * 2 Xeon E5-2670 (total of 16 cores), 48 GB RAM (split evenly over 2 NU= MA nodes) * 1 NVIDIA K20m GPU, with gigabit ethernet networking * 10Gbe and Infiniband adapters, but neither are in use Guest: * CentOS 6.4 with 2.6.32-358.23.2 kernel * 20 GB RAM and 8 physical cores from NUMA node 0 * default networking * K20m GPU using PCIe passthrough The qemu command line: qemu-system-x86_64 -enable-kvm -M q35 -m 20576 -cpu host -smp 8,sockets= =3D1,cores=3D8,threads=3D1 -device ahci,bus=3Dpcie.0,id=3Dahci -bios /u= sr/share/qemu/bios.bin -drive file=3D/root/centos_6.4/centos_flat.img,i= d=3Ddisk,format=3Draw -device ide-hd,bus=3Dahci.0,drive=3Ddisk -vnc 0.0= =2E0.0:1 -redir tcp:52109::22 -device pci-assign,host=3D08:00.0 I=E2=80=99ve ensured that the VM runs entirely within a single NUMA nod= e by creating a cpuset with the appropriate physical cores and memory n= odes. I=E2=80=99ve done the same for the host system tests. I=E2=80=99= ve also loaded the host system with CentOS 6.4 and rerun the same exper= iments, hoping that this issue was related to the host system kernel or= Arch Linux. It wasn=E2=80=99t. =20 So far, I=E2=80=99ve tried disabling unused PCIe devices on the host, h= oping that doing so would speed up the host side experiments, but it di= dn=E2=80=99t. I=E2=80=99ve disabled transparent huge pages after I not= iced that the VM memory appears to be backed by them. This reduced the= performance of the guest slightly, but did not come close to canceling= out the performance gains of the VM. I=E2=80=99ve experimented with s= everal combinations of NUMA-related scheduler options, with virtually n= o effect. Drivers and libraries are identical between host and guest. = Does anyone have any suggestions for tracking down either where I=E2=80= =99m losing performance on the host, or gaining performance in the VM? thanks for any help or suggestions, JP-- To unsubscribe from this list: send the line "unsubscribe kvm" in the b= ody of a message to majordomo@vger.kernel.org More majordomo info at h= ttp://vger.kernel.org/majordomo-info.html