From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56599) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bCTFb-0004Uo-9K for qemu-devel@nongnu.org; Mon, 13 Jun 2016 10:54:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bCTFX-0002tT-31 for qemu-devel@nongnu.org; Mon, 13 Jun 2016 10:54:02 -0400 Received: from mx1.redhat.com ([209.132.183.28]:48235) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bCTFW-0002t0-SQ for qemu-devel@nongnu.org; Mon, 13 Jun 2016 10:53:59 -0400 References: <20160421162348.GA24178@cbox> <57192EED.2040501@suse.de> <20160422100118.GD25288@cbox> <5719F7BC.4090703@suse.de> <20160422101546.GB30824@cbox> From: Auger Eric Message-ID: Date: Mon, 13 Jun 2016 16:53:53 +0200 MIME-Version: 1.0 In-Reply-To: <20160422101546.GB30824@cbox> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] Performance regression using KVM/ARM List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Christoffer Dall , Alexander Graf Cc: Marc Zyngier , Peter Maydell , Paolo Bonzini , qemu-devel@nongnu.org, "Michael S. Tsirkin" , Alex Williamson Hi, Le 22/04/2016 =E0 12:15, Christoffer Dall a =E9crit : > On Fri, Apr 22, 2016 at 12:06:52PM +0200, Alexander Graf wrote: >> On 04/22/2016 12:01 PM, Christoffer Dall wrote: >>> On Thu, Apr 21, 2016 at 09:50:05PM +0200, Alexander Graf wrote: >>>> >>>> On 21.04.16 18:23, Christoffer Dall wrote: >>>>> Hi, >>>>> >>>>> Commit 9fac18f (oslib: allocate PROT_NONE pages on top of RAM, >>>>> 2015-09-10) had the unfortunate side effect that memory slots regis= tered >>>>> with KVM no longer contain a userspace address that is aligned to a= 2M >>>>> boundary, causing the use of THP to fail in the kernel. >>>>> >>>>> I fail to see where in the QEMU code we should be asking for a 2M >>>>> alignment of our memory region. Can someone help pointing me to th= e >>>>> right place to fix this or suggest a patch? >>>>> >>>>> This causes a performance regssion of hackbench on KVM/ARM of about= 62% >>>>> compared to the workload running with THP. >>>>> >>>>> We have verified that this is indeed the cause of the failure by ad= ding >>>>> various prints to QEMU and the kernel, but unfortunatley my QEMU >>>>> knowledge is not sufficient for me to fix it myself. >>>>> >>>>> Any help would be much appreciated! >>>> The code changed quite heavily since I last looked at it, but could = you >>>> please try whether the (untested) patch below makes a difference? >>>> >>>> >>> Unfortunately this doesn't make any difference. It feels to me like >>> we're missing specifying a 2M alignemnt in QEMU somewhere, but I can'= t >>> properly understand the links between the actual allocation, register= ing >>> mem slots with the KVM part of QEMU, and actually setting up KVM user >>> memory regions. >>> >>> What has to happen is that the resulting struct >>> kvm_userspace_memory_region() has the same alignment offset from 2M (= the >>> huge page size) of the ->guest_phys_addr and ->userspace-addr fields. >> >> Well, I would expect that the guest address space is always very big >> aligned - and definitely at least 2MB - so we're safe there. >> >> That means we only need to align the qemu virtual address. There >> used to be a memalign() call for that, but it got replaced with >> direct mmap() and then a lot of code changed on top. Looking at the >> logs, I'm sure Paolo knows the answer though :) >> > Peter just pointed me to a change I remember doing for ARM, so perhaps > this fix is the right one? As shared with Christoffer, the patch below also alters multiple vfio platform device assignment (outcome of a bisect). I guess this relates to the GPA allocation on the platform bus but I need to further investigate. I plan to work on a QEMU fix this week. Thanks Eric >=20 >=20 > diff --git a/util/oslib-posix.c b/util/oslib-posix.c > index d25f671..a36e734 100644 > --- a/util/oslib-posix.c > +++ b/util/oslib-posix.c > @@ -35,7 +35,7 @@ > extern int daemon(int, int); > #endif > =20 > -#if defined(__linux__) && (defined(__x86_64__) || defined(__arm__)) > +#if defined(__linux__) && (defined(__x86_64__) || defined(__arm__)) ||= defined(__aarch64__) > /* Use 2 MiB alignment so transparent hugepages can be used by KVM. > Valgrind does not support alignments larger than 1 MiB, > therefore we need special code which handles running on Valgrind= . */ >=20 >=20 > Thanks, > -Christoffer >=20