From mboxrd@z Thu Jan 1 00:00:00 1970 From: Peter Lieven Subject: Re: performance trouble Date: Tue, 27 Mar 2012 17:39:10 +0200 Message-ID: <4F71DF1E.8000308@dlh.net> References: <20120222163356.GE26955@nfs-rbx.ovh.net> <201203271343.16866.vrozenfe@redhat.com> <4F71D263.8000601@dlh.net> <201203271737.25952.vrozenfe@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Gleb Natapov , David Cure , Avi Kivity , kvm@vger.kernel.org To: Vadim Rozenfeld Return-path: Received: from ssl.dlh.net ([91.198.192.8]:53295 "EHLO ssl.dlh.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750784Ab2C0PjL (ORCPT ); Tue, 27 Mar 2012 11:39:11 -0400 In-Reply-To: <201203271737.25952.vrozenfe@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: On 27.03.2012 17:37, Vadim Rozenfeld wrote: > On Tuesday, March 27, 2012 04:44:51 PM Peter Lieven wrote: >> On 27.03.2012 13:43, Vadim Rozenfeld wrote: >>> On Tuesday, March 27, 2012 12:49:58 PM Peter Lieven wrote: >>>> On 27.03.2012 12:40, Vadim Rozenfeld wrote: >>>>> On Tuesday, March 27, 2012 11:26:29 AM Peter Lieven wrote: >>>>>> On 27.03.2012 11:23, Vadim Rozenfeld wrote: >>>>>>> On Tuesday, March 27, 2012 10:56:05 AM Gleb Natapov wrote: >>>>>>>> On Mon, Mar 26, 2012 at 10:11:43PM +0200, Vadim Rozenfeld wrote: >>>>>>>>> On Monday, March 26, 2012 08:54:50 PM Peter Lieven wrote: >>>>>>>>>> On 26.03.2012 20:36, Vadim Rozenfeld wrote: >>>>>>>>>>> On Monday, March 26, 2012 07:52:49 PM Gleb Natapov wrote: >>>>>>>>>>>> On Mon, Mar 26, 2012 at 07:46:03PM +0200, Vadim Rozenfeld wrote: >>>>>>>>>>>>> On Monday, March 26, 2012 07:00:32 PM Peter Lieven wrote: >>>>>>>>>>>>>> On 22.03.2012 10:38, Vadim Rozenfeld wrote: >>>>>>>>>>>>>>> On Thursday, March 22, 2012 10:52:42 AM Peter Lieven wrote: >>>>>>>>>>>>>>>> On 22.03.2012 09:48, Vadim Rozenfeld wrote: >>>>>>>>>>>>>>>>> On Thursday, March 22, 2012 09:53:45 AM Gleb Natapov wrote: >>>>>>>>>>>>>>>>>> On Wed, Mar 21, 2012 at 06:31:02PM +0100, Peter Lieven > wrote: >>>>>>>>>>>>>>>>>>> On 21.03.2012 12:10, David Cure wrote: >>>>>>>>>>>>>>>>>>>> hello, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Le Tue, Mar 20, 2012 at 02:38:22PM +0200, Gleb Natapov >>>>>>> ecrivait : >>>>>>>>>>>>>>>>>>>>> Try to add >>>>>>>>>>>>>>>>>>>>> to cpu definition in XML and check command line. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> ok I try this but I can't use to map >>> the >>> >>>>>>>>>>>>>>>>>>>> host cpu >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> (my libvirt is 0.9.8) so I use : >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Opteron_G3 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> (the physical server use Opteron CPU). >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> The log is here : >>>>>>>>>>>>>>>>>>>> http://www.roullier.net/Report/report-3.2-vhost-net-1vcp >>>>>>>>>>>>>>>>>>>> u- cp u.tx t.gz >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> And now with only 1 vcpu, the response time is 8.5s, >>>>>>>>>>>>>>>>>>>> great >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> improvment. We keep this configuration for production : >>>>>>>>>>>>>>>>>>>> we check the response time when some other users are >>>>>>>>>>>>>>>>>>>> connected. >>>>>>>>>>>>>>>>>>> please keep in mind, that setting -hypervisor, disabling >>>>>>>>>>>>>>>>>>> hpet and only one vcpu >>>>>>>>>>>>>>>>>>> makes windows use tsc as clocksource. you have to make >>>>>>>>>>>>>>>>>>> sure, that your vm is not switching between physical >>>>>>>>>>>>>>>>>>> sockets on your system and that you have constant_tsc >>>>>>>>>>>>>>>>>>> feature to have a stable tsc between the cores in the >>>>>>>>>>>>>>>>>>> same socket. its also likely that the vm will crash when >>>>>>>>>>>>>>>>>>> live migrated. >>>>>>>>>>>>>>>>>> All true. I asked to try -hypervisor only to verify where >>>>>>>>>>>>>>>>>> we loose performance. Since you get good result with it >>>>>>>>>>>>>>>>>> frequent access to PM timer is probably the reason. I do >>>>>>>>>>>>>>>>>> not recommend using -hypervisor for production! >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> @gleb: do you know whats the state of in-kernel hyper-v >>>>>>>>>>>>>>>>>>> timers? >>>>>>>>>>>>>>>>>> Vadim is working on it. I'll let him answer. >>>>>>>>>>>>>>>>> It would be nice to have synthetic timers supported. But, >>>>>>>>>>>>>>>>> at the moment, I'm only researching this feature. >>>>>>>>>>>>>>>> So it will take months at least? >>>>>>>>>>>>>>> I would say weeks. >>>>>>>>>>>>>> Is there a way, we could contribute and help you with this? >>>>>>>>>>>>> Hi Peter, >>>>>>>>>>>>> You are welcome to add an appropriate handler. >>>>>>>>>>>> I think Vadim refers to this HV MSR >>>>>>>>>>>> http://msdn.microsoft.com/en-us/library/windows/hardware/ff54263 >>>>>>>>>>>> 3% 28 v=vs .85 %29.aspx >>>>>>>>>>> This one is pretty simple to support. Please see attachments for >>>>>>>>>>> more details. I was thinking about synthetic timers >>>>>>>>>>> http://msdn.microsoft.com/en- >>>>>>>>>>> us/library/windows/hardware/ff542758(v=vs.85).aspx >>>>>>>>>> is this what microsoft qpc uses as clocksource in hyper-v? >>>>>>>>> Yes, it should be enough for Win7 / W2K8R2. >>>>>>>> To clarify the thing that microsoft qpc uses is what is implemented >>>>>>>> by the patch Vadim attached to his previous email. But I believe >>>>>>>> that additional qemu patch is needed for Windows to actually use >>>>>>>> it. >>>>>>> You are right. >>>>>>> bits 1 and 9 must be set to on in leaf 0x40000003 and HPET >>>>>>> should be completely removed from ACPI. >>>>>> could you advise how to do this and/or make a patch? >>>>> Gleb mentioned that it properly handled in upstream, >>>>> otherwise just comment the entire HPET section in >>>>> acpi-dsdt.dsl file. >>>> i have upstream bios installed. so -no-hpet should disable hpet >>>> completely. can you give a hint, what >>>> "bits 1 and 9 must be set to on in leaf 0x40000003" means? >>> I mean the following code: >>> + if (hyperv_ref_counter_enabled()) { >>> + c->eax |= HV_X64_MSR_TIME_REF_COUNT_AVAILABLE; >>> + c->eax |= 0x200; >>> + } >>> >>> Please see attached file for more information. >>> >>>>>> the stuff you send yesterday is for qemu, right? would >>>>>> it be possible to use it in qemu-kvm also? >>>>> Yes, but don't forget about kvm patch as well. >>>> ok, i will try my best. would you consider your patch a quick hack >>>> or do you think it would be worth to be uploaded to the upstream >>>> repository? >>> It was just a brief attempt from my side, mostly inspirited by our with >>> Gleb conversation, to see what it worth to turn this option on. >>> It is not fully tested. It will crash Win8 (as well as the rest of the >>> currently introduced hyper-v features). >> i can confirm that windows 8 installer does not start and resets the >> vm continously. it tries to access hv msr 0x40000021 >> > Win8 needs more comprehensive Hyper-V support. yes it seems. i read your comment wrong. i was believing the hv_refcnt breaks the other hv_features and windows 8, but i guess you said any of the hv_features will break win 8?! peter > >> http://msdn.microsoft.com/en-us/library/windows/hardware/ff542648%28v=vs.85 >> %29.aspx >> >> it is possible to tell the guest that the host is not iTSC (how they >> call it) capable. i will try to hack >> a patch for this. >> >> peter >> >>> I wouldn't commit this code without comprehensive testing. >>> Vadim. >>> >>>> peter >>>> >>>>>> peter >>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Gleb.