From mboxrd@z Thu Jan 1 00:00:00 1970 From: Paolo Bonzini Subject: Re: Windows Server 2008R2 KVM guest performance issues Date: Tue, 27 Aug 2013 10:38:04 +0200 Message-ID: <521C656C.9090208@redhat.com> References: <521BA934.3050507@gameservers.com> <521C52A8.40801@redhat.com> <20130827073833.GD22899@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Brian Rak , kvm@vger.kernel.org To: Gleb Natapov Return-path: Received: from mail-qe0-f53.google.com ([209.85.128.53]:46007 "EHLO mail-qe0-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752549Ab3H0IiM (ORCPT ); Tue, 27 Aug 2013 04:38:12 -0400 Received: by mail-qe0-f53.google.com with SMTP id 1so2410024qee.12 for ; Tue, 27 Aug 2013 01:38:11 -0700 (PDT) In-Reply-To: <20130827073833.GD22899@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: Il 27/08/2013 09:38, Gleb Natapov ha scritto: > On Tue, Aug 27, 2013 at 09:18:00AM +0200, Paolo Bonzini wrote: >>> I've captured 20,000 lines of kvm trace output. This can be found >>> https://gist.github.com/devicenull/fa8f49d4366060029ee4/raw/fb89720d34b43920be22e3e9a1d88962bf305da8/trace >> >> The guest is doing quite a lot of exits per second, mostly to (a) access >> the ACPI timer > I see a lot of PM timer access not ACPI timer. The solution for that is > the patchset Brian linked. ACPI timer = PM timer, no? >> (b) service NMIs. In fact, every NMI is reading the >> timer too and causing an exit to QEMU. >> > Do you mean "kvm_exit: reason EXCEPTION_NMI rip 0xfffff800016dcf84 info > 0 80000307"? Those are not NMIs, single NMI will kill Windows, they are #NM > exceptions. Oops, yes. > Brian, is your workload uses floating point calculation? Yeah, it looks like it does, there are a lot of kvm_fpu tracepoints too. Basically the problem is that every exit to userspace unloads the FPU in vcpu_put. qemu-system-x86-16189 [003] d... 9439.144500: kvm_entry: vcpu 12 qemu-system-x86-16189 [003] d... 9439.144502: kvm_exit: reason EXCEPTION_NMI rip 0xfffff800016dcf84 info 0 80000307 qemu-system-x86-16189 [003] .... 9439.144502: kvm_fpu: load qemu-system-x86-16189 [003] d... 9439.144503: kvm_entry: vcpu 12 qemu-system-x86-16189 [003] d... 9439.144505: kvm_exit: reason IO_INSTRUCTION rip 0xfffff8000162d17d info b008000b 0 qemu-system-x86-16189 [003] .... 9439.144506: kvm_emulate_insn: 0:fffff8000162d17d:ed (prot64) qemu-system-x86-16189 [003] .... 9439.144506: kvm_pio: pio_read at 0xb008 size 4 count 1 qemu-system-x86-16189 [003] .... 9439.144507: kvm_userspace_exit: reason KVM_EXIT_IO (2) qemu-system-x86-16189 [003] .... 9439.144508: kvm_fpu: unload qemu-system-x86-16189 [003] d... 9439.144578: kvm_entry: vcpu 12 qemu-system-x86-16189 [003] d... 9439.144579: kvm_exit: reason EXCEPTION_NMI rip 0xfffff800016dcf84 info 0 80000307 qemu-system-x86-16189 [003] .... 9439.144581: kvm_fpu: load qemu-system-x86-16189 [003] d... 9439.144581: kvm_entry: vcpu 12 qemu-system-x86-16189 [003] d... 9439.144583: kvm_exit: reason IO_INSTRUCTION rip 0xfffff8000162d17d info b008000b 0 qemu-system-x86-16189 [003] .... 9439.144585: kvm_emulate_insn: 0:fffff8000162d17d:ed (prot64) qemu-system-x86-16189 [003] .... 9439.144585: kvm_pio: pio_read at 0xb008 size 4 count 1 qemu-system-x86-16189 [003] .... 9439.144586: kvm_userspace_exit: reason KVM_EXIT_IO (2) qemu-system-x86-16189 [003] .... 9439.144587: kvm_fpu: unload qemu-system-x86-16189 [003] d... 9439.144787: kvm_entry: vcpu 12 qemu-system-x86-16189 [003] d... 9439.144788: kvm_exit: reason EXCEPTION_NMI rip 0xfffff800016dcf84 info 0 80000307 qemu-system-x86-16189 [003] .... 9439.144789: kvm_fpu: load qemu-system-x86-16189 [003] d... 9439.144789: kvm_entry: vcpu 12 qemu-system-x86-16189 [003] d... 9439.144791: kvm_exit: reason IO_INSTRUCTION rip 0xfffff8000162d17d info b008000b 0 qemu-system-x86-16189 [003] .... 9439.144792: kvm_emulate_insn: 0:fffff8000162d17d:ed (prot64) qemu-system-x86-16189 [003] .... 9439.144793: kvm_pio: pio_read at 0xb008 size 4 count 1 qemu-system-x86-16189 [003] .... 9439.144794: kvm_userspace_exit: reason KVM_EXIT_IO (2) qemu-system-x86-16189 [003] .... 9439.144794: kvm_fpu: unload It should be interesting to analyze the cost in kvm-unit-tests. I'll look at it. Paolo