From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx2.suse.de ([195.135.220.15]:59250 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753186AbdKNHap (ORCPT ); Tue, 14 Nov 2017 02:30:45 -0500 Subject: Re: [PATCH RFC v3 1/6] x86/paravirt: Add pv_idle_ops to paravirt ops To: Quan Xu , Quan Xu , kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, x86@kernel.org, xen-devel@lists.xenproject.org Cc: Yang Zhang , Alok Kataria , Rusty Russell , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" References: <1510567565-5118-1-git-send-email-quan.xu0@gmail.com> <1510567565-5118-2-git-send-email-quan.xu0@gmail.com> <07fac696-e3d4-8f35-8f3d-764d7ab41204@suse.com> <902da704-1e4f-583b-91c3-1a62ccd6e73d@gmail.com> From: Juergen Gross Message-ID: Date: Tue, 14 Nov 2017 08:30:40 +0100 MIME-Version: 1.0 In-Reply-To: <902da704-1e4f-583b-91c3-1a62ccd6e73d@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On 14/11/17 08:02, Quan Xu wrote: > > > On 2017/11/13 18:53, Juergen Gross wrote: >> On 13/11/17 11:06, Quan Xu wrote: >>> From: Quan Xu >>> >>> So far, pv_idle_ops.poll is the only ops for pv_idle. .poll is called >>> in idle path which will poll for a while before we enter the real idle >>> state. >>> >>> In virtualization, idle path includes several heavy operations >>> includes timer access(LAPIC timer or TSC deadline timer) which will >>> hurt performance especially for latency intensive workload like message >>> passing task. The cost is mainly from the vmexit which is a hardware >>> context switch between virtual machine and hypervisor. Our solution is >>> to poll for a while and do not enter real idle path if we can get the >>> schedule event during polling. >>> >>> Poll may cause the CPU waste so we adopt a smart polling mechanism to >>> reduce the useless poll. >>> >>> Signed-off-by: Yang Zhang >>> Signed-off-by: Quan Xu >>> Cc: Juergen Gross >>> Cc: Alok Kataria >>> Cc: Rusty Russell >>> Cc: Thomas Gleixner >>> Cc: Ingo Molnar >>> Cc: "H. Peter Anvin" >>> Cc: x86@kernel.org >>> Cc: virtualization@lists.linux-foundation.org >>> Cc: linux-kernel@vger.kernel.org >>> Cc: xen-devel@lists.xenproject.org >> Hmm, is the idle entry path really so critical to performance that a new >> pvops function is necessary? > Juergen, Here is the data we get when running benchmark netperf: >  1. w/o patch and disable kvm dynamic poll (halt_poll_ns=0): >     29031.6 bit/s -- 76.1 %CPU > >  2. w/ patch and disable kvm dynamic poll (halt_poll_ns=0): >     35787.7 bit/s -- 129.4 %CPU > >  3. w/ kvm dynamic poll: >     35735.6 bit/s -- 200.0 %CPU > >  4. w/patch and w/ kvm dynamic poll: >     42225.3 bit/s -- 198.7 %CPU > >  5. idle=poll >     37081.7 bit/s -- 998.1 %CPU > > > >  w/ this patch, we will improve performance by 23%.. even we could improve >  performance by 45.4%, if we use w/patch and w/ kvm dynamic poll. also the >  cost of CPU is much lower than 'idle=poll' case.. I don't question the general idea. I just think pvops isn't the best way to implement it. >> Wouldn't a function pointer, maybe guarded >> by a static key, be enough? A further advantage would be that this would >> work on other architectures, too. > > I assume this feature will be ported to other archs.. a new pvops makes > code > clean and easy to maintain. also I tried to add it into existed pvops, > but it > doesn't match. You are aware that pvops is x86 only? I really don't see the big difference in maintainability compared to the static key / function pointer variant: void (*guest_idle_poll_func)(void); struct static_key guest_idle_poll_key __read_mostly; static inline void guest_idle_poll(void) { if (static_key_false(&guest_idle_poll_key)) guest_idle_poll_func(); } And KVM would just need to set guest_idle_poll_func and enable the static key. Works on non-x86 architectures, too. Juergen