From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: apic-v reduce network performance in my test case Date: Mon, 2 Feb 2015 09:59:24 -0500 Message-ID: <20150202145923.GA2186@l.oracle.com> References: <54CCAE71.5070006@huawei.com> <54CF5BA7020000780005BCF4@mail.emea.novell.com> <54CF82A1.3060505@huawei.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <54CF82A1.3060505@huawei.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "Liuqiming (John)" Cc: Jan Beulich , xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org On Mon, Feb 02, 2015 at 09:58:57PM +0800, Liuqiming (John) wrote: > Hi Jan, > Thanks for the reply. > > On 2015/2/2 18:12, Jan Beulich wrote: > >>>> On 31.01.15 at 11:29, wrote: > >> Recently I met an odd performance problem: when I turn on APIC > >> Virtualization feature (apicv=1), the network performance of a windows > >> guest become worse. > >> > >> My test case like this: host only have one windows 2008 R2 HVM > >> guest running,and this guest has a SR-IOV VF network passthrough to it. > >> Guest using this network access a NAS device. No fontend or backend of > >> network and storage, all data transfered through network. > >> > >> The xentrace data shows: the mainly difference between apicv and > >> non-apicv, is the way guest write apic registers, and > >> EXIT_REASON_MSR_WRITE vmexit cost much more time than > >> EXIT_REASON_APIC_WRITE, but when using WRMSR, the PAUSE vmexit is much > >> less than using APIC-v. > > > >There being heavier use of the pause VMEXIT doesn't by itself say > >anything, I'm afraid. It may suggest that you have a C-state exit > >latency problem - try lowering the maximum C-state allowed, or > >disabling use of C-states altogether. > Sorry, I forgot to mention my test scenario: > Its a video test suite,I am not sure what the logic inside the tools exactly (not opensource tool). > The basic flow is: > 1) test suite start several thread to read video file from disk (from NAS through network in my case) > 2) decode these video data as a frame one by one > 3) if any frame delay more than 40ms, then mark as lost > > test result: > apicv=1, there can be 15 thread running at the same time without lost frame > apicv=0, there can be 22 thread running at the same time without lost frame > > so when I'm saying apicv reduce the performance, I got the conclusion from the test result not from what xentrace shows. > > > >> In commit 7f2e992b824ec62a2818e64390ac2ccfbd74e6b7 > >> "VMX/Viridian: suppress MSR-based APIC suggestion when having APIC-V", > >> msr based apic is disabled when apic-v is on, I wonder can they co-exist > >> in some way? seems for windows guest msr-based apic has better performance. > > > >The whole purpose is to avoid the costly MSR access exits. Why > >would you want to reintroduce that overhead? > > > >Jan > > > I agree to avoid the MSR access vmexit by using apicv, I just do not know what's the side effect. > Because from the test result, apicv replacing msr-based access brings performance reduction. It sounds like it brings latency disruption, not neccessarily performance reduction. What happens if you run with the cpufreq turned to performance, or as Jan suggested - with disabling C-states? > > > >. > > > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel