From mboxrd@z Thu Jan 1 00:00:00 1970 From: Paolo Bonzini Subject: Re: x86: strange behavior of invlpg Date: Mon, 16 May 2016 18:56:32 +0200 Message-ID: <5739FBC0.3070309@redhat.com> References: <2C79F043-EF77-4C44-BE36-1CEDE16E788F@gmail.com> <573992B7.20300@redhat.com> <20368BC3-CA33-4D1F-9421-D743084AF16F@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: kvm@vger.kernel.org To: Nadav Amit Return-path: Received: from mx1.redhat.com ([209.132.183.28]:50172 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753274AbcEPQ4i (ORCPT ); Mon, 16 May 2016 12:56:38 -0400 In-Reply-To: <20368BC3-CA33-4D1F-9421-D743084AF16F@gmail.com> Sender: kvm-owner@vger.kernel.org List-ID: On 16/05/2016 18:51, Nadav Amit wrote: > Thanks! I appreciate it. >=20 > I think your experiment with global paging just corraborate that the > latency is caused by TLB misses. I measured TLB misses (and especiall= y STLB > misses) in other experiments but not in this one. I will run some mor= e > experiments, specifically to test how AMD behaves. I'm curious about AMD too now... with invlpg: 285,639,427 with full flush: 584,419,299 invlpg only 70,681,128 full flushes only 265,238,766 access net 242,538,804 w/full flush net 319,180,533 w/invlpg net 214,958,299 Roughly the same with and without pte.g. So AMD behaves as it should. > I should note this is a byproduct of a study I did, and it is not as = if I was > looking for strange behaviors (no more validation papers for me!). >=20 > The strangest thing is that on bare-metal I don=92t see this phenomen= on - I doubt > it is a CPU =93feature=94. Once we understand it, the very least it m= ay affect > the recommended value of =93tlb_single_page_flush_ceiling=94, that co= ntrols when > the kernel performs full TLB flush vs. selective flushes. Do you have a kernel module to reproduce the test on bare metal? (/me i= s lazy). Paolo