Re: 8bf00a529967dafbbb210b377c38a15834d1e979 - performance regression?

[Gleb Natapov <gleb@redhat.com>]

On Tue, Nov 05, 2013 at 12:22:49PM +0200, Gleb Natapov wrote:
> On Tue, Nov 05, 2013 at 12:18:57PM +0200, Michael S. Tsirkin wrote:
> > On Mon, Nov 04, 2013 at 10:44:43PM +0200, Gleb Natapov wrote:
> > > On Mon, Nov 04, 2013 at 10:33:57PM +0200, Michael S. Tsirkin wrote:
> > > > On Mon, Nov 04, 2013 at 10:13:39PM +0200, Gleb Natapov wrote:
> > > > > On Mon, Nov 04, 2013 at 10:11:33PM +0200, Michael S. Tsirkin wrote:
> > > > > > On Thu, Oct 31, 2013 at 09:48:08AM +0200, Gleb Natapov wrote:
> > > > > > > On Thu, Oct 31, 2013 at 02:21:46AM +0200, Michael S. Tsirkin wrote:
> > > > > > > > commit 8bf00a529967dafbbb210b377c38a15834d1e979:
> > > > > > > > "    KVM: VMX: add support for switching of PERF_GLOBAL_CTRL " was
> > > > > > > > as far as I can tell supposed to bring about performance improvement
> > > > > > > > on hardware that supports it?
> > > > > > > No, it (and commits after it) supposed to fix a bug which it did.
> > > > > > > 
> > > > > > > > Instead it seems to make the typical case (not running guest
> > > > > > > > under perf) a bit slower than it used to be.
> > > > > > > > the cost of VMexit goes up by about 50 cycles
> > > > > > > > on sandy bridge where the optimization in question
> > > > > > > > actually is activated.
> > > > > > > >
> > > > > > > You seams to be confused. 8bf00a529967dafbbb210 adds support for special
> > > > > > > PERF_GLOBAL_CTRL switching, but does not add code to switch anything,
> > > > > > > so the commit itself is a nop.
> > > > > > 
> > > > > > It does add code to add_atomic_switch_msr.
> > > > > > 
> > > > > So what? You do not read what I wrote.
> > > > 
> > > > 
> > > > It's simple: if I revert 8bf00a529967dafbbb210 then exit latency
> > > > is reduced.
> > > > You seem to tell me it should be a nop, but in practice it isn't.
> > > > 
> > > 
> > > No, if you read below I am saying that it looks like you are claiming that
> > > generic msr switch mechanism is faster and I am not buying that. If you
> > > believe this to be the case ask Intel for explanation. Your claim about
> > > "not running guest under perf" is even stranger since in this case no msr
> > > switch should happen regardless of the aforementioned commit (unless guest
> > > or host runs nmi watchdog, but then switch will happen no matter if perf
> > > is running, so again not running guest under perf" does not make sense).
> > > So, in short, you do not really know where the slow down is coming
> > > from.
> > 
> > That's true.
> > 
> Then dig dipper.
> 
So quick and dirty patch to not needlessly write into VM_ENTRY_CONTROLS
when no PERF_GLOBAL_CTRL switching needed removes all the overhead. But we
probably need more generic code to shadow entire VM_ENTRY_CONTROLS/VM_EXIT_CONTROLS.


diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index e293a62..be64221 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -413,6 +413,7 @@ struct vcpu_vmx {
 	struct shared_msr_entry *guest_msrs;
 	int                   nmsrs;
 	int                   save_nmsrs;
+	bool                  core_perf_global_ctrl;
 	unsigned long	      host_idt_base;
 #ifdef CONFIG_X86_64
 	u64 		      msr_host_kernel_gs_base;
@@ -1432,9 +1433,12 @@ static void clear_atomic_switch_msr(struct vcpu_vmx *vmx, unsigned msr)
 		break;
 	case MSR_CORE_PERF_GLOBAL_CTRL:
 		if (cpu_has_load_perf_global_ctrl) {
+			if (vmx->core_perf_global_ctrl) {
 			clear_atomic_switch_msr_special(
 					VM_ENTRY_LOAD_IA32_PERF_GLOBAL_CTRL,
 					VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL);
+				vmx->core_perf_global_ctrl = false;
+			}
 			return;
 		}
 		break;
@@ -1488,6 +1492,7 @@ static void add_atomic_switch_msr(struct vcpu_vmx *vmx, unsigned msr,
 					GUEST_IA32_PERF_GLOBAL_CTRL,
 					HOST_IA32_PERF_GLOBAL_CTRL,
 					guest_val, host_val);
+			vmx->core_perf_global_ctrl = true;
 			return;
 		}
 		break;
--
			Gleb.