From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9278A23750 for ; Wed, 4 Oct 2023 22:06:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="fzdBXODe" Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6FC15BD for ; Wed, 4 Oct 2023 15:06:01 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-d816fa2404aso460839276.0 for ; Wed, 04 Oct 2023 15:06:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1696457160; x=1697061960; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=JQhr64Gt8dKyHZjOzMcheFeKf4AHIRHG09SiFUQfy3g=; b=fzdBXODevY/Q3gIWKFCdoVcO9SGfxtkiGP37f+mR5qKBcufPII7Y9XDYfwsHPuAemX eqPiNW9os3uQ4MDVaL04sMwGlhzeOS8WbL6z2RmTdiuAqkMeXyAKZh7HjWBcfazWzDbM iohmEcSKia7Wh7NeYWXImcjh0ripNjWgO4Z47KV5yNUPWvc1awLFXQ52LPps81S4eHh5 bGWf0Ug0Woc9q5f3QA1jpbGknb1hK+HRgyWNAxLl4mGWUwo5wVnIUloCMVVQTE8zKKc5 JdYTYPW93EuI+xGT9KuBgaGCFl/tN2LuJ8bqS6H+5Jq+Oj7qJxP/MpjSon1oUgBa/9/S ik+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696457160; x=1697061960; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=JQhr64Gt8dKyHZjOzMcheFeKf4AHIRHG09SiFUQfy3g=; b=D1ExDJB9bLjIk31kxBWQIMzQjVDFaVZ9ptG8Mg8Tav6aaakQidsa9kJoSVBmd6oPJn pjWM0uTVcfwgtDZCmNL/EJV7DTw20Jv8oUuLYXkbvNZSwO/rwpyN6BknPglOsnWcEv+m VgYYbCB8t200BdqNvBqXDeJqFWhuWdAAPNxcYTWdSxoz1j1N3DJ71BP5i3TObDRl2BES WdjgmZ4p1SIUimdPNehZMLU3Hi7t3Uuo8OQFh9FxFtjmVPK43XJpWNL2c7LX4GHN7Kq0 0VxLfL0gv4Gyqt9I8ZntgI/QNxd4vUAK1hiZVD8UZq64jkke7BBocz0PJEYgNLVTzXMJ NkBA== X-Gm-Message-State: AOJu0YwNb4IElGKSA8oziRhygNy8HWwdWyRmnAskB6VWtxj23oOgDbs3 UU8f3pjOkjhvL/ITKewYdlcZsUkDN2o= X-Google-Smtp-Source: AGHT+IHH7NjbwhwW4+NO0g7zeY04sTLxW9j5SbKQaY4wvrnVGWvxS+MDJv/NzzkAEBxTE44icmyJ25+CXGM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:5f01:0:b0:d77:f7c3:37db with SMTP id t1-20020a255f01000000b00d77f7c337dbmr62293ybb.8.1696457160665; Wed, 04 Oct 2023 15:06:00 -0700 (PDT) Date: Wed, 4 Oct 2023 15:05:59 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: linux-perf-users@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20231002115718.GB13957@noisy.programming.kicks-ass.net> <20231002204017.GB27267@noisy.programming.kicks-ass.net> <20231003081616.GE27267@noisy.programming.kicks-ass.net> <20231004112152.GA5947@noisy.programming.kicks-ass.net> Message-ID: Subject: Re: [Patch v4 07/13] perf/x86: Add constraint for guest perf metrics event From: Sean Christopherson To: Mingwei Zhang Cc: Peter Zijlstra , Ingo Molnar , Dapeng Mi , Paolo Bonzini , Arnaldo Carvalho de Melo , Kan Liang , Like Xu , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , kvm@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Zhenyu Wang , Zhang Xiong , Lv Zhiyuan , Yang Weijiang , Dapeng Mi , Jim Mattson , David Dunn , Thomas Gleixner Content-Type: text/plain; charset="us-ascii" X-Spam-Status: No, score=-9.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net On Wed, Oct 04, 2023, Sean Christopherson wrote: > Thinking about this more, what if we do a blend of KVM's FPU swapping and debug > register swapping? > > A. Load guest PMU state in vcpu_enter_guest() after IRQs are disabled > B. Put guest PMU state (and load host state) in vcpu_enter_guest() before IRQs > are enabled, *if and only if* the current CPU has one or perf events that > wants to use the hardware PMU > C. Put guest PMU state at vcpu_put() > D. Add a perf callback that is invoked from IRQ context when perf wants to > configure a new PMU-based events, *before* actually programming the MSRs, > and have KVM's callback put the guest PMU state > > If there are host perf events that want to use the PMU, then KVM will swap fairly > aggressively and the "downtime" of the host perf events will be limited to the > small window around VM-Enter/VM-Exit. > > If there are no such host events, KVM will swap on the first entry to the guest, > and keep the guest PMU loaded until the vCPU is put. > > The perf callback in (D) would allow perf to program system-wide events on all > CPUs without clobbering guest PMU state. > > I think that would make everyone happy. As long as our hosts don't create perf > events, then we get the "swap as little as possible" behavior without significantly > impacting the host's ability to utilize perf. If our host screws up and creates > perf events on CPUs that are running vCPUs, then the degraded vCPU performance is > on us. > > Rough sketch below, minus the perf callback or any of actual swapping logic. Another reason to go for an approach that doesn't completely kill off host PMU usage: just because we don't plan on enable perf events in *production*, there will undoubtedly be times where we want to enable perf events to debug issues (outside of prod) in the host kernel/KVM that affect VMs with a passthrough PMU. So I'll add a self-NAK to the idea of completely disabling the host PMU, I think that would burn us quite badly at some point.