From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2A41E936ED for ; Wed, 4 Oct 2023 22:06:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230226AbjJDWGL (ORCPT ); Wed, 4 Oct 2023 18:06:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45452 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229973AbjJDWGH (ORCPT ); Wed, 4 Oct 2023 18:06:07 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7286BD8 for ; Wed, 4 Oct 2023 15:06:01 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-d816fa2404aso460838276.0 for ; Wed, 04 Oct 2023 15:06:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1696457160; x=1697061960; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=JQhr64Gt8dKyHZjOzMcheFeKf4AHIRHG09SiFUQfy3g=; b=fzdBXODevY/Q3gIWKFCdoVcO9SGfxtkiGP37f+mR5qKBcufPII7Y9XDYfwsHPuAemX eqPiNW9os3uQ4MDVaL04sMwGlhzeOS8WbL6z2RmTdiuAqkMeXyAKZh7HjWBcfazWzDbM iohmEcSKia7Wh7NeYWXImcjh0ripNjWgO4Z47KV5yNUPWvc1awLFXQ52LPps81S4eHh5 bGWf0Ug0Woc9q5f3QA1jpbGknb1hK+HRgyWNAxLl4mGWUwo5wVnIUloCMVVQTE8zKKc5 JdYTYPW93EuI+xGT9KuBgaGCFl/tN2LuJ8bqS6H+5Jq+Oj7qJxP/MpjSon1oUgBa/9/S ik+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696457160; x=1697061960; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=JQhr64Gt8dKyHZjOzMcheFeKf4AHIRHG09SiFUQfy3g=; b=aB2ruQ3e7dwNB0fY0uStNPNjhTbnj1rWAGUV3Z4V8HWRLLtIK5DsoDviK1ZsDe6NMW y1pGt0ppvmmQjPvmN4nl7xjexAiYyjQQ6shJ+8Gu3j6xsQlxH5gDMFl53yybJ5T7Jjmc C4UneQn5N/vyVS3xXgJ4QzNWUHl9UYTnPl/E2/DX0h72sNWBYX0he9Kv6SoSzl9o17Qs QD4l81iyZfdYTCoH53dcm/Ua4iDDT3xgYseBE7XOUon8z7is0KbV185JhO0Ft0vkah8M Tb3as/n766sJvQXkiKkjEXper7tgpIaZVAdLPGa4S/yWX4ZJt3peCJWJmZInoOoh92UT 7cQg== X-Gm-Message-State: AOJu0YwpYfuSuMma/nvS2OdQt8zbcDWAsgdjBwOKHr7Ujfr18L9BpeWT jomUN+Pn2dlmPufysgYkqMYXnY2NIdU= X-Google-Smtp-Source: AGHT+IHH7NjbwhwW4+NO0g7zeY04sTLxW9j5SbKQaY4wvrnVGWvxS+MDJv/NzzkAEBxTE44icmyJ25+CXGM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a25:5f01:0:b0:d77:f7c3:37db with SMTP id t1-20020a255f01000000b00d77f7c337dbmr62293ybb.8.1696457160665; Wed, 04 Oct 2023 15:06:00 -0700 (PDT) Date: Wed, 4 Oct 2023 15:05:59 -0700 In-Reply-To: Mime-Version: 1.0 References: <20231002115718.GB13957@noisy.programming.kicks-ass.net> <20231002204017.GB27267@noisy.programming.kicks-ass.net> <20231003081616.GE27267@noisy.programming.kicks-ass.net> <20231004112152.GA5947@noisy.programming.kicks-ass.net> Message-ID: Subject: Re: [Patch v4 07/13] perf/x86: Add constraint for guest perf metrics event From: Sean Christopherson To: Mingwei Zhang Cc: Peter Zijlstra , Ingo Molnar , Dapeng Mi , Paolo Bonzini , Arnaldo Carvalho de Melo , Kan Liang , Like Xu , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , kvm@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, Zhenyu Wang , Zhang Xiong , Lv Zhiyuan , Yang Weijiang , Dapeng Mi , Jim Mattson , David Dunn , Thomas Gleixner Content-Type: text/plain; charset="us-ascii" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Oct 04, 2023, Sean Christopherson wrote: > Thinking about this more, what if we do a blend of KVM's FPU swapping and debug > register swapping? > > A. Load guest PMU state in vcpu_enter_guest() after IRQs are disabled > B. Put guest PMU state (and load host state) in vcpu_enter_guest() before IRQs > are enabled, *if and only if* the current CPU has one or perf events that > wants to use the hardware PMU > C. Put guest PMU state at vcpu_put() > D. Add a perf callback that is invoked from IRQ context when perf wants to > configure a new PMU-based events, *before* actually programming the MSRs, > and have KVM's callback put the guest PMU state > > If there are host perf events that want to use the PMU, then KVM will swap fairly > aggressively and the "downtime" of the host perf events will be limited to the > small window around VM-Enter/VM-Exit. > > If there are no such host events, KVM will swap on the first entry to the guest, > and keep the guest PMU loaded until the vCPU is put. > > The perf callback in (D) would allow perf to program system-wide events on all > CPUs without clobbering guest PMU state. > > I think that would make everyone happy. As long as our hosts don't create perf > events, then we get the "swap as little as possible" behavior without significantly > impacting the host's ability to utilize perf. If our host screws up and creates > perf events on CPUs that are running vCPUs, then the degraded vCPU performance is > on us. > > Rough sketch below, minus the perf callback or any of actual swapping logic. Another reason to go for an approach that doesn't completely kill off host PMU usage: just because we don't plan on enable perf events in *production*, there will undoubtedly be times where we want to enable perf events to debug issues (outside of prod) in the host kernel/KVM that affect VMs with a passthrough PMU. So I'll add a self-NAK to the idea of completely disabling the host PMU, I think that would burn us quite badly at some point.