From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2CEC013BC08 for ; Fri, 19 Apr 2024 19:14:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713554066; cv=none; b=ZmsA4fW3qWJzZ/CxO1gQz7m7qg6Rys2YGoSDbeqU6tyl+PWDUgOu9omxqa6A8b77kp2U2Y0Dq5APnsc3PdwpzwNSW8ON5LBtCAG3LumtN06toSpd6mrAoK0x2yuOF0r6mr+GKFYcv7MPPDfx9c9lWrruPjeShYz+WXb+6epTcjI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713554066; c=relaxed/simple; bh=ghHRvfsnLZoBuD+thj2tzA2nIrRJDK62IHkGCKIFt9A=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Xp/ENG26a+w2AjzLyjohdnz73C+g63Ot9/8eUsWPiugqfD2lhyUc7qGPJ+31tu7n1mclhtEiKSBlnt72NUuHzYtgzMMhifeClm1N2Z/ooR/Y8Bn/p2pf1PQpdCoa6YuBc66Yl9baTEmlRCXQfOv8t6ZSZ5ijqkhSRmM/4krCAGs= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Phyt+OmO; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Phyt+OmO" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-de45daf49deso4937261276.2 for ; Fri, 19 Apr 2024 12:14:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1713554064; x=1714158864; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=LxyLjfyND/X0CfR1qvIubJS+urbDCxyrK24QGGBvcRg=; b=Phyt+OmOLRhTdZkGxZSe2A7lyD8q0bdQAvdmS0mUtCeCjk/hEm+PpA70c7g9yvOR+9 NcvK1QiLKQJJDu1IVCizbI3tFm3Ttq83m7aHboMwtXJMJUnsA/ZTXz9GY6RcCAQe1G4R ltj3kBgL73AE0WbAKKB/NTeHlfZy3nWGmIViEAvV7/SJWTj7z77jzH8jdyvNqrf4YuG0 2aXk+90jUYDyMMMGJSM9GTFpWginoX2BH99C8G3qHT5xYkb8wSOyfscR50xbefu1Ckcx 3thmYxD8qDOl+Iyaah8Z2PGvMijJSwbf067mBjDmQCZz9adnirJPAcZxptWV3VkIKNn+ BiJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713554064; x=1714158864; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=LxyLjfyND/X0CfR1qvIubJS+urbDCxyrK24QGGBvcRg=; b=gDEijBRPatHRLTZSbAGGK3gNvNfzVKj/OYiWoWasNGS0+BV6ZXdu/qjrqVMHX+Snfe UmmsEt/PgcaH85nqyv4mMj4YRNZvoy9xXYh5A/dwqteffJ7IPtM7noVhYdi249C3ljxw u63kavGwHR7PuDhcv+bnrNokqImrygttgC6dcQ+0EcVLTIbAw8wWo0GmSn3DFN7POe6B quq6nRWkxv3fd71c1lpB5ZppN762i6u/2ItkksRmSdZDrf32Q48mNFR8xNfl0467Jkyz 0jzG3klRR9Lhf5T8KY6NCJkti3WQHuxNGp0ouZg3gjvYIh6LWxRaUPoBHYl0z4CQ4UlX 9OPQ== X-Forwarded-Encrypted: i=1; AJvYcCVbQ3jiqtDuzPjG8VIlP4WEEyJTD27y8kw1EGrwUP/6aMK1Ew6Xj4vi/3mn4wbd/AQNt0oZcdjVxEKaFcXv7rngXFXpACpfHP8tqbRKAaoong== X-Gm-Message-State: AOJu0YzO19Yv0eJhqJRLy9QmjaobONE/paXHyVdwdibCpe1rD9TLLhQc HvKrS0JWmYK+6ZBkfY3Lxfh5/4siejCkVqkAM/KjLaKHjNoxobio6/OEnv002u4V0x9C1Z/IuwB 37w== X-Google-Smtp-Source: AGHT+IGVgn8Gf0JE3BoAkURXxw8LyhcDmWzr3Q2r8XR7+A5tBJUam6WylNWXRK1YN/QeAIyjeaEID9m1Od4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:c01:b0:dcd:b431:7f5b with SMTP id fs1-20020a0569020c0100b00dcdb4317f5bmr824560ybb.0.1713554064161; Fri, 19 Apr 2024 12:14:24 -0700 (PDT) Date: Fri, 19 Apr 2024 12:14:22 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: linux-perf-users@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240126085444.324918-1-xiong.y.zhang@linux.intel.com> Message-ID: Subject: Re: [RFC PATCH 00/41] KVM: x86/pmu: Introduce passthrough vPM From: Sean Christopherson To: Mingwei Zhang Cc: Xiong Zhang , pbonzini@redhat.com, peterz@infradead.org, kan.liang@intel.com, zhenyuw@linux.intel.com, dapeng1.mi@linux.intel.com, jmattson@google.com, kvm@vger.kernel.org, linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, zhiyuan.lv@intel.com, eranian@google.com, irogers@google.com, samantha.alt@intel.com, like.xu.linux@gmail.com, chao.gao@intel.com Content-Type: text/plain; charset="us-ascii" On Thu, Apr 18, 2024, Mingwei Zhang wrote: > On Thu, Apr 11, 2024, Sean Christopherson wrote: > > > > > > I think we should call this a mediated PMU, not a passthrough PMU. KVM still > > emulates the control plane (controls and event selectors), while the data is > > fully passed through (counters). > > > > > Sean, > > I feel "mediated PMU" seems to be a little bit off the ..., no? In > KVM, almost all of features are mediated. In our specific case, the > legacy PMU is mediated by KVM and perf subsystem on the host. In new > design, it is mediated by KVM only. Currently, at a feature level, I mentally bin things into two rough categories in KVM: 1. Virtualized - Guest state is loaded into hardware, or hardware supports running with both host and guest state (e.g. TSC scaling), and the guest has full read/write access to its state while running. 2. Emulated - Guest state is never loaded into hardware, and instead the feature/state is emulated in software. There is no "Passthrough" because that's (mostly) covered by my Virtualized definition. And because I also think of passthrough as being about *assets*, not about the features themselves. They are far from perfect definitions, e.g. individual assets can be passed through, virtualized by hardware, or emulated in software. But for the most part, I think classifying features as virtualized vs. emulated works well, as it helps reason about the expected behavior and performance of a feature. E.g. for some virtualized features, certain assets may need to be explicitly passed through, e.g. access to x2APIC MSRs for APICv. But APICv itself still falls into the virtualized category, e.g. the "real" APIC state isn't passed through to the guest. If KVM didn't already have a PMU implementation to deal with, this wouldn't be an issue, e.g. we'd just add "enable_pmu" and I'd mentally bin it into the virtualized category. But we need to distinguish between the two PMU models, and using "enable_virtualized_pmu" would be comically confusing for users. :-) And because this is user visible, I would like to come up with a name that (some) KVM users will already be familiar with, i.e. will have some chance of intuitively understand without having to go read docs. Which is why I proposed "mediated"; what we are proposing for the PMU is similar to the "mediated device" concepts in VFIO. And I also think "mediated" is a good fit in general, e.g. this becomes my third classification: 3. Mediated - Guest is context switched at VM-Enter/VM-Exit, i.e. is loaded into hardware, but the guest does NOT have full read/write access to the feature. But my main motiviation for using "mediated" really is that I hope that it will help KVM users grok the basic gist of the design without having to read and understand KVM documentation, because there is already existing terminology in the broader KVM space. > We intercept the control plan in current design, but the only thing > we do is the event filtering. No fancy code change to emulate the control > registers. So, it is still a passthrough logic. It's not though. Passthrough very specifically means the guest has unfettered access to some asset, and/or KVM does no filtering/adjustments whatseover. "Direct" is similar, e.g. KVM's uses "direct" in MMU context to refer to addresses that don't require KVM to intervene and translate. E.g. entire MMUs can be direct, but individual shadow pages can also be direct (no corresponding guest PTE to translate). For this flavor of PMU, it's not full passthrough or direct. Some assets are passed through, e.g. PMCs, but others are not. > In some (rare) business cases, I think maybe we could fully passthrough > the control plan as well. For instance, sole-tenant machine, or > full-machine VM + full offload. In case if there is a cpu errata, KVM > can force vmexit and dynamically intercept the selectors on all vcpus > with filters checked. It is not supported in current RFC, but maybe > doable in later versions. Heh, that's an argument for using something other than "passthrough", because if we ever do support such a use case, we'd end up with enable_fully_passthrough_pmu, or in the spirit of KVM shortlogs, really_passthrough_pmu :-) Though I think even then I would vote for "enable_dedicated_pmu", or something along those lines, purely to avoid overloading "passthrough", i.e. to try to use passhtrough strictly when talking about assets, not features. And because unless we can also passthrough LVTPC, it still wouldn't be a complete passthrough of the PMU as KVM would be emulating PMIs.