From: "Mi, Dapeng" <dapeng1.mi@linux.intel.com>
To: Xiong Zhang <xiong.y.zhang@intel.com>, kvm@vger.kernel.org
Cc: seanjc@google.com, like.xu.linux@gmail.com, zhiyuan.lv@intel.com,
zhenyu.z.wang@intel.com, kan.liang@intel.com
Subject: Re: [PATCH 2/9] KVM: x85/pmu: Add Streamlined FREEZE_LBR_ON_PMI for vPMU v4
Date: Wed, 6 Sep 2023 17:49:56 +0800 [thread overview]
Message-ID: <75563bcd-23e9-e4b0-d7a8-48b04dd8d0ea@linux.intel.com> (raw)
In-Reply-To: <20230901072809.640175-3-xiong.y.zhang@intel.com>
On 9/1/2023 3:28 PM, Xiong Zhang wrote:
> Arch PMU version 4 adds a streamlined FREEZE_LBR_ON_PMI feature, this
> feature adds LBR_FRZ[bit 58] into IA32_PERF_GLOBAL_STATUS, this bit is
> set due to the following conditions:
> -- IA32_DEBUGCTL.FREEZE_LBR_ON_PMI has been set
> -- A performance counter, configured to generate PMI, has overflowed to
> signal a PMI. Consequently the LBR stack is frozen.
> Effectively, this bit also serves as a control to enabled capturing
> data in the LBR stack. When this bit is set, LBR stack is frozen, and
> new LBR records won't be filled.
>
> The sequence of streamlined freeze LBR is:
> 1. Profiling agent set IA32_DEBUGCTL.FREEZE_LBR_ON_PMI, and enable
> a performance counter to generate PMI on overflow.
> 2. Processor generates PMI and sets IA32_PERF_GLOBAL_STATUS.LBR_FRZ,
> then LBR stack is forzen.
> 3. Profiling agent PMI handler handles overflow, and clears
> IA32_PERF_GLOBAL_STATUS.
> 4. When IA32_PERF_GLOBAL_STATUS.LBR_FRZ is cleared in step 3,
> processor resume LBR stack, and new LBR records can be filled
> again.
>
> In order to emulate this behavior, LBR stack must be frozen on PMI.
> KVM has two choice to do this:
> 1. KVM stops vLBR event through perf_event_pause(), and put vLBR
> event into off state, then vLBR lose LBR hw resource, finally guest
> couldn't read LBR records in guest PMI handler. This choice couldn't
> be used.
> 2. KVM clear guest DEBUGCTLMSR_LBR bit in VMCS on PMI, so when guest
> is running, LBR HW stack is disabled, while vLBR event is still active
> and own LBR HW, so guest could still read LBR records in guest PMI
> handler. But the sequence of streamlined freeze LBR doesn't clear
> DEBUGCTLMSR_LBR bit, so when guest read guest DEBUGCTL_MSR, KVM will
> return a value with DEBUGCTLMSR_LBR bit set during LBR freezing. Once
> guest clears IA32_PERF_GLOBAL_STATUS.LBR_FRZ in step 4, KVM will
> re-enable guest LBR through setting guest DEBUGCTL_LBR bit in VMCS.
>
> As KVM will re-enable guest LBR when guest clears global status, the
> handling of GLOBAL_OVF_CTRL MSR is moved from common pmu.c into
> vmx/pmu_intel.c.
>
> Signed-off-by: Xiong Zhang <xiong.y.zhang@intel.com>
> ---
> arch/x86/include/asm/msr-index.h | 1 +
> arch/x86/kvm/pmu.c | 8 ------
> arch/x86/kvm/vmx/pmu_intel.c | 44 ++++++++++++++++++++++++++++++++
> arch/x86/kvm/vmx/vmx.c | 3 +++
> 4 files changed, 48 insertions(+), 8 deletions(-)
>
> diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
> index 3aedae61af4f..4fce37ae5a90 100644
> --- a/arch/x86/include/asm/msr-index.h
> +++ b/arch/x86/include/asm/msr-index.h
> @@ -1041,6 +1041,7 @@
> /* PERF_GLOBAL_OVF_CTL bits */
> #define MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT 55
> #define MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI (1ULL << MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI_BIT)
> +#define MSR_CORE_PERF_GLOBAL_OVF_CTRL_LBR_FREEZE BIT_ULL(58)
> #define MSR_CORE_PERF_GLOBAL_OVF_CTRL_OVF_BUF_BIT 62
> #define MSR_CORE_PERF_GLOBAL_OVF_CTRL_OVF_BUF (1ULL << MSR_CORE_PERF_GLOBAL_OVF_CTRL_OVF_BUF_BIT)
> #define MSR_CORE_PERF_GLOBAL_OVF_CTRL_COND_CHGD_BIT 63
> diff --git a/arch/x86/kvm/pmu.c b/arch/x86/kvm/pmu.c
> index edb89b51b383..4b6a508f3f0b 100644
> --- a/arch/x86/kvm/pmu.c
> +++ b/arch/x86/kvm/pmu.c
> @@ -640,14 +640,6 @@ int kvm_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
> reprogram_counters(pmu, diff);
> }
> break;
> - case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
> - /*
> - * GLOBAL_OVF_CTRL, a.k.a. GLOBAL STATUS_RESET, clears bits in
> - * GLOBAL_STATUS, and so the set of reserved bits is the same.
> - */
> - if (data & pmu->global_status_mask)
> - return 1;
> - fallthrough;
> case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR:
> if (!msr_info->host_initiated)
> pmu->global_status &= ~data;
> diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c
> index 3a36a91638c6..ba7695a64ff1 100644
> --- a/arch/x86/kvm/vmx/pmu_intel.c
> +++ b/arch/x86/kvm/vmx/pmu_intel.c
> @@ -426,6 +426,29 @@ static int intel_pmu_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
>
> pmu->pebs_data_cfg = data;
> break;
> + case MSR_CORE_PERF_GLOBAL_OVF_CTRL:
> + /*
> + * GLOBAL_OVF_CTRL, a.k.a. GLOBAL STATUS_RESET, clears bits in
> + * GLOBAL_STATUS, and so the set of reserved bits is the same.
> + */
> + if (data & pmu->global_status_mask)
> + return 1;
> + if (pmu->version >= 4 && !msr_info->host_initiated &&
> + (data & MSR_CORE_PERF_GLOBAL_OVF_CTRL_LBR_FREEZE)) {
> + u64 debug_ctl = vmcs_read64(GUEST_IA32_DEBUGCTL);
> + struct lbr_desc *lbr_desc = vcpu_to_lbr_desc(vcpu);
> +
> + if (!(debug_ctl & DEBUGCTLMSR_LBR) &&
> + lbr_desc->freeze_on_pmi) {
> + debug_ctl |= DEBUGCTLMSR_LBR;
> + vmcs_write64(GUEST_IA32_DEBUGCTL, debug_ctl);
> + lbr_desc->freeze_on_pmi = false;
> + }
> + }
> +
> + if (!msr_info->host_initiated)
> + pmu->global_status &= ~data;
> + break;
> default:
> if ((pmc = get_gp_pmc(pmu, msr, MSR_IA32_PERFCTR0)) ||
> (pmc = get_gp_pmc(pmu, msr, MSR_IA32_PMC0))) {
> @@ -565,6 +588,9 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu)
> if (vmx_pt_mode_is_host_guest())
> pmu->global_status_mask &=
> ~MSR_CORE_PERF_GLOBAL_OVF_CTRL_TRACE_TOPA_PMI;
> + if (pmu->version >= 4)
> + pmu->global_status_mask &=
> + ~MSR_CORE_PERF_GLOBAL_OVF_CTRL_LBR_FREEZE;
>
> entry = kvm_find_cpuid_entry_index(vcpu, 7, 0);
> if (entry &&
> @@ -675,6 +701,22 @@ static void intel_pmu_legacy_freezing_lbrs_on_pmi(struct kvm_vcpu *vcpu)
> }
> }
>
> +static void intel_pmu_streamlined_freezing_lbrs_on_pmi(struct kvm_vcpu *vcpu)
> +{
> + u64 data = vmcs_read64(GUEST_IA32_DEBUGCTL);
> + struct kvm_pmu *pmu = vcpu_to_pmu(vcpu);
> +
> + /*
> + * Even if streamlined freezing LBR won't clear LBR_EN like legacy
> + * freezing LBR, here legacy freezing LBR is called to freeze LBR HW
> + * for streamlined freezing LBR when guest run. But guest VM will
> + * see a fake guest DEBUGCTL MSR with LBR_EN bit set.
> + */
> + intel_pmu_legacy_freezing_lbrs_on_pmi(vcpu);
> + if ((data & DEBUGCTLMSR_FREEZE_LBRS_ON_PMI) && (data & DEBUGCTLMSR_LBR))
> + pmu->global_status |= MSR_CORE_PERF_GLOBAL_OVF_CTRL_LBR_FREEZE;
> +}
> +
> static void intel_pmu_deliver_pmi(struct kvm_vcpu *vcpu)
> {
> u8 version = vcpu_to_pmu(vcpu)->version;
> @@ -684,6 +726,8 @@ static void intel_pmu_deliver_pmi(struct kvm_vcpu *vcpu)
>
> if (version > 1 && version < 4)
> intel_pmu_legacy_freezing_lbrs_on_pmi(vcpu);
> + else if (version >= 4)
> + intel_pmu_streamlined_freezing_lbrs_on_pmi(vcpu);
> }
>
> static void vmx_update_intercept_for_lbr_msrs(struct kvm_vcpu *vcpu, bool set)
> diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
> index 199d0da1dbee..3bd64879aab3 100644
> --- a/arch/x86/kvm/vmx/vmx.c
> +++ b/arch/x86/kvm/vmx/vmx.c
> @@ -2098,6 +2098,9 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info)
> break;
> case MSR_IA32_DEBUGCTLMSR:
> msr_info->data = vmcs_read64(GUEST_IA32_DEBUGCTL);
> + if (vcpu_to_lbr_desc(vcpu)->freeze_on_pmi &&
> + vcpu_to_pmu(vcpu)->version >= 4)
> + msr_info->data |= DEBUGCTLMSR_LBR;
> break;
> default:
> find_uret_msr:
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
next prev parent reply other threads:[~2023-09-06 9:50 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-09-01 7:28 [PATCH 0/9] Upgrade vPMU version to 5 Xiong Zhang
2023-09-01 7:28 ` [PATCH 1/9] KVM: x86/PMU: Don't release vLBR caused by PMI Xiong Zhang
2023-09-06 9:47 ` Mi, Dapeng
2023-09-12 11:54 ` Like Xu
2023-09-13 6:00 ` Zhang, Xiong Y
2023-09-01 7:28 ` [PATCH 2/9] KVM: x85/pmu: Add Streamlined FREEZE_LBR_ON_PMI for vPMU v4 Xiong Zhang
2023-09-06 9:49 ` Mi, Dapeng [this message]
2023-09-01 7:28 ` [PATCH 3/9] KVM: x86/pmu: Add PERF_GLOBAL_STATUS_SET MSR emulation Xiong Zhang
2023-09-01 7:28 ` [PATCH 4/9] KVM: x86/pmu: Add MSR_PERF_GLOBAL_INUSE emulation Xiong Zhang
2023-09-12 11:41 ` Like Xu
2023-09-13 5:11 ` Zhang, Xiong Y
2023-09-01 7:28 ` [PATCH 5/9] KVM: x86/pmu: Check CPUID.0AH.ECX consistency Xiong Zhang
2023-09-06 9:44 ` Mi, Dapeng
2023-09-12 0:45 ` Zhang, Xiong Y
2023-09-12 11:31 ` Like Xu
2023-09-13 4:25 ` Zhang, Xiong Y
2023-09-01 7:28 ` [PATCH 6/9] KVM: x86/pmu: Add Intel PMU supported fixed counters mask Xiong Zhang
2023-09-06 10:08 ` Mi, Dapeng
2023-09-01 7:28 ` [PATCH 7/9] KVM: x86/pmu: Add fixed counter enumeration for pmu v5 Xiong Zhang
2023-09-12 11:24 ` Like Xu
2023-09-13 4:11 ` Zhang, Xiong Y
2023-09-01 7:28 ` [PATCH 8/9] KVM: x86/pmu: Upgrade pmu version to 5 on intel processor Xiong Zhang
2023-09-12 11:19 ` Like Xu
2023-09-13 3:34 ` Zhang, Xiong Y
2023-09-01 7:28 ` [PATCH 9/9] KVM: selftests: Add fixed counters enumeration test case Xiong Zhang
2023-09-11 3:03 ` Mi, Dapeng
2023-09-12 0:35 ` Zhang, Xiong Y
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=75563bcd-23e9-e4b0-d7a8-48b04dd8d0ea@linux.intel.com \
--to=dapeng1.mi@linux.intel.com \
--cc=kan.liang@intel.com \
--cc=kvm@vger.kernel.org \
--cc=like.xu.linux@gmail.com \
--cc=seanjc@google.com \
--cc=xiong.y.zhang@intel.com \
--cc=zhenyu.z.wang@intel.com \
--cc=zhiyuan.lv@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox