From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-oa1-f72.google.com (mail-oa1-f72.google.com [209.85.160.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BBEAD4B4894 for ; Thu, 24 Sep 2026 17:29:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.72 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790270995; cv=none; b=Y2bK+UO9Bakhp6fnu/5cdV9vect3l8Z+7u069sVEcw1c389dbJ6gy2B2PlawkRkZt4oI3tBvsJXzvQgjGqCDQlECSen46XeFlyytbTIfM2A1UrhrHX2MF4NTk7trayJYM4UqKupMqXrms8PHm30GH+0wlqWDKmGeHJI5p1p1QuE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790270995; c=relaxed/simple; bh=TnKyEpcgBA6CB8nQw6+EyO5hcJkdWh4SiArNmuixhFY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=k15At2mmaw4w8D57MDYzvLIMtX19KDAvIJCFWVCL+PlZN/lFO6lzpAE1b7KNBv9GVSlasLdOs+T+YNdbHyEfA/1CpWi3f7nYKSDKhf6+xLxoPJnoddlZ7dkCZieqANiSIDuixXS06CEnw1mi2Ed9/n5Uzp7RclKHumZXbBKBhNE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--coltonlewis.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=oVXhn6UW; arc=none smtp.client-ip=209.85.160.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--coltonlewis.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="oVXhn6UW" Received: by mail-oa1-f72.google.com with SMTP id 586e51a60fabf-47d75bcf3ebso162557fac.0 for ; Thu, 24 Sep 2026 10:29:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1790270989; x=1790875789; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=Sx4i7rLrLjUKxNonKTZBsgb0Jj/BfKfmEgD+eiXTprc=; b=oVXhn6UWk0PKBqLWSSm5Iul/zTXh8W0GzJaZq3Cm2MgnKw/NugVxZArNEFLblFEZXS fvNoYRGODoixmx6gQXeSvvueBi6e1l3jXrMoZNjXSnk/NR30K4/mcSn0QkDTH1+6xtKA mtub4peMQbLE+omrhLogOx9eMNqGqfh1mcBoyv+U481QHgifyw2AhMbNJu8I7q+KYP2T KmHU3mumuhtF1l7/1N3x0MBEY07lQQHgtwLVqeW/Om4eGe38tzFutaNqN3Q+PQ3R2Pmb drSZn1PJDv94wgprJ0Fpq+J0IhP3weF+nZVZI6HNMQpAmWfzCDTGKQpXu4I96XTPgrd9 oyoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790270989; x=1790875789; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Sx4i7rLrLjUKxNonKTZBsgb0Jj/BfKfmEgD+eiXTprc=; b=mwWgpEvR9sj9HjNH5ImDaMYdRWnwVGTJ5guhyU5b/HpeLxzRy2ku8qiVxDT1iWDY99 wdBV1Vu/TtrxnDoGHDGyBBchwiWQr22IXc2LXhFeEVWDEPsUFFO9P2jA3jY8qVkYfpBL D9hULOehJgiNwL/Suv9DFJNvnzDapp3VOVgWmr/4YIMXwbI7Peq4nnltM1Wp5aSF9w/1 9vvGCauMcwiogdG4sGfhh7mkggLHN3SQm1JmPvg/4lXKe0gDBnvNGBz4lGqi/mZ6mqcJ D6W6Qp1tEYi7q/VRomCUHcP6hrdCAfvaQkenXXCqX9Hh4YkJmUeftmojMyX2HkAOIMUb mC8g== X-Gm-Message-State: AFuF++kDM1WkcSvwuu1+TxtPu8ADVAEbN3cH8hIHChCQDLQNnxAR6yQx M26CeJ+DE/3OETlIqP7DP5v+9JvJg4cusM3IOQyD0IMGfdSyfybwQ7YqGskTreYpAznUM2Nh8dE giNQ0FNA5hSuj2BPegz5AsB/KcrfElMYcVhu5lBA+5Y4KMs0sPDmYWeWku0kkMjUsEjJjU+htIH FQAJi7DW4FyI0jaENXCaqybVLVhl75k6uGpIBuSYUHISMzt1MQGNU7esYsfFI= X-Received: from ilpx16.prod.google.com ([2002:a92:d650:0:b0:509:637c:4c67]) (user=coltonlewis job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6808:4fe0:b0:4d6:9133:23fc with SMTP id 5614622812f47-4d72b44c8aemr3907567b6e.50.1790270988101; Thu, 24 Sep 2026 10:29:48 -0700 (PDT) Date: Thu, 24 Sep 2026 17:29:18 +0000 In-Reply-To: <20260924172928.2110956-1-coltonlewis@google.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260924172928.2110956-1-coltonlewis@google.com> X-Mailer: git-send-email 2.56.0.rc1.315.gc6ed9934b7-goog Message-ID: <20260924172928.2110956-13-coltonlewis@google.com> Subject: [PATCH v9 12/22] KVM: arm64: Context swap Partitioned PMU guest registers From: Colton Lewis To: kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org Cc: Marc Zyngier , Oliver Upton , Oliver Upton , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Fuad Tabba , Catalin Marinas , Will Deacon , Mark Rutland , Paolo Bonzini , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , James Clark , Robin Murphy , Zide Chen , Alexandru Elisei , Ganapatrao Kulkarni , Mingwei Zhang , Jonathan Corbet , Russell King , Shuah Khan , linux-perf-users@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Colton Lewis Content-Type: text/plain; charset="UTF-8" Save and restore newly untrapped registers that can be directly accessed by the guest when the PMU is partitioned. - PMEVCNTRn_EL0 - PMCCNTR_EL0 - PMSELR_EL0 - PMCR_EL0 - PMCNTEN_EL0 - PMINTEN_EL1 If we know we are not partitioned (that is, using the emulated vPMU), then return immediately. A later patch will make this lazy so the context swaps don't happen unless the guest has accessed the PMU. PMEVTYPER is handled in a following patch since we must apply the KVM event filter before writing values to hardware. PMOVS guest counters are cleared to avoid the possibility of generating spurious interrupts when PMINTEN is written. This is fine because the virtual register for PMOVS is always the canonical value. Signed-off-by: Colton Lewis --- arch/arm64/kvm/arm.c | 4 +- arch/arm64/kvm/pmu-direct.c | 153 +++++++++++++++++++++++++++++++++++- arch/arm64/kvm/sys_regs.c | 6 +- include/kvm/arm_pmu.h | 5 ++ 4 files changed, 164 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 8b080804bc90b..75e0f746623ec 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -714,6 +714,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) if (has_vhe()) kvm_vcpu_load_vhe(vcpu); kvm_arch_vcpu_load_fp(vcpu); + kvm_pmu_load(vcpu); kvm_vcpu_pmu_restore_guest(vcpu); if (kvm_arm_is_pvtime_enabled(&vcpu->arch)) kvm_make_request(KVM_REQ_RECORD_STEAL, vcpu); @@ -755,13 +756,14 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) vcpu_set_flag(vcpu, PKVM_HOST_STATE_DIRTY); } + kvm_pmu_put(vcpu); + kvm_vcpu_pmu_restore_host(vcpu); kvm_vcpu_put_debug(vcpu); kvm_arch_vcpu_put_fp(vcpu); if (has_vhe()) kvm_vcpu_put_vhe(vcpu); kvm_timer_vcpu_put(vcpu); kvm_vgic_put(vcpu); - kvm_vcpu_pmu_restore_host(vcpu); if (vcpu_has_nv(vcpu)) kvm_vcpu_put_hw_mmu(vcpu); kvm_arm_vmid_clear_active(); diff --git a/arch/arm64/kvm/pmu-direct.c b/arch/arm64/kvm/pmu-direct.c index 5045051e91cd3..92eec0fa33867 100644 --- a/arch/arm64/kvm/pmu-direct.c +++ b/arch/arm64/kvm/pmu-direct.c @@ -155,7 +155,6 @@ static u64 kvm_vcpu_pmu_guest_counter_mask(struct kvm_vcpu *vcpu) return 0; } - /** * kvm_pmu_guest_counter_mask() - Compute bitmask of guest-reserved counters * @@ -169,3 +168,155 @@ u64 kvm_pmu_guest_counter_mask(void) { return kvm_vcpu_pmu_guest_counter_mask(kvm_get_running_vcpu()); } + +/** + * kvm_pmu_load() - Load untrapped PMU registers + * @vcpu: Pointer to struct kvm_vcpu + * + * Load all untrapped PMU registers from the VCPU into the PCPU. Mask + * to only bits belonging to guest-reserved counters and leave + * host-reserved counters alone in bitmask registers. + */ +void kvm_pmu_load(struct kvm_vcpu *vcpu) +{ + unsigned long guest_counters; + u64 mask; + u8 i; + u64 val; + + /* + * If we aren't guest-owned then we know the guest isn't using + * the PMU anyway, so no need to bother with the swap. + */ + if (!kvm_pmu_is_partitioned(vcpu->kvm)) + return; + + preempt_disable(); + + guest_counters = kvm_vcpu_pmu_guest_counter_mask(vcpu); + + for_each_set_bit(i, &guest_counters, ARMPMU_MAX_HWEVENTS) { + val = __vcpu_sys_reg(vcpu, PMEVCNTR0_EL0 + i); + + if (i == ARMV8_PMU_CYCLE_IDX) + write_pmccntr(val); + else + write_pmevcntrn(i, val); + } + + val = __vcpu_sys_reg(vcpu, PMSELR_EL0); + write_sysreg(val, pmselr_el0); + + if (!(vcpu->arch.mdcr_el2 & MDCR_EL2_TPM)) { + val = __vcpu_sys_reg(vcpu, PMUSERENR_EL0); + write_sysreg(val, pmuserenr_el0); + } + + /* Save only the stateful writable bits. */ + val = __vcpu_sys_reg(vcpu, PMCR_EL0); + mask = ARMV8_PMU_PMCR_MASK & + ~(ARMV8_PMU_PMCR_P | ARMV8_PMU_PMCR_C); + write_sysreg(val & mask, pmcr_el0); + + /* + * When handling these: + * 1. Apply only the bits for guest counters (indicated by mask) + * 2. Use the different registers for set and clear + */ + mask = guest_counters; + + /* Clear the hardware overflow flags so there is no chance of + * creating spurious interrupts. The hardware here is never + * the canonical version anyway. + */ + write_sysreg(mask, pmovsclr_el0); + + val = __vcpu_sys_reg(vcpu, PMCNTENSET_EL0); + write_sysreg(val & mask, pmcntenset_el0); + write_sysreg(~val & mask, pmcntenclr_el0); + + val = __vcpu_sys_reg(vcpu, PMINTENSET_EL1); + write_sysreg(val & mask, pmintenset_el1); + write_sysreg(~val & mask, pmintenclr_el1); + + preempt_enable(); +} + +/** + * kvm_pmu_put() - Put untrapped PMU registers + * @vcpu: Pointer to struct kvm_vcpu + * + * Put all untrapped PMU registers from the VCPU into the PCPU. Mask + * to only bits belonging to guest-reserved counters and leave + * host-reserved counters alone in bitmask registers. + */ +void kvm_pmu_put(struct kvm_vcpu *vcpu) +{ + unsigned long guest_counters; + unsigned long flags; + u64 mask; + u8 i; + u64 val; + + /* + * If we aren't guest-owned then we know the guest is not + * accessing the PMU anyway, so no need to bother with the + * swap. + */ + if (!kvm_pmu_is_partitioned(vcpu->kvm)) + return; + + preempt_disable(); + + guest_counters = kvm_vcpu_pmu_guest_counter_mask(vcpu); + mask = guest_counters; + + /* Mask these to only save the guest relevant bits. */ + val = read_sysreg(pmcntenset_el0); + __vcpu_assign_sys_reg(vcpu, PMCNTENSET_EL0, val & mask); + + val = read_sysreg(pmintenset_el1); + __vcpu_assign_sys_reg(vcpu, PMINTENSET_EL1, val & mask); + + /* Stop guest counters and disable interrupts in hardware first. */ + write_sysreg(mask, pmcntenclr_el0); + write_sysreg(mask, pmintenclr_el1); + isb(); + + for_each_set_bit(i, &guest_counters, ARMPMU_MAX_HWEVENTS) { + if (i == ARMV8_PMU_CYCLE_IDX) + val = read_pmccntr(); + else + val = read_pmevcntrn(i); + + __vcpu_assign_sys_reg(vcpu, PMEVCNTR0_EL0 + i, val); + } + + val = read_sysreg(pmselr_el0); + __vcpu_assign_sys_reg(vcpu, PMSELR_EL0, val); + + if (!(vcpu->arch.mdcr_el2 & MDCR_EL2_TPM)) { + val = read_sysreg(pmuserenr_el0); + __vcpu_assign_sys_reg(vcpu, PMUSERENR_EL0, val); + } + + val = read_sysreg(pmcr_el0); + __vcpu_rmw_sys_reg(vcpu, PMCR_EL0, &=, ~ARMV8_PMU_PMCR_MASK); + __vcpu_rmw_sys_reg(vcpu, PMCR_EL0, |=, val & ARMV8_PMU_PMCR_MASK); + + val = ARMV8_PMU_PMCR_LC; + if (pmu && pmu->pmuver >= ID_AA64DFR0_EL1_PMUVer_V3P5) + val |= ARMV8_PMU_PMCR_LP; + if (vcpu->arch.mdcr_el2 & MDCR_EL2_HPME) + val |= ARMV8_PMU_PMCR_E; + write_sysreg(val, pmcr_el0); + + /* Save pending guest hardware overflows. */ + local_irq_save(flags); + val = read_sysreg(pmovsset_el0); + __vcpu_rmw_sys_reg(vcpu, PMOVSSET_EL0, |=, val & mask); + write_sysreg(val & mask, pmovsclr_el0); + local_irq_restore(flags); + + preempt_enable(); +} diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index c8210a8b10389..ebcf52261df65 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -1189,7 +1189,8 @@ static void pmu_reg_write(struct kvm_vcpu *vcpu, enum vcpu_sysreg reg, u64 val, local_irq_restore(flags); break; case PMUSERENR_EL0: - if (kvm_pmu_is_partitioned(vcpu->kvm)) + if (kvm_pmu_is_partitioned(vcpu->kvm) && + !(vcpu->arch.mdcr_el2 & MDCR_EL2_TPM)) write_sysreg(val, pmuserenr_el0); __vcpu_assign_sys_reg(vcpu, reg, val); break; @@ -1274,7 +1275,8 @@ static u64 pmu_reg_read(struct kvm_vcpu *vcpu, enum vcpu_sysreg reg) local_irq_restore(flags); break; case PMUSERENR_EL0: - if (kvm_pmu_is_partitioned(vcpu->kvm)) + if (kvm_pmu_is_partitioned(vcpu->kvm) && + !(vcpu->arch.mdcr_el2 & MDCR_EL2_TPM)) val = read_sysreg(pmuserenr_el0); else val = __vcpu_sys_reg(vcpu, reg); diff --git a/include/kvm/arm_pmu.h b/include/kvm/arm_pmu.h index a24788243ac99..2604a6a46d5f3 100644 --- a/include/kvm/arm_pmu.h +++ b/include/kvm/arm_pmu.h @@ -102,6 +102,9 @@ void kvm_pmu_direct_pmcr_write(struct kvm_vcpu *vcpu, u64 val); u64 kvm_pmu_direct_pmcr_read(struct kvm_vcpu *vcpu); u64 kvm_pmu_host_counter_mask(void); u64 kvm_pmu_guest_counter_mask(void); +void kvm_pmu_load(struct kvm_vcpu *vcpu); +void kvm_pmu_put(struct kvm_vcpu *vcpu); + /* * Updates the vcpu's view of the pmu events for this cpu. * Must be called before every vcpu run after disabling interrupts, to ensure @@ -150,6 +153,8 @@ static inline u64 kvm_pmu_direct_pmcr_read(struct kvm_vcpu *vcpu) { return 0; } +static inline void kvm_pmu_load(struct kvm_vcpu *vcpu) {} +static inline void kvm_pmu_put(struct kvm_vcpu *vcpu) {} static inline void kvm_pmu_set_counter_value(struct kvm_vcpu *vcpu, u64 select_idx, u64 val) {} static inline void kvm_pmu_set_counter_value_user(struct kvm_vcpu *vcpu, -- 2.56.0.rc1.310.g51773c2048-goog