From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 084556AB6 for ; Thu, 12 Jan 2023 12:38:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BA016C433F0; Thu, 12 Jan 2023 12:38:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1673527134; bh=QuIuSKe9BuOyhqawV9dcZNTeA/CmLCt8kn8h7pV5oaI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=acgsMkEjh8QKqnh2cuwadHbb4yl8y5ty0LuiVviZKQ/oHTxgo+A/liLeGuX8Q/osi BGq+3UZyMVOkDcOs6XYAnibun6s16Hyt5XRxGr2zxAkzn4DV4oag1srgJqOYrMoCzj vDjaaAZG9YltT2s7riMaiwMgnOrOqbyFNLICKsNAilBvlgL6e0TaTWtsTPFQLocC8q hndLOTuVHIOMbhLoxeS2lUQpnQjaothczFaBZQL18ENylv6XonhtY4SZkVYIY6igx5 JdqZIj51epDo3/qjjNtcEjvVJtcve+jBjscpClg4p7izek1cfPfJ7yuFP+nLOLIgWx Z0GEkHZeP8MfA== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1pFwr6-001BBa-Tz; Thu, 12 Jan 2023 12:38:52 +0000 From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, , , kvm@vger.kernel.org Cc: D Scott Phillips , Ganapatrao Kulkarni , James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu Subject: [PATCH 2/3] KVM: arm64: Reduce overhead of trapped timer sysreg accesses Date: Thu, 12 Jan 2023 12:38:28 +0000 Message-Id: <20230112123829.458912-3-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230112123829.458912-1-maz@kernel.org> References: <20230112123829.458912-1-maz@kernel.org> Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvmarm@lists.linux.dev, kvm@vger.kernel.org, scott@os.amperecomputing.com, gankulkarni@os.amperecomputing.com, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Each read/write to a trapped timer system register results in a whole kvm_timer_vcpu_put/load() cycle which affects all of the timers, and a bit more. There is no need for such a thing, and we can limit the impact to the timer being affected, and only this one. This drastically simplifies the emulated case, and limits the damage for trapped accesses. This also brings some performance back for NV. Whilst we're at it, fix a comment that didn't quite capture why we always set CNTVOFF_EL2 to 0 when disabling the virtual timer. Signed-off-by: Marc Zyngier --- arch/arm64/kvm/arch_timer.c | 73 ++++++++++++++++++++++++------------- 1 file changed, 48 insertions(+), 25 deletions(-) diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c index 587d87aec33f..1a1d7e258aba 100644 --- a/arch/arm64/kvm/arch_timer.c +++ b/arch/arm64/kvm/arch_timer.c @@ -434,6 +434,11 @@ static void timer_emulate(struct arch_timer_context *ctx) soft_timer_start(&ctx->hrtimer, kvm_timer_compute_delta(ctx)); } +static void set_cntvoff(u64 cntvoff) +{ + kvm_call_hyp(__kvm_timer_set_cntvoff, cntvoff); +} + static void timer_save_state(struct arch_timer_context *ctx) { struct arch_timer_cpu *timer = vcpu_timer(ctx->vcpu); @@ -457,6 +462,22 @@ static void timer_save_state(struct arch_timer_context *ctx) write_sysreg_el0(0, SYS_CNTV_CTL); isb(); + /* + * The kernel may decide to run userspace after + * calling vcpu_put, so we reset cntvoff to 0 to + * ensure a consistent read between user accesses to + * the virtual counter and kernel access to the + * physical counter of non-VHE case. + * + * For VHE, the virtual counter uses a fixed virtual + * offset of zero, so no need to zero CNTVOFF_EL2 + * register, but this is actually useful when switching + * between EL1/vEL2 with NV. + * + * Do it unconditionally, as this is either unavoidable + * or dirt cheap. + */ + set_cntvoff(0); break; case TIMER_PTIMER: timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTP_CTL)); @@ -530,6 +551,7 @@ static void timer_restore_state(struct arch_timer_context *ctx) switch (index) { case TIMER_VTIMER: + set_cntvoff(timer_get_offset(ctx)); write_sysreg_el0(timer_get_cval(ctx), SYS_CNTV_CVAL); isb(); write_sysreg_el0(timer_get_ctl(ctx), SYS_CNTV_CTL); @@ -550,11 +572,6 @@ static void timer_restore_state(struct arch_timer_context *ctx) local_irq_restore(flags); } -static void set_cntvoff(u64 cntvoff) -{ - kvm_call_hyp(__kvm_timer_set_cntvoff, cntvoff); -} - static inline void set_timer_irq_phys_active(struct arch_timer_context *ctx, bool active) { int r; @@ -629,8 +646,6 @@ void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu) kvm_timer_vcpu_load_nogic(vcpu); } - set_cntvoff(timer_get_offset(map.direct_vtimer)); - kvm_timer_unblocking(vcpu); timer_restore_state(map.direct_vtimer); @@ -686,15 +701,6 @@ void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu) if (kvm_vcpu_is_blocking(vcpu)) kvm_timer_blocking(vcpu); - - /* - * The kernel may decide to run userspace after calling vcpu_put, so - * we reset cntvoff to 0 to ensure a consistent read between user - * accesses to the virtual counter and kernel access to the physical - * counter of non-VHE case. For VHE, the virtual counter uses a fixed - * virtual offset of zero, so no need to zero CNTVOFF_EL2 register. - */ - set_cntvoff(0); } /* @@ -924,14 +930,22 @@ u64 kvm_arm_timer_read_sysreg(struct kvm_vcpu *vcpu, enum kvm_arch_timers tmr, enum kvm_arch_timer_regs treg) { + struct arch_timer_context *timer; + struct timer_map map; u64 val; + get_timer_map(vcpu, &map); + timer = vcpu_get_timer(vcpu, tmr); + + if (timer == map.emul_ptimer) + return kvm_arm_timer_read(vcpu, timer, treg); + preempt_disable(); - kvm_timer_vcpu_put(vcpu); + timer_save_state(timer); - val = kvm_arm_timer_read(vcpu, vcpu_get_timer(vcpu, tmr), treg); + val = kvm_arm_timer_read(vcpu, timer, treg); - kvm_timer_vcpu_load(vcpu); + timer_restore_state(timer); preempt_enable(); return val; @@ -965,13 +979,22 @@ void kvm_arm_timer_write_sysreg(struct kvm_vcpu *vcpu, enum kvm_arch_timer_regs treg, u64 val) { - preempt_disable(); - kvm_timer_vcpu_put(vcpu); - - kvm_arm_timer_write(vcpu, vcpu_get_timer(vcpu, tmr), treg, val); + struct arch_timer_context *timer; + struct timer_map map; - kvm_timer_vcpu_load(vcpu); - preempt_enable(); + get_timer_map(vcpu, &map); + timer = vcpu_get_timer(vcpu, tmr); + if (timer == map.emul_ptimer) { + soft_timer_cancel(&timer->hrtimer); + kvm_arm_timer_write(vcpu, timer, treg, val); + timer_emulate(timer); + } else { + preempt_disable(); + timer_save_state(timer); + kvm_arm_timer_write(vcpu, timer, treg, val); + timer_restore_state(timer); + preempt_enable(); + } } static int kvm_timer_starting_cpu(unsigned int cpu) -- 2.34.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 90349C677F1 for ; Thu, 12 Jan 2023 12:40:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=mQmNZAidGHwJyeVd+W51wmIR3zTDCB+OjDRNIV7gdkU=; b=J+Nw7oCHZQ3qwt tjJ+jmW1cXNVj15ZoEk4FSw5EbEmWBYaAqcEuLhvg5UGZNByLTe6mdpCPJrg27c4sWsyRjilk8Y4M ZKyermWt+DCXwZbTebNRovy89LHXb/vkpJZgLMNuM5QKhFrZ/5disA/oZGKGaHqjr3UI+HWWZXOno F1LtGACpbFzyIw0KNNbzi3Wp2a54rVFJLPVhJ0yc+p5/1KUJzLO6WI3AF+72IuGIhf0tQDH6N30Qo 5Gmzw0bjMT5Xu3C8kyLhmQfVKrVCE54yRkGWEi2nQJ9Ih9ZTUkRK5a4NLnJ2TCI9fozHZ1wSiBm/e g3zAlRcy5dGL1hP5UZkA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pFwrM-00F0ka-5b; Thu, 12 Jan 2023 12:39:08 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pFwrA-00F0hg-UK for linux-arm-kernel@lists.infradead.org; Thu, 12 Jan 2023 12:38:59 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 56EB260A4C; Thu, 12 Jan 2023 12:38:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BA016C433F0; Thu, 12 Jan 2023 12:38:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1673527134; bh=QuIuSKe9BuOyhqawV9dcZNTeA/CmLCt8kn8h7pV5oaI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=acgsMkEjh8QKqnh2cuwadHbb4yl8y5ty0LuiVviZKQ/oHTxgo+A/liLeGuX8Q/osi BGq+3UZyMVOkDcOs6XYAnibun6s16Hyt5XRxGr2zxAkzn4DV4oag1srgJqOYrMoCzj vDjaaAZG9YltT2s7riMaiwMgnOrOqbyFNLICKsNAilBvlgL6e0TaTWtsTPFQLocC8q hndLOTuVHIOMbhLoxeS2lUQpnQjaothczFaBZQL18ENylv6XonhtY4SZkVYIY6igx5 JdqZIj51epDo3/qjjNtcEjvVJtcve+jBjscpClg4p7izek1cfPfJ7yuFP+nLOLIgWx Z0GEkHZeP8MfA== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1pFwr6-001BBa-Tz; Thu, 12 Jan 2023 12:38:52 +0000 From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, , , kvm@vger.kernel.org Cc: D Scott Phillips , Ganapatrao Kulkarni , James Morse , Suzuki K Poulose , Oliver Upton , Zenghui Yu Subject: [PATCH 2/3] KVM: arm64: Reduce overhead of trapped timer sysreg accesses Date: Thu, 12 Jan 2023 12:38:28 +0000 Message-Id: <20230112123829.458912-3-maz@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230112123829.458912-1-maz@kernel.org> References: <20230112123829.458912-1-maz@kernel.org> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvmarm@lists.linux.dev, kvm@vger.kernel.org, scott@os.amperecomputing.com, gankulkarni@os.amperecomputing.com, james.morse@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, yuzenghui@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230112_043857_101327_67D5418D X-CRM114-Status: GOOD ( 20.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Each read/write to a trapped timer system register results in a whole kvm_timer_vcpu_put/load() cycle which affects all of the timers, and a bit more. There is no need for such a thing, and we can limit the impact to the timer being affected, and only this one. This drastically simplifies the emulated case, and limits the damage for trapped accesses. This also brings some performance back for NV. Whilst we're at it, fix a comment that didn't quite capture why we always set CNTVOFF_EL2 to 0 when disabling the virtual timer. Signed-off-by: Marc Zyngier --- arch/arm64/kvm/arch_timer.c | 73 ++++++++++++++++++++++++------------- 1 file changed, 48 insertions(+), 25 deletions(-) diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c index 587d87aec33f..1a1d7e258aba 100644 --- a/arch/arm64/kvm/arch_timer.c +++ b/arch/arm64/kvm/arch_timer.c @@ -434,6 +434,11 @@ static void timer_emulate(struct arch_timer_context *ctx) soft_timer_start(&ctx->hrtimer, kvm_timer_compute_delta(ctx)); } +static void set_cntvoff(u64 cntvoff) +{ + kvm_call_hyp(__kvm_timer_set_cntvoff, cntvoff); +} + static void timer_save_state(struct arch_timer_context *ctx) { struct arch_timer_cpu *timer = vcpu_timer(ctx->vcpu); @@ -457,6 +462,22 @@ static void timer_save_state(struct arch_timer_context *ctx) write_sysreg_el0(0, SYS_CNTV_CTL); isb(); + /* + * The kernel may decide to run userspace after + * calling vcpu_put, so we reset cntvoff to 0 to + * ensure a consistent read between user accesses to + * the virtual counter and kernel access to the + * physical counter of non-VHE case. + * + * For VHE, the virtual counter uses a fixed virtual + * offset of zero, so no need to zero CNTVOFF_EL2 + * register, but this is actually useful when switching + * between EL1/vEL2 with NV. + * + * Do it unconditionally, as this is either unavoidable + * or dirt cheap. + */ + set_cntvoff(0); break; case TIMER_PTIMER: timer_set_ctl(ctx, read_sysreg_el0(SYS_CNTP_CTL)); @@ -530,6 +551,7 @@ static void timer_restore_state(struct arch_timer_context *ctx) switch (index) { case TIMER_VTIMER: + set_cntvoff(timer_get_offset(ctx)); write_sysreg_el0(timer_get_cval(ctx), SYS_CNTV_CVAL); isb(); write_sysreg_el0(timer_get_ctl(ctx), SYS_CNTV_CTL); @@ -550,11 +572,6 @@ static void timer_restore_state(struct arch_timer_context *ctx) local_irq_restore(flags); } -static void set_cntvoff(u64 cntvoff) -{ - kvm_call_hyp(__kvm_timer_set_cntvoff, cntvoff); -} - static inline void set_timer_irq_phys_active(struct arch_timer_context *ctx, bool active) { int r; @@ -629,8 +646,6 @@ void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu) kvm_timer_vcpu_load_nogic(vcpu); } - set_cntvoff(timer_get_offset(map.direct_vtimer)); - kvm_timer_unblocking(vcpu); timer_restore_state(map.direct_vtimer); @@ -686,15 +701,6 @@ void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu) if (kvm_vcpu_is_blocking(vcpu)) kvm_timer_blocking(vcpu); - - /* - * The kernel may decide to run userspace after calling vcpu_put, so - * we reset cntvoff to 0 to ensure a consistent read between user - * accesses to the virtual counter and kernel access to the physical - * counter of non-VHE case. For VHE, the virtual counter uses a fixed - * virtual offset of zero, so no need to zero CNTVOFF_EL2 register. - */ - set_cntvoff(0); } /* @@ -924,14 +930,22 @@ u64 kvm_arm_timer_read_sysreg(struct kvm_vcpu *vcpu, enum kvm_arch_timers tmr, enum kvm_arch_timer_regs treg) { + struct arch_timer_context *timer; + struct timer_map map; u64 val; + get_timer_map(vcpu, &map); + timer = vcpu_get_timer(vcpu, tmr); + + if (timer == map.emul_ptimer) + return kvm_arm_timer_read(vcpu, timer, treg); + preempt_disable(); - kvm_timer_vcpu_put(vcpu); + timer_save_state(timer); - val = kvm_arm_timer_read(vcpu, vcpu_get_timer(vcpu, tmr), treg); + val = kvm_arm_timer_read(vcpu, timer, treg); - kvm_timer_vcpu_load(vcpu); + timer_restore_state(timer); preempt_enable(); return val; @@ -965,13 +979,22 @@ void kvm_arm_timer_write_sysreg(struct kvm_vcpu *vcpu, enum kvm_arch_timer_regs treg, u64 val) { - preempt_disable(); - kvm_timer_vcpu_put(vcpu); - - kvm_arm_timer_write(vcpu, vcpu_get_timer(vcpu, tmr), treg, val); + struct arch_timer_context *timer; + struct timer_map map; - kvm_timer_vcpu_load(vcpu); - preempt_enable(); + get_timer_map(vcpu, &map); + timer = vcpu_get_timer(vcpu, tmr); + if (timer == map.emul_ptimer) { + soft_timer_cancel(&timer->hrtimer); + kvm_arm_timer_write(vcpu, timer, treg, val); + timer_emulate(timer); + } else { + preempt_disable(); + timer_save_state(timer); + kvm_arm_timer_write(vcpu, timer, treg, val); + timer_restore_state(timer); + preempt_enable(); + } } static int kvm_timer_starting_cpu(unsigned int cpu) -- 2.34.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel