From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 991D9C74A5B for ; Wed, 29 Mar 2023 12:04:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=nz/W4qSGhKKNoKPpPXgDNS4aHoZLUhTXqM5SoalZ98g=; b=NwyKDF8Pbb0Wvg xIOwY/Ulai2PvZtwOBD2HWM9KPvAao81yBL0I0PLSP/bnAYi8P/URHNMqqrrGv6C1z2lBj6mpUIut UumKGtAH0yioudoW+HoeQhh7ryZtyoz5qJnqVxoP+dhKXyuiDBk/QF2IAv14CChO74ieUnBRUFJ15 zVH6QJdpGbx2Z9nyNH1P/8S1xNPyFgiWrLWZfGh0ZpAJTIfDyLs2WQJXDFFWU3vYBJlcNNkFiN0i2 Hq7oDe2e8Eqr8meWDm246IiPR9GLELa2HWcIeMYmK4ix4R86JONWOZYizdo1v+324VVnlrbF2AmKN 1bjDB7AIQycViO2opIzA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1phUWa-0007lf-0G; Wed, 29 Mar 2023 12:03:32 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1phUWP-0007iY-2w for linux-arm-kernel@lists.infradead.org; Wed, 29 Mar 2023 12:03:26 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id E13CF61CE4; Wed, 29 Mar 2023 12:03:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AC6D2C433EF; Wed, 29 Mar 2023 12:03:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1680091400; bh=fflAurITT6wZZq4CT/l1tDmWHMLH9l/Mga/pG41Cz+M=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=AjBvwfsGmjKar/CDpcP8h8PH+aRHUZ2RQa3IWgToLnrNOFCJOcY5Xjl42kYc0gzpk CngXUH8UxRVGEO08KMsCAQzYDNUHl9ieS6dwiYwTpksdyBsBtc6EQ8hOoGKeYqBvcV LucHyo1CSRUrvGg1OoL9plvm0shcSYLEZ4pvMmVDkfpKO4v8BWLZeTyrW0UJi1eTEV 8elo/FT86F7Q2QgahBCvFnOh6iNMU3QMwu6lXa+GFsiiCWK1Q/2jXnm8hDN816aLLP 8JmIEcedx9cU60vLNty5r1wgI1NDLRTDze2sfePn6klhZo+o2KEabkPip8f/41/7Rt OdTk/i5tkOY4A== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1phUWM-0042VN-C4; Wed, 29 Mar 2023 13:03:18 +0100 Date: Wed, 29 Mar 2023 13:03:18 +0100 Message-ID: <86jzyzwyrd.wl-maz@kernel.org> From: Marc Zyngier To: Reiji Watanabe Cc: Oliver Upton , kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, James Morse , Alexandru Elisei , Zenghui Yu , Suzuki K Poulose , Paolo Bonzini , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , Will Deacon , Rob Herring Subject: Re: [PATCH v1 2/2] KVM: arm64: PMU: Ensure to trap PMU access from EL0 to EL2 In-Reply-To: <20230329002136.2463442-3-reijiw@google.com> References: <20230329002136.2463442-1-reijiw@google.com> <20230329002136.2463442-3-reijiw@google.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) X-TUID: VeERMf+sjuAB MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: reijiw@google.com, oliver.upton@linux.dev, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, alexandru.elisei@arm.com, yuzenghui@huawei.com, suzuki.poulose@arm.com, pbonzini@redhat.com, ricarkol@google.com, jingzhangos@google.com, rananta@google.com, will@kernel.org, robh@kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230329_050322_044510_38C06B5C X-CRM114-Status: GOOD ( 33.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, 29 Mar 2023 01:21:36 +0100, Reiji Watanabe wrote: > > Currently, with VHE, KVM sets ER, CR, SW and EN bits of > PMUSERENR_EL0 to 1 on vcpu_load(). So, if the value of those bits > are cleared after vcpu_load() (the perf subsystem would do when PMU > counters are programmed for the guest), PMU access from the guest EL0 > might be trapped to the guest EL1 directly regardless of the current > PMUSERENR_EL0 value of the vCPU. + RobH. Is that what is done when the event is created and armv8pmu_start() called? This is... crap. The EL0 access thing breaks everything, and nobody tested it with KVM, obviously. I would be tempted to start mitigating it with the following: diff --git a/arch/arm64/kernel/perf_event.c b/arch/arm64/kernel/perf_event.c index dde06c0f97f3..8063525bf3dd 100644 --- a/arch/arm64/kernel/perf_event.c +++ b/arch/arm64/kernel/perf_event.c @@ -806,17 +806,19 @@ static void armv8pmu_disable_event(struct perf_event *event) static void armv8pmu_start(struct arm_pmu *cpu_pmu) { - struct perf_event_context *ctx; - int nr_user = 0; + if (sysctl_perf_user_access) { + struct perf_event_context *ctx; + int nr_user = 0; - ctx = perf_cpu_task_ctx(); - if (ctx) - nr_user = ctx->nr_user; + ctx = perf_cpu_task_ctx(); + if (ctx) + nr_user = ctx->nr_user; - if (sysctl_perf_user_access && nr_user) - armv8pmu_enable_user_access(cpu_pmu); - else - armv8pmu_disable_user_access(); + if (nr_user) + armv8pmu_enable_user_access(cpu_pmu); + else + armv8pmu_disable_user_access(); + } /* Enable all counters */ armv8pmu_pmcr_write(armv8pmu_pmcr_read() | ARMV8_PMU_PMCR_E); but that's obviously not enough as we want it to work with EL0 access enabled on the host as well. What we miss is something that tells the PMU code "we're in a context where host userspace isn't present", and this would be completely skipped, relying on KVM to restore the appropriate state on vcpu_put(). But then the IPI stuff that controls EL0 can always come in and wreck things. Gahhh... I'm a bit reluctant to use the "save/restore all the time" hammer, because it only hides that the EL0 counter infrastructure is a bit broken. > With VHE, fix this by setting those bits of the register on every > guest entry (as with nVHE). Also, opportunistically make the similar > change for PMSELR_EL0, which is cleared by vcpu_load(), to ensure it > is always set to zero on guest entry (PMXEVCNTR_EL0 access might cause > UNDEF at EL1 instead of being trapped to EL2, depending on the value > of PMSELR_EL0). I think that would be more robust, although I don't > find any kernel code that writes PMSELR_EL0. This was changed a while ago to avoid using the selection register, see 0fdf1bb75953 ("arm64: perf: Avoid PMXEV* indirection"), and the rationale behind the reset of PMSELR_EL0 in 21cbe3cc8a48 ("arm64: KVM: pmu: Reset PMSELR_EL0.SEL to a sane value before entering the guest"). We *could* simply drop this zeroing of PMSELR_EL0 now that there is nothing else host-side that writes to it. But we need to agree on how to fix the above first. Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel