From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4F079CF8878 for ; Sat, 5 Oct 2024 10:33:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:In-Reply-To:Subject:Cc:To:From:Message-ID:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=OwX+BQDJzAj35+Y0L6AiLAuSBl0DdlKJEJNkNkJjuSI=; b=m4TFBi9McAfTcAeG0QVXF9Qqwn NemkTVasXC1MhWWpXrO1XHn8l3V7xjO2xd/oJQmMuybfSOfs4w0eEL2dH20Kj72/KwviT6uQ7/FGj SdT2xO5q2QBn/nsQaeLbmtKuLtx3DNQvBtQPFThumC65wxeWm9X/kA1MP/2CeLH0hEMyCtjCJ/JYV 6EBmFuwEYuRAIKlr3p9JxW6sWhXTwQ4hTzCFd7Un4ZQbz8d4+9FZrH51xoMDeSGXFfGrGiKuLcKGn +QAbp9G304aMVHc1wSI9DT0eu3bA+p/c9MyCTgJKnv0lfLc/35zNmksd5Nv5xMhUXnwbsyhGcQKfU 8gfJ5iMg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sx262-0000000FCiu-1CVT; Sat, 05 Oct 2024 10:33:10 +0000 Received: from nyc.source.kernel.org ([147.75.193.91]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1sx24k-0000000FCPc-14mK for linux-arm-kernel@lists.infradead.org; Sat, 05 Oct 2024 10:31:51 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id AF2FCA40390; Sat, 5 Oct 2024 10:31:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D2FAEC4CEC2; Sat, 5 Oct 2024 10:31:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728124308; bh=dRfL9j9fox9VAJqc0lt4h4UbPL3AelM/FiWAIcNnBLk=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=NaA1b/izRRpGy/Uf2jvZ6tDEg8GmC5EAzwKkwpPGT531aHilzWSVikSp/xeg3vJXF AZ1MKt49KnY28ZaOyxgw1RGpt1RAgVn0Ee5euMoiu7BzSghRMkAT/LDmHjIq4SEfMk YJQEOhI/KpjzA0tziqpWGorvzg2U87cFgi9K+BEdJy8a8/Fxqn9xQOM65K+Ikl7EkU SarcI7gNmfRfw4gyniazOgpihH4zU1+snZ0RKQc+UXxqyaqNSwJMWePibMVGO0XgsZ cYyQNqt6neknw0DZRQU3N/0GuqhqLlDCLjahFfPViCE7AZUJIUjAMwZeKWtxl+tysE iO4u/m4eQI9kA== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sx24g-000YxZ-Jq; Sat, 05 Oct 2024 11:31:46 +0100 Date: Sat, 05 Oct 2024 11:31:45 +0100 Message-ID: <864j5q7sdq.wl-maz@kernel.org> From: Marc Zyngier To: Ahmad Fatoum Cc: Peter Maydell , qemu-arm@nongnu.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, Pengutronix Kernel Team , "linux-arm-kernel@lists.infradead.org" , Enrico Joerns Subject: Re: [BUG] ARM64 KVM: Data abort executing post-indexed LDR on MMIO address In-Reply-To: <4d559b9e-c208-46f3-851a-68086dc8a50f@pengutronix.de> References: <89f184d6-5b61-4c77-9f3b-c0a8f6a75d60@pengutronix.de> <4d559b9e-c208-46f3-851a-68086dc8a50f@pengutronix.de> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.4 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: a.fatoum@pengutronix.de, peter.maydell@linaro.org, qemu-arm@nongnu.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, kernel@pengutronix.de, linux-arm-kernel@lists.infradead.org, ejo@pengutronix.de X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241005_033150_448185_F83500F6 X-CRM114-Status: GOOD ( 23.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, 04 Oct 2024 20:50:18 +0100, Ahmad Fatoum wrote: > > Hi, > > On 04.10.24 14:10, Peter Maydell wrote: > > On Fri, 4 Oct 2024 at 12:51, Ahmad Fatoum wrote: > >> On 04.10.24 12:40, Peter Maydell wrote: > >>> Don't do this -- KVM doesn't support it. For access to MMIO, > >>> stick to instructions which will set the ISV bit in ESR_EL1. > >>> > >>> That is: > >>> > >>> * AArch64 loads and stores of a single general-purpose register > >>> (including the register specified with 0b11111, including those > >>> with Acquire/Release semantics, but excluding Load Exclusive > >>> or Store Exclusive and excluding those with writeback). > >>> * AArch32 instructions where the instruction: > >>> - Is an LDR, LDA, LDRT, LDRSH, LDRSHT, LDRH, LDAH, LDRHT, > >>> LDRSB, LDRSBT, LDRB, LDAB, LDRBT, STR, STL, STRT, STRH, > >>> STLH, STRHT, STRB, STLB, or STRBT instruction. > >>> - Is not performing register writeback. > >>> - Is not using R15 as a source or destination register. > >>> > >>> Your instruction is doing writeback. Do the address update > >>> as a separate instruction. > > With readl/writel implemented in assembly, I get beyond that point, but > now I get a data abort running an DC IVAC instruction on address 0x1000, > where the cfi-flash is located. This instruction is part of a routine > to remap the cfi-flash to start a page later, so the zero page can be > mapped faulting. > > Simple reproducer: > > start: > ldr x0, =0x1000 > ldr x1, =0x1040 > bl v8_inv_dcache_range > > mov w10, '!' > bl putch > > ret > > v8_inv_dcache_range: > mrs x3, ctr_el0 > lsr x3, x3, #16 > and x3, x3, #0xf > mov x2, #0x4 > lsl x2, x2, x3 > sub x3, x2, #0x1 > bic x0, x0, x3 > 1: > dc ivac, x0 > add x0, x0, x2 > cmp x0, x1 > b.cc 1b > dsb sy > ret > > This prints ! without KVM, but triggers a data abort before that with -enable-kvm: > > DABT (current EL) exception (ESR 0x96000010) at 0x0000000000001000 > elr: 000000007fbe0550 lr : 000000007fbe01ac > [snip] > > Call trace: > [<7fbe0550>] (v8_inv_dcache_range+0x1c/0x34) from [<7fbe0218>] (arch_remap_range+0x64/0x70) > [<7fbe0218>] (arch_remap_range+0x64/0x70) from [<7fb8795c>] (of_platform_device_create+0x1e8/0x22c) > [<7fb8795c>] (of_platform_device_create+0x1e8/0x22c) from [<7fb87a04>] (of_platform_bus_create+0x64/0xbc) > [snip] > > Any idea what this is about? IIRC, the QEMU flash is implemented as a read-only memslot. A data cache invalidation is a *write*, as it can be (and is) upgraded to a clean+invalidate by the HW. KVM cannot satisfy the write, for obvious reasons, and tells the guest to bugger off (__gfn_to_pfn_memslot() returns KVM_PFN_ERR_RO_FAULT, which satisfies is_error_noslot_pfn() -- a slight oddity, but hey, why not). In the end, you get an exception. We could relax this by special-casing CMOs to RO memslots, but this doesn't look great. The real question is: what are you trying to achieve with this? M. -- Without deviation from the norm, progress is not possible.