From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6AE9313CABC; Sat, 5 Oct 2024 10:31:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728124309; cv=none; b=fnAZ2dILRZccsaNNeQaZld4SLZTcEbwbzGnffAKbFUCMQp/Psiwhz5yB+vpgQAuXxD+L/ySQyQSHvcJyIsoX67ls2xuCfHhTmchY3aoRvOm23p2ChhNZXQEVbeehWB6KnqukmNNQQDvzGEmXBCIQ6DGBq9b/608t6GemQ3Pkly0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728124309; c=relaxed/simple; bh=dRfL9j9fox9VAJqc0lt4h4UbPL3AelM/FiWAIcNnBLk=; h=Date:Message-ID:From:To:Cc:Subject:In-Reply-To:References: MIME-Version:Content-Type; b=L0fbRCL84dCv07y0gUB1kHubwzLJ3pechk5VLFCtG/nM7va1QGwVrN/jYBF2am0/5hunnMvZC2hZxr3T9LAnXhttQyQA7QHVqgUuHwz8JkXY0anPmYry79tbsgHnJAmpG51GfdRzXE1Tjj0rseo5iJ5NQYjZb8ah/jeO0Utz3yk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NaA1b/iz; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NaA1b/iz" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D2FAEC4CEC2; Sat, 5 Oct 2024 10:31:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728124308; bh=dRfL9j9fox9VAJqc0lt4h4UbPL3AelM/FiWAIcNnBLk=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=NaA1b/izRRpGy/Uf2jvZ6tDEg8GmC5EAzwKkwpPGT531aHilzWSVikSp/xeg3vJXF AZ1MKt49KnY28ZaOyxgw1RGpt1RAgVn0Ee5euMoiu7BzSghRMkAT/LDmHjIq4SEfMk YJQEOhI/KpjzA0tziqpWGorvzg2U87cFgi9K+BEdJy8a8/Fxqn9xQOM65K+Ikl7EkU SarcI7gNmfRfw4gyniazOgpihH4zU1+snZ0RKQc+UXxqyaqNSwJMWePibMVGO0XgsZ cYyQNqt6neknw0DZRQU3N/0GuqhqLlDCLjahFfPViCE7AZUJIUjAMwZeKWtxl+tysE iO4u/m4eQI9kA== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sx24g-000YxZ-Jq; Sat, 05 Oct 2024 11:31:46 +0100 Date: Sat, 05 Oct 2024 11:31:45 +0100 Message-ID: <864j5q7sdq.wl-maz@kernel.org> From: Marc Zyngier To: Ahmad Fatoum Cc: Peter Maydell , qemu-arm@nongnu.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, Pengutronix Kernel Team , "linux-arm-kernel@lists.infradead.org" , Enrico Joerns Subject: Re: [BUG] ARM64 KVM: Data abort executing post-indexed LDR on MMIO address In-Reply-To: <4d559b9e-c208-46f3-851a-68086dc8a50f@pengutronix.de> References: <89f184d6-5b61-4c77-9f3b-c0a8f6a75d60@pengutronix.de> <4d559b9e-c208-46f3-851a-68086dc8a50f@pengutronix.de> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.4 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: a.fatoum@pengutronix.de, peter.maydell@linaro.org, qemu-arm@nongnu.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, kernel@pengutronix.de, linux-arm-kernel@lists.infradead.org, ejo@pengutronix.de X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Fri, 04 Oct 2024 20:50:18 +0100, Ahmad Fatoum wrote: > > Hi, > > On 04.10.24 14:10, Peter Maydell wrote: > > On Fri, 4 Oct 2024 at 12:51, Ahmad Fatoum wrote: > >> On 04.10.24 12:40, Peter Maydell wrote: > >>> Don't do this -- KVM doesn't support it. For access to MMIO, > >>> stick to instructions which will set the ISV bit in ESR_EL1. > >>> > >>> That is: > >>> > >>> * AArch64 loads and stores of a single general-purpose register > >>> (including the register specified with 0b11111, including those > >>> with Acquire/Release semantics, but excluding Load Exclusive > >>> or Store Exclusive and excluding those with writeback). > >>> * AArch32 instructions where the instruction: > >>> - Is an LDR, LDA, LDRT, LDRSH, LDRSHT, LDRH, LDAH, LDRHT, > >>> LDRSB, LDRSBT, LDRB, LDAB, LDRBT, STR, STL, STRT, STRH, > >>> STLH, STRHT, STRB, STLB, or STRBT instruction. > >>> - Is not performing register writeback. > >>> - Is not using R15 as a source or destination register. > >>> > >>> Your instruction is doing writeback. Do the address update > >>> as a separate instruction. > > With readl/writel implemented in assembly, I get beyond that point, but > now I get a data abort running an DC IVAC instruction on address 0x1000, > where the cfi-flash is located. This instruction is part of a routine > to remap the cfi-flash to start a page later, so the zero page can be > mapped faulting. > > Simple reproducer: > > start: > ldr x0, =0x1000 > ldr x1, =0x1040 > bl v8_inv_dcache_range > > mov w10, '!' > bl putch > > ret > > v8_inv_dcache_range: > mrs x3, ctr_el0 > lsr x3, x3, #16 > and x3, x3, #0xf > mov x2, #0x4 > lsl x2, x2, x3 > sub x3, x2, #0x1 > bic x0, x0, x3 > 1: > dc ivac, x0 > add x0, x0, x2 > cmp x0, x1 > b.cc 1b > dsb sy > ret > > This prints ! without KVM, but triggers a data abort before that with -enable-kvm: > > DABT (current EL) exception (ESR 0x96000010) at 0x0000000000001000 > elr: 000000007fbe0550 lr : 000000007fbe01ac > [snip] > > Call trace: > [<7fbe0550>] (v8_inv_dcache_range+0x1c/0x34) from [<7fbe0218>] (arch_remap_range+0x64/0x70) > [<7fbe0218>] (arch_remap_range+0x64/0x70) from [<7fb8795c>] (of_platform_device_create+0x1e8/0x22c) > [<7fb8795c>] (of_platform_device_create+0x1e8/0x22c) from [<7fb87a04>] (of_platform_bus_create+0x64/0xbc) > [snip] > > Any idea what this is about? IIRC, the QEMU flash is implemented as a read-only memslot. A data cache invalidation is a *write*, as it can be (and is) upgraded to a clean+invalidate by the HW. KVM cannot satisfy the write, for obvious reasons, and tells the guest to bugger off (__gfn_to_pfn_memslot() returns KVM_PFN_ERR_RO_FAULT, which satisfies is_error_noslot_pfn() -- a slight oddity, but hey, why not). In the end, you get an exception. We could relax this by special-casing CMOs to RO memslots, but this doesn't look great. The real question is: what are you trying to achieve with this? M. -- Without deviation from the norm, progress is not possible.