From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C2982CFB43F for ; Sat, 5 Oct 2024 22:47:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:In-Reply-To:Subject:Cc:To:From:Message-ID:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/ph5e/ooB5DbMO9pPsaPBZoZvZyscRnw9zmtso6XVLE=; b=IR1IGyZD/0ZV3qjYGNI1anWNRu vY/zuievb15SyAdtYAj0TjTOxO9uhq3e3jHOBKR3qkgheXwC9EpwzzCYFTcZgcNRVEXHxW9LMD3Q1 hKQMyXIk7eHrce3If3HTRiyojaTfMrnHNTlFuaPlyvruSEgaE8ptKaN5MtN4KqKjGFgG34pMBuSGC 9yirL0XqXlI24cdUGdl6g2YWmvU7qAVmB7C384Rcxi//AzJMuqm7aoUbEQdu9ECXk82FwA8/+3qTa 7+E+wyujlCt3DVnpVXgQXcv6f+fEmCS2YDg79pw62OhORMFCmA6IMMhKkPEK66TDRhteA7NVCm9re QCfSwawg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sxDYR-0000000G46O-0xhp; Sat, 05 Oct 2024 22:47:15 +0000 Received: from nyc.source.kernel.org ([147.75.193.91]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1sxCRN-0000000FzzW-20hH for linux-arm-kernel@lists.infradead.org; Sat, 05 Oct 2024 21:35:56 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id EC16EA406E5; Sat, 5 Oct 2024 21:35:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 21F30C4CECC; Sat, 5 Oct 2024 21:35:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728164152; bh=4sfPDWl1oJaJQh9DQfDD4n4dpgiSjjXdaE92+qki+no=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=nOpfuY9RkAvbGr1+JirOtWdENC8a8hWxm/fOFqgfo4xdotxuvlbrkvdQJVCOzVYed oEDTlkg4EHoaUSYvw0EafAfcLbV2SAfH53l8Zgyv7dx1xfKDWsf269E3LGWmsOdtyn 2uf1vXvDzqMiUOUJlwUe3t+/mGiAjlT/khk8qmaKdIY2Ww+Rwxq96pP4K9MWFCqdZe p1FcHyhU0XyArpT9kczq4YZeT1imC4CahdX8/sP+kuFmwkPl9d/tXGTunaPlxCdl6d 0SiTSmoOXKMRIbhrJUw9dOnDDfJnJus7M+I/qcnyadOv8wdrsU0Vc4UocmvcjLC6h0 uqhqGrSdG5krA== Received: from sofa.misterjones.org ([185.219.108.64] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1sxCRK-000h6U-1X; Sat, 05 Oct 2024 22:35:50 +0100 Date: Sat, 05 Oct 2024 22:35:49 +0100 Message-ID: <87a5fiutai.wl-maz@kernel.org> From: Marc Zyngier To: Ahmad Fatoum Cc: Peter Maydell , qemu-arm@nongnu.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, Pengutronix Kernel Team , "linux-arm-kernel@lists.infradead.org" , Enrico Joerns Subject: Re: [BUG] ARM64 KVM: Data abort executing post-indexed LDR on MMIO address In-Reply-To: <65ab10d7-6594-490c-be07-39f83ac3559a@pengutronix.de> References: <89f184d6-5b61-4c77-9f3b-c0a8f6a75d60@pengutronix.de> <4d559b9e-c208-46f3-851a-68086dc8a50f@pengutronix.de> <864j5q7sdq.wl-maz@kernel.org> <65ab10d7-6594-490c-be07-39f83ac3559a@pengutronix.de> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.4 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: a.fatoum@pengutronix.de, peter.maydell@linaro.org, qemu-arm@nongnu.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, kernel@pengutronix.de, linux-arm-kernel@lists.infradead.org, ejo@pengutronix.de X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241005_143554_645800_76544CAA X-CRM114-Status: GOOD ( 33.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Sat, 05 Oct 2024 19:38:23 +0100, Ahmad Fatoum wrote: > > Hello Marc, > > On 05.10.24 12:31, Marc Zyngier wrote: > > On Fri, 04 Oct 2024 20:50:18 +0100, > > Ahmad Fatoum wrote: > >> With readl/writel implemented in assembly, I get beyond that point, but > >> now I get a data abort running an DC IVAC instruction on address 0x1000, > >> where the cfi-flash is located. This instruction is part of a routine > >> to remap the cfi-flash to start a page later, so the zero page can be > >> mapped faulting. > > [snip] > > >> Any idea what this is about? > > > > IIRC, the QEMU flash is implemented as a read-only memslot. A data > > cache invalidation is a *write*, as it can be (and is) upgraded to a > > clean+invalidate by the HW. > > So it's a write, even if there are no dirty cache lines? Yes. At the point where this is handled, the CPU has no clue about the dirty state of an arbitrary cache line, at an arbitrary cache level. The CPU simply forward the CMO downstream, and the permission check happens way before that. > > KVM cannot satisfy the write, for obvious reasons, and tells the guest > > to bugger off (__gfn_to_pfn_memslot() returns KVM_PFN_ERR_RO_FAULT, > > which satisfies is_error_noslot_pfn() -- a slight oddity, but hey, why > > not). > > > > In the end, you get an exception. We could relax this by > > special-casing CMOs to RO memslots, but this doesn't look great. > > > > The real question is: what are you trying to achieve with this? > > barebox sets up the MMU, but tries to keep a 1:1 mapping. On Virt, we > want to map the zero page faulting, but still have access to the first > block of the cfi-flash. > > Therefore, barebox will map the cfi-flash one page later > (virt 0x1000,0x2000,... -> phys 0x0000,0x1000,...) and so on, so the first > page can be mapped faulting. > > The routine[1] that does this remapping invalidates the virtual address range, > because the attributes may change[2]. This invalidate also happens for cfi-flash, > but we should never encounter dirty cache lines there as the remap is done > before driver probe. > > Can you advise what should be done differently? If you always map the flash as Device memory, there is no need for CMOs. Same thing if you map it as NC. And even if you did map it as Cacheable, it wouldn't matter. KVM already handles coherency when the flash is switching between memory-mapped and programming mode, as the physical address space changes (the flash literally drops from the memory map). In general, issuing CMOs to a device is a bizarre concept, because it is pretty rare that a device can handle a full cache-line as write-back. Devices tend to handle smaller, register-sized accesses, not a full 64-byte eviction. Now, I'm still wondering whether we should simply forward the CMO to userspace as we do for other writes, and let the VMM deal with it. The main issue is that there is no current way to describe this. The alternative would be to silently handle the trap and pretend it never occurred, as we do for other bizarre behaviours. But that'd be something only new kernels would support, and I guess you'd like your guest to work today, not tomorrow. M. -- Without deviation from the norm, progress is not possible.