From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A153FC4332F for ; Mon, 13 Nov 2023 00:43:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vc1aYzS9AtsQl+ZguTQ59FPy+gu6FLJ4lB1hZJzKLfE=; b=HA7whXubld7uSq 0HHu6Hc/ZBZMY+sEnCgEOal13WEdhXA/F6N/cPjidQAaj8TzQyPjXzsj1rCJGcO9MgHuTDAo1/P5v R3lcKCW5HOL/M5fBsgMCsHUE+gDr9AdRXj1QkegGrknxhQOHVOSq9ZofWl0eTTpn4Or1kiHbE3Dze KA3vW0c7gR/RyL3iIGD7EZ2Z4h9vNxzTJ0GDGJ5Z+VldWIN+vLvxOXeOrYBgRZF7OjecdnHw5B7Fl StFPCjTwdJaYkbuegychFym1ndalwn7VX35LDJXNNFuMgXwuDhy2xcjrTm6gmBBWKUbfRbt4qKofS o/3O68wP2HXnmAn5Dyhg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r2L2s-00Cw75-1q; Mon, 13 Nov 2023 00:43:18 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1r2L2p-00Cw5l-0B for linux-arm-kernel@lists.infradead.org; Mon, 13 Nov 2023 00:43:16 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id 38426B80B25; Mon, 13 Nov 2023 00:43:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E3FCBC433C7; Mon, 13 Nov 2023 00:43:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1699836192; bh=qo73hMpbT5UB0fyH1NpcXGG0dpmwEMtmdXWgdp/vY8A=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=I4o+wfYUYjOpLGSebbsZMJ0/0DLrfENueQdXl0m0SbRp8hElJyTaF7bkd/mzmy3JX d1BdMxZvYnO7u3xUKbJjVWKZxWhmosAmtau7cn4H45FVS5nvhiDlhpj1RhJQ/xJtjW HEOOcIcZT9O22E8Vfp42G3uBF/wvHKx8hmd4/OXLPyLrAY16AXQhncK8BSrohVHQ7i bIgwxruX7vJLSi41Z7ekqS3nDmuokZ8Y2RS3/ODv3Rqj6S3jTlYt8Vj8EALz7vuUiH Bg2fl7HWwtDMFvJoFMjWILWyjjShVRlAIHwbPkZFCLNFbvwV6iPCG2NU/Sdrcg0pm/ lzbwpoLJUpkOA== Date: Mon, 13 Nov 2023 01:42:58 +0100 From: Lorenzo Pieralisi To: Jason Gunthorpe Cc: Catalin Marinas , Will Deacon , ankita@nvidia.com, maz@kernel.org, oliver.upton@linux.dev, aniketa@nvidia.com, cjia@nvidia.com, kwankhede@nvidia.com, targupta@nvidia.com, vsethi@nvidia.com, acurrid@nvidia.com, apopple@nvidia.com, jhubbard@nvidia.com, danw@nvidia.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Subject: Re: [PATCH v1 2/2] KVM: arm64: allow the VM to select DEVICE_* and NORMAL_NC for IO memory Message-ID: References: <20231012123541.GB11824@willie-the-truck> <20231012144807.GA12374@willie-the-truck> <20231013092934.GA13524@willie-the-truck> <20231110142649.GO4488@nvidia.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20231110142649.GO4488@nvidia.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231112_164315_353384_D36F0CE8 X-CRM114-Status: GOOD ( 23.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Nov 10, 2023 at 10:26:49AM -0400, Jason Gunthorpe wrote: > On Thu, Nov 09, 2023 at 04:34:10PM +0100, Lorenzo Pieralisi wrote: > > > Relaxing S2 KVM device MMIO mappings to Normal-NC is not expected to > > trigger any issue on guest device reclaim use cases either (ie device > > MMIO unmap followed by a device reset) at least for PCIe devices, in that > > in PCIe a device reset is architected and carried out through PCI config > > space transactions that are naturally ordered wrt MMIO transactions > > according to the PCI ordering rules. > > This is not how I see that thread concluding.. > > The device reclaim problem belongs solely to VFIO, not KVM. VFIO must > ensure global ordering of access before the VMA is unmaped and access > after, that includes ordering whatever mechanism the VFIO driver uses > for reset. > > If there are quirky SOCs, or non-PCI devices that need something > stronger than the TLBI/etc sequence it should be fixed in VFIO (or > maybe even the arch code), not by blocking NORMAL_NC in the KVM. Such > a quirky SOC would broadly have security issues beyond KVM. https://lore.kernel.org/linux-arm-kernel/20231013092934.GA13524@willie-the-truck I think that Will's point _was_ related to the change we are making for KVM S2 mappings and related device transactions on device reclaim - ie reset, that's what I tried to convey (I probably simplified too much) that for PCI at least that should not trigger any regression/issue, in that BAR MMIO and reset transactions are decoupled streams and must follow the PCI ordering rules. Yes, it is VFIO responsibility but changing the S2 KVM mappings may have (for non-PCI devices) side effects compared to what we have today, I am not saying this should be a blocker I just summarized the thread above, the paragraph can be expanded. Lorenzo > > > It is worth noting that currently, to map devices MMIO space to user > > space in a device pass-through use case the VFIO framework applies memory > > attributes derived from pgprot_noncached() settings applied to VMAs, which > > Sometimes. VFIO uses a mix of pgprot_noncached and pgprot_device. AFAIK > we should change to to always use pgprot_device.. > > Thanks, > Jason _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel