From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 06589CDB474 for ; Fri, 20 Oct 2023 14:04:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=fLj3WHPNcTC3fqwwIuW+zquxL9LV6FjCggIE7Zoud/c=; b=jYRP9ypg82kxlP YVgZMnDqb64cNvXuBXF/3wvkh3j8Pv1RSdzQaUGMshQYbdm3WO3CdIazfFpbLzXn/pgOqSDTF4Lrb q1l0AsaHlckn0Q39Oa0FUjBiowVI2x3Eh/vdmdfzS819utI96SwUZDvbVX22jv+1fZWIjjXm52Crt 2KQc3ugETLLwWuMOkL6QdqRq5b1PjPKBO2dMAnr8NmM5EIslMdK6HD3lOtlOqpfFjszpcyuYViD65 BQMSj9XEuXm+/6pGPQ5SRXw4gtjbu6OR5OBebIxOGGraKCDFqadlrMl4/rd6hsqMflIYamGZSehmv 3ooMgd8x7/911G06aSYQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qtq6o-002S5N-0S; Fri, 20 Oct 2023 14:04:14 +0000 Received: from sin.source.kernel.org ([145.40.73.55]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qtq6k-002S4L-1L for linux-arm-kernel@lists.infradead.org; Fri, 20 Oct 2023 14:04:12 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id CCEABCE38C9; Fri, 20 Oct 2023 14:04:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B3145C433C7; Fri, 20 Oct 2023 14:04:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697810645; bh=jGn4CadJUGcXIko3cgZzK66leI6kgRcHdeUCJtNcI0A=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=UYFEkk/IKlj8DgLmPwUeClMRfq90ihEuyx4yBSkcedVTNV2yJxDvd+OsLv3APhEwV gbNf84Kh/fml3EUS9EBW41n5YkPP7+9sq0NW5kBqJUGfp4Yg4M5LW/Hy4CcBYi32AW yJvS4FgEl8L4weSx1t9K7jvndBgl7RaFAUK2LdqRAv4NMJkxc6Bp2XsoqBMtnfZ7UD /uP63cRltfuqbX1WVXlvpPN3I0I4Uw4wEIgM/bUIP1Rv0EYKBktdwRj7z6TgO4WnKd 5zaSAg2zy3xB4fcmHpccuc8ckj4Nw9bfiUzZRZ2QEYoXuNSm9U/jU8L9Ljucb3dDOE t9rUyTLVj36HA== Date: Fri, 20 Oct 2023 16:03:57 +0200 From: Lorenzo Pieralisi To: Jason Gunthorpe Cc: Catalin Marinas , Will Deacon , ankita@nvidia.com, maz@kernel.org, oliver.upton@linux.dev, aniketa@nvidia.com, cjia@nvidia.com, kwankhede@nvidia.com, targupta@nvidia.com, vsethi@nvidia.com, acurrid@nvidia.com, apopple@nvidia.com, jhubbard@nvidia.com, danw@nvidia.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org Subject: Re: [PATCH v1 2/2] KVM: arm64: allow the VM to select DEVICE_* and NORMAL_NC for IO memory Message-ID: References: <20231012144807.GA12374@willie-the-truck> <20231013092934.GA13524@willie-the-truck> <20231013134541.GP3952@nvidia.com> <20231019115142.GQ3952@nvidia.com> <20231020114719.GE3952@nvidia.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20231020114719.GE3952@nvidia.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231020_070410_800873_D850C6A0 X-CRM114-Status: GOOD ( 36.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Oct 20, 2023 at 08:47:19AM -0300, Jason Gunthorpe wrote: > On Fri, Oct 20, 2023 at 12:21:50PM +0100, Catalin Marinas wrote: > > On Thu, Oct 19, 2023 at 08:51:42AM -0300, Jason Gunthorpe wrote: > > > On Thu, Oct 19, 2023 at 12:07:42PM +0100, Catalin Marinas wrote: > > > > Talking to Will earlier, I think we can deem the PCIe scenario > > > > (somewhat) safe but not as a generic mechanism for other non-PCIe > > > > devices (e.g. platform). With this concern, can we make this Stage 2 > > > > relaxation in KVM only for vfio-pci mappings? I don't have an example of > > > > non-PCIe device assignment to figure out how this should work though. > > > > > > It is not a KVM problem. As I implied above it is VFIO's > > > responsibility to reliably reset the device, not KVMs. If for some > > > reason VFIO cannot do that on some SOC then VFIO devices should not > > > exist. > > > > > > It is not KVM's job to double guess VFIO's own security properties. > > > > I'd argue that since KVM is the one relaxing the memory attributes > > beyond what the VFIO driver allows the VMM to use, it is KVM's job to > > consider the security implications. This is fine for vfio-pci and > > Normal_NC but I'm not sure we can generalise. > > I can see that, but I belive we should take this responsibility into > VFIO as a requirement. As I said in the other email we do want to > extend VFIO to support NormalNC VMAs for DPDK, so we need to take this > anyhow. > > > > Specifically about platform the generic VFIO platform driver is the > > > ACPI based one. If the FW provides an ACPI method for device reset > > > that is not properly serializing, that is a FW bug. We can quirk it in > > > VFIO and block using those devices if they actually exist. > > > > > > I expect the non-generic VFIO platform drivers to take care of this > > > issue internally with, barriers, read from devices, whatver is > > > required to make their SOCs order properly. Just as I would expect a > > > normal Linux platform driver to directly manage whatever > > > implementation specific ordering quirks the SOC may have. > > > > This would be a new requirement if an existing VFIO platform driver > > relied on all mappings being Device. But maybe that's just theoretical > > at the moment, are there any concrete examples outside vfio-pci? If not, > > we can document it as per Lorenzo's suggestion to summarise this > > discussion under Documentation/. > > My point is if this becomes a real world concern we have a solid > answer on how to resolve it - fix the VFIO driver to have a stronger > barrier before reset. Just to make sure I am parsing this correctly: this case above is related to a non-PCI VFIO device passthrough where a guest would want to map the device MMIO at stage-1 with normal-NC memory type (well, let's say with a memory attribute != device-nGnRE - that combined with the new stage-2 default might cause transactions ordering/grouping trouble with eg device resets), correct ? IIRC, all requests related to honouring "write-combine" style stage-1 mappings were for PCI(e) devices but that's as far as what *I* was made aware of goes. > I'm confident it is not a problem for PCI and IIRC the remaining ARM > platform drivers were made primarily for DPDK, not KVM. > > So I agree with documenting and perhaps a comment someplace in VFIO is > also warranted. We will do that, I will start adding the recent discussions to the new documentation file. Side note: for those who attend LPC it would be useful to review the resulting documentation together there, it should happen around v6.7-rc1. Lorenzo _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel