From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A2AEFC25B75 for ; Mon, 3 Jun 2024 17:19:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:References: List-Owner; bh=FxZYA26d1qjczA6doYPlk9KrEZj5ZfPc+6HW2iQkxPM=; b=izIGevsnkXTWA2 ZTxKGMoomJ+NYLHsUV8Ctf4s7//hEWNFbNsbW7J2uK79MhKGdB6xWu+gnCOm3VCOfkcdkZPpXf46A ei1YvgLNKysQO24DPqiCuocKq0Nj2ITfuv4iwgwfSK7H5sM0sKI32ngkVIiBb0SsPFR5Qr22m0cO0 joEKESKnJuydzFho76G4nMNFQhmlXbTrgUOwYQE3t1wMK+7DtDBr1XUyxPoapp5++tJvVp8u4Ha6m GgPinZeaLH7K9ZLYGovhg1FrCPpMSppvSRaIMuRX/73xoYx7+5nm251IbsNG9izXdsH3jiMuGZHhg 9pnPsIsWuJpIH51qDKxA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEBLF-0000000HUG3-2XX2; Mon, 03 Jun 2024 17:19:29 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEBLC-0000000HUEr-0UUK for linux-arm-kernel@lists.infradead.org; Mon, 03 Jun 2024 17:19:27 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 1021960EB9; Mon, 3 Jun 2024 17:19:24 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6583AC2BD10; Mon, 3 Jun 2024 17:19:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1717435163; bh=o+oMkH1I/WbYDZBa3qcXo8pOhVZODTPPaT5jeLn7qIU=; h=Date:From:To:Cc:Subject:In-Reply-To:From; b=hhwFWKSSd4vV2hTYoNib68G4aE53A9hjm/qJTppKcaTVXqZ3K4F8FnfKdI32g6R3a luQD8Z1RYHtLwXs/M4NQm1fJAZTRopSgdO0Qd1pfmpGSIjWtjcPMnzrzxaKgxM9H33 CxTXWuL1hlBkHGtPKRVAcKxUyGElF8oR2vHTLhHFWTJ20+qfZJwH52g8tNJs1qQkOX d56VBM/qpQ/5gtIL6YY7XK+4+BS34uQx8Rw3B2QRMGeObdl9V/+S8TC/qp9mJY6MtV CdUAFpYDItJ8i6wpM/BKPyNsSLCDzPpJ38jiex4vFTOBdxidxBBuNCg4W4RS3deaJt NFjppavpkC5Jg== Date: Mon, 3 Jun 2024 12:19:21 -0500 From: Bjorn Helgaas To: Robin Murphy Cc: Frank Li , Richard Zhu , Lucas Stach , Lorenzo Pieralisi , Krzysztof =?utf-8?Q?Wilczy=C5=84ski?= , Rob Herring , Bjorn Helgaas , Shawn Guo , Sascha Hauer , Pengutronix Kernel Team , Fabio Estevam , NXP Linux Team , Philipp Zabel , Liam Girdwood , Mark Brown , Manivannan Sadhasivam , Krzysztof Kozlowski , Conor Dooley , linux-pci@vger.kernel.org, imx@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, devicetree@vger.kernel.org, Will Deacon , Joerg Roedel , Jason Gunthorpe , Alyssa Rosenzweig , Marc Zyngier Subject: Re: [PATCH v5 08/12] PCI: imx6: Config look up table(LUT) to support MSI ITS and IOMMU for i.MX95 Message-ID: <20240603171921.GA685838@bhelgaas> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <974f1d23-aba8-432e-85b5-0e4b1c2005e7@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240603_101926_313927_1D28E413 X-CRM114-Status: GOOD ( 36.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, May 31, 2024 at 03:58:49PM +0100, Robin Murphy wrote: > On 2024-05-31 12:08 am, Bjorn Helgaas wrote: > > [+cc IOMMU and pcie-apple.c folks for comment] > > > > On Tue, May 28, 2024 at 03:39:21PM -0400, Frank Li wrote: > > > For the i.MX95, configuration of a LUT is necessary to convert Bus Device > > > Function (BDF) to stream IDs, which are utilized by both IOMMU and ITS. > > > This involves examining the msi-map and smmu-map to ensure consistent > > > mapping of PCI BDF to the same stream IDs. Subsequently, LUT-related > > > registers are configured. In the absence of an msi-map, the built-in MSI > > > controller is utilized as a fallback. > > > > > > Additionally, register a PCI bus notifier to trigger imx_pcie_add_device() > > > upon the appearance of a new PCI device and when the bus is an iMX6 PCI > > > controller. This function configures the correct LUT based on Device Tree > > > Settings (DTS). > > > > This scheme is pretty similar to apple_pcie_bus_notifier(). If we > > have to do this, I wish it were *more* similar, i.e., copy the > > function names, bitmap tracking, code structure, etc. > > > > I don't really know how stream IDs work, but I assume they are used on > > most or all arm64 platforms, so I'm a little surprised that of all the > > PCI host drivers used on arm64, only pcie-apple.c and pci-imx6.c need > > this notifier. > > This is one of those things that's mostly at the mercy of the PCIe root > complex implementation. Typically the SMMU StreamID and/or GIC ITS DeviceID > is derived directly from the PCI RID, sometimes with additional high-order > bits hard-wired to disambiguate PCI segments. I believe this RID-translation > LUT is a particular feature of the the Synopsys IP - I know there's also one > on the NXP Layerscape platforms, but on those it's programmed by the > bootloader, which also generates the appropriate "msi-map" and "iommu-map" > properties to match. Ideally that's what i.MX should do as well, but hey. Maybe this RID-translation is a feature of i.MX, not of Synopsys? I see that the LUT CSR accesses use IMX95_* definitions. > If it's really necessary to do this programming from Linux, then there's > still no point in it being dynamic - the mappings cannot ever change, since > the rest of the kernel believes that what the DT said at boot time was > already a property of the hardware. It would be a lot more logical, and > likely simpler, for the driver to just read the relevant map property and > program the entire LUT to match, all in one go at controller probe time. > Rather like what's already commonly done with the parsing of "dma-ranges" to > program address-translation LUTs for inbound windows. > > Plus that would also give a chance of safely dealing with bad DTs specifying > invalid ID mappings (by refusing to probe at all). As it is, returning an > error from a child's BUS_NOTIFY_ADD_DEVICE does nothing except prevent any > further notifiers from running at that point - the device will still be > added, allowed to bind a driver, and able to start sending DMA/MSI traffic > without the controller being correctly programmed, which at best won't work > and at worst may break the whole system. Frank, could the imx LUT be programmed once at boot-time instead of at device-add time? I'm guessing maybe not because apparently there is a risk of running out of LUT entries? It sounds like the consequences of running out of LUT entries are catastrophic, e.g., memory corruption from mis-directed DMA? If that's possible, I think we need to figure out how to prevent the device from being used, not just dev_warn() about it. Bjorn _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel