From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1EAB2C3DA4A for ; Thu, 5 Sep 2024 16:34:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To: Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID:Subject:Cc:To: From:Date:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:References:List-Owner; bh=lxj9YoxQYLWPMt12tzozpOEGa6EXYUFEMxBetpIm3OU=; b=KjcI5OtvO3GTuXVSyJA703wg+Y zVSd4ia/Eo6Rj5MEx6Y4f9CkuW7/g1bFkBO6aPSNuFmYNP/xi+iwhg5thTiUMH2bLtQkn7Vy4Bjea qEILQB0Ut1LOQQy2PC55jt48xZEhGHFIa399Z0PPoqrMk1Srzyk+wKUitxAYV8X9iSRTw6B7n4Qak j5S2RwYu1eGaTCWlSDxoBcSknV1/ZUQTxgRCqcais+wEq3QvDqmzB4cJwIhQv1JPvFzsoe0xGX4q8 2v3opwqSBYM1e+rHbhhgUgXL3UH3g0Te5R5TZPGgbqZe8/feWoXlrOp4aFLvSahitLGD0HSb3LwdY eBYJ3Xnw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1smFRO-000000099BT-0B0O; Thu, 05 Sep 2024 16:34:38 +0000 Received: from nyc.source.kernel.org ([147.75.193.91]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1smFQM-000000098qx-3D63 for linux-arm-kernel@lists.infradead.org; Thu, 05 Sep 2024 16:33:36 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id F3997A44621; Thu, 5 Sep 2024 16:33:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A2B71C4CEC3; Thu, 5 Sep 2024 16:33:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725554012; bh=9Tk0qytElXOqijUu2mwaQ/iNQpnr3HTIkzbW8O/Fgio=; h=Date:From:To:Cc:Subject:In-Reply-To:From; b=gW+wBkXHJDhGVXXAfJCPQj2kpceVBR+QWpc5EB5VaM3VbU5KzSYjuyS+kU0KHYUgr Vbw7YD74hyIp9ASlVYw9i/S5T5hvkcMNS9g/sUfc0qUDPsWVcUe83qD/tkw+3Zopkl gakIRnJo3HMLUH5yZ4lTGGxnc6tViQlacP8tMH8RsMzC0ubDQA5nzKrqqDH95p5brW cxYC6wohcOlvlYD7LAPD+Eg0h5S+xtyw++MGlyUy5KeL/zsq62UTBqy+mp8R6fB01I piX0p5So+agRaK10NXmxRvxmrVanNSNsu9iQFOQ7iTY653UNcqqm8TUNkQVWBoRWym eScmz75uH8lrg== Date: Thu, 5 Sep 2024 11:33:29 -0500 From: Bjorn Helgaas To: Jan Kiszka Cc: Nishanth Menon , Santosh Shilimkar , Vignesh Raghavendra , Tero Kristo , Rob Herring , Krzysztof Kozlowski , Conor Dooley , devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, Siddharth Vadapalli , Bao Cheng Su , Hua Qian Li , Diogo Ivo , Lorenzo Pieralisi , Krzysztof =?utf-8?Q?Wilczy=C5=84ski?= , Bjorn Helgaas , Kishon Vijay Abraham I Subject: Re: [PATCH v4 4/7] PCI: keystone: Add supported for PVU-based DMA isolation on AM654 Message-ID: <20240905163329.GA389144@bhelgaas> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <361441d35d781b3c474b05921634bcae08d1a7b4.1725444016.git.jan.kiszka@siemens.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240905_093335_060405_EF4BB98E X-CRM114-Status: GOOD ( 37.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org [+cc Kishon, just in case you have time/interest ;)] On Wed, Sep 04, 2024 at 12:00:13PM +0200, Jan Kiszka wrote: > From: Jan Kiszka > > The AM654 lacks an IOMMU, thus does not support isolating DMA requests > from untrusted PCI devices to selected memory regions this way. Use > static PVU-based protection instead. > > For this, we use the availability of restricted-dma-pool memory regions > as trigger and register those as valid DMA targets with the PVU. I guess the implication is that DMA *outside* the restricted-dma-pool just gets dropped, and the Requester would see Completion Timeouts or something for reads? > In > addition, we need to enable the mapping of requester IDs to VirtIDs in > the PCI RC. We only use a single VirtID so far, catching all devices. > This may be extended later on. > > Signed-off-by: Jan Kiszka > --- > CC: Lorenzo Pieralisi > CC: "Krzysztof Wilczyński" > CC: Bjorn Helgaas > CC: linux-pci@vger.kernel.org Regrettably we don't really have anybody taking care of pci-keystone.c (at least per MAINTAINERS). > --- > drivers/pci/controller/dwc/pci-keystone.c | 101 ++++++++++++++++++++++ > 1 file changed, 101 insertions(+) > > diff --git a/drivers/pci/controller/dwc/pci-keystone.c b/drivers/pci/controller/dwc/pci-keystone.c > index 2219b1a866fa..96b871656da4 100644 > --- a/drivers/pci/controller/dwc/pci-keystone.c > +++ b/drivers/pci/controller/dwc/pci-keystone.c > @@ -19,6 +19,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -26,6 +27,7 @@ > #include > #include > #include > +#include > > #include "../../pci.h" > #include "pcie-designware.h" > @@ -111,6 +113,16 @@ > > #define PCI_DEVICE_ID_TI_AM654X 0xb00c > > +#define KS_PCI_VIRTID 0 > + > +#define PCIE_VMAP_xP_CTRL 0x0 > +#define PCIE_VMAP_xP_REQID 0x4 > +#define PCIE_VMAP_xP_VIRTID 0x8 > + > +#define PCIE_VMAP_xP_CTRL_EN BIT(0) > + > +#define PCIE_VMAP_xP_VIRTID_VID_MASK 0xfff > + > struct ks_pcie_of_data { > enum dw_pcie_device_mode mode; > const struct dw_pcie_host_ops *host_ops; > @@ -1125,6 +1137,89 @@ static const struct of_device_id ks_pcie_of_match[] = { > { }, > }; > > +#ifdef CONFIG_TI_PVU > +static const char *ks_vmap_res[] = {"vmap_lp", "vmap_hp"}; > + > +static int ks_init_restricted_dma(struct platform_device *pdev) > +{ > + struct device *dev = &pdev->dev; > + struct of_phandle_iterator it; > + bool init_vmap = false; > + struct resource phys; > + struct resource *res; > + void __iomem *base; > + unsigned int n; > + u32 val; > + int err; > + > + of_for_each_phandle(&it, err, dev->of_node, "memory-region", > + NULL, 0) { > + if (!of_device_is_compatible(it.node, "restricted-dma-pool")) > + continue; > + > + err = of_address_to_resource(it.node, 0, &phys); > + if (err < 0) { > + dev_err(dev, "failed to parse memory region %pOF: %d\n", > + it.node, err); > + continue; > + } > + > + err = ti_pvu_create_region(KS_PCI_VIRTID, &phys); > + if (err < 0) > + return err; > + > + init_vmap = true; > + } if (!init_vmap) return 0; would unindent the following. > + > + if (init_vmap) { > + for (n = 0; n < ARRAY_SIZE(ks_vmap_res); n++) { Since the only use of ks_vmap_res is here, this might be more readable if there were a helper that would be called twice with the constant strings, e.g., helper(pdev, "vmap_lp"); helper(pdev, "vmap_hp"); > + res = platform_get_resource_byname(pdev, IORESOURCE_MEM, > + ks_vmap_res[n]); Seems like we should check "res" for error before using it? > + base = devm_pci_remap_cfg_resource(dev, res); > + if (IS_ERR(base)) > + return PTR_ERR(base); > + > + writel(0, base + PCIE_VMAP_xP_REQID); > + > + val = readl(base + PCIE_VMAP_xP_VIRTID); > + val &= ~PCIE_VMAP_xP_VIRTID_VID_MASK; > + val |= KS_PCI_VIRTID; > + writel(val, base + PCIE_VMAP_xP_VIRTID); > + > + val = readl(base + PCIE_VMAP_xP_CTRL); > + val |= PCIE_VMAP_xP_CTRL_EN; > + writel(val, base + PCIE_VMAP_xP_CTRL); Since there's no explicit use of "restricted-dma-pool" elsewhere in this patch, I assume the setup above causes the controller to drop any DMA accesses outside that pool? I think a comment about how the controller behavior is being changed would be useful. Basically the same comment as for the commit log. Would there be any value in a dmesg note about a restriction being enforced? Seems like it's dependent on both CONFIG_TI_PVU and some DT properties, and since those are invisible in the log, maybe a note would help understand/debug any issues? > + } > + } > + > + return 0; > +} > + > +static void ks_release_restricted_dma(struct platform_device *pdev) > +{ > + struct of_phandle_iterator it; > + struct resource phys; > + int err; > + > + of_for_each_phandle(&it, err, pdev->dev.of_node, "memory-region", > + NULL, 0) { > + if (of_device_is_compatible(it.node, "restricted-dma-pool") && > + of_address_to_resource(it.node, 0, &phys) == 0) > + ti_pvu_remove_region(KS_PCI_VIRTID, &phys); I guess it's not important to undo the PCIE_VMAP_xP_CTRL_EN and related setup that was done by ks_init_restricted_dma()? > + } > +} > +#else > +static inline int ks_init_restricted_dma(struct platform_device *pdev) > +{ > + return 0; > +} > + > +static inline void ks_release_restricted_dma(struct platform_device *pdev) > +{ > +} > +#endif > + > static int ks_pcie_probe(struct platform_device *pdev) > { > const struct dw_pcie_host_ops *host_ops; > @@ -1273,6 +1368,10 @@ static int ks_pcie_probe(struct platform_device *pdev) > if (ret < 0) > goto err_get_sync; > > + ret = ks_init_restricted_dma(pdev); > + if (ret < 0) > + goto err_get_sync; > + > switch (mode) { > case DW_PCIE_RC_TYPE: > if (!IS_ENABLED(CONFIG_PCI_KEYSTONE_HOST)) { > @@ -1354,6 +1453,8 @@ static void ks_pcie_remove(struct platform_device *pdev) > int num_lanes = ks_pcie->num_lanes; > struct device *dev = &pdev->dev; > > + ks_release_restricted_dma(pdev); > + > pm_runtime_put(dev); > pm_runtime_disable(dev); > ks_pcie_disable_phy(ks_pcie); > -- > 2.43.0 >