From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0988C433F5 for ; Fri, 17 Sep 2021 09:31:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 98ADE60F93 for ; Fri, 17 Sep 2021 09:31:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240126AbhIQJcg (ORCPT ); Fri, 17 Sep 2021 05:32:36 -0400 Received: from sibelius.xs4all.nl ([83.163.83.176]:57329 "EHLO sibelius.xs4all.nl" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239040AbhIQJcf (ORCPT ); Fri, 17 Sep 2021 05:32:35 -0400 Received: from localhost (bloch.sibelius.xs4all.nl [local]) by bloch.sibelius.xs4all.nl (OpenSMTPD) with ESMTPA id 4f816570; Fri, 17 Sep 2021 11:31:10 +0200 (CEST) Date: Fri, 17 Sep 2021 11:31:10 +0200 (CEST) From: Mark Kettenis To: Marc Zyngier Cc: sven@svenpeter.dev, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, bhelgaas@google.com, robh+dt@kernel.org, lorenzo.pieralisi@arm.com, kw@linux.com, alyssa@rosenzweig.io, stan@corellium.com, kettenis@openbsd.org, marcan@marcan.st, Robin.Murphy@arm.com, kernel-team@android.com In-Reply-To: <87mtobblvd.wl-maz@kernel.org> (message from Marc Zyngier on Fri, 17 Sep 2021 10:19:02 +0100) Subject: Re: [PATCH v3 10/10] PCI: apple: Configure RID to SID mapper on device addition References: <20210913182550.264165-1-maz@kernel.org> <20210913182550.264165-11-maz@kernel.org> <87y27zbiu3.wl-maz@kernel.org> <56145a72aa978ebd@bloch.sibelius.xs4all.nl> <87mtobblvd.wl-maz@kernel.org> Message-ID: <56146608f4547f39@bloch.sibelius.xs4all.nl> Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org > Date: Fri, 17 Sep 2021 10:19:02 +0100 > From: Marc Zyngier > > On Tue, 14 Sep 2021 10:56:05 +0100, > Mark Kettenis wrote: > > > > > Date: Tue, 14 Sep 2021 10:35:32 +0100 > > > From: Marc Zyngier > > > > > > On Mon, 13 Sep 2021 21:45:13 +0100, > > > "Sven Peter" wrote: > > > > > > > > On Mon, Sep 13, 2021, at 20:25, Marc Zyngier wrote: > > > > > The Apple PCIe controller doesn't directly feed the endpoint's > > > > > Requester ID to the IOMMU (DART), but instead maps RIDs onto > > > > > Stream IDs (SIDs). The DART and the PCIe controller must thus > > > > > agree on the SIDs that are used for translation (by using > > > > > the 'iommu-map' property). > > > > > > > > > > For this purpose, parse the 'iommu-map' property each time a > > > > > device gets added, and use the resulting translation to configure > > > > > the PCIe RID-to-SID mapper. Similarily, remove the translation > > > > > if/when the device gets removed. > > > > > > > > > > This is all driven from a bus notifier which gets registered at > > > > > probe time. Hopefully this is the only PCI controller driver > > > > > in the whole system. > > > > > > > > > > Signed-off-by: Marc Zyngier > > > > > --- > > > > > drivers/pci/controller/pcie-apple.c | 158 +++++++++++++++++++++++++++- > > > > > 1 file changed, 156 insertions(+), 2 deletions(-) > > > > > > > > > > diff --git a/drivers/pci/controller/pcie-apple.c > > > > > b/drivers/pci/controller/pcie-apple.c > > > > > index 76344223245d..68d71eabe708 100644 > > > > > --- a/drivers/pci/controller/pcie-apple.c > > > > > +++ b/drivers/pci/controller/pcie-apple.c > > > > > @@ -23,8 +23,10 @@ > > > > > #include > > > > > #include > > > > > #include > > > > > +#include > > > > > #include > > > > > #include > > > > > +#include > > > > > #include > > > > > #include > > > > > > > > > > @@ -116,6 +118,8 @@ > > > > > #define PORT_TUNSTAT_PERST_ACK_PEND BIT(1) > > > > > #define PORT_PREFMEM_ENABLE 0x00994 > > > > > > > > > > +#define MAX_RID2SID 64 > > > > > > > > Do these actually have 64 slots? I thought that was only for > > > > the Thunderbolt controllers and that these only had 16. > > > > > > You are indeed right, and I blindly used the limit used in the > > > Correlium driver. Using entries from 16 onward result in a non booting > > > system. The registers do not fault though, and simply ignore writes. I > > > came up with an simple fix for this, see below. > > > > Or should be add a property to the DT binding to indicate the number > > of entries (using a default of 16)? We don't have to add that > > property right away; we can delay that until we actually try to > > support the Thunderbolt ports. > > I'd rather only add a property for things we cannot discover > ourselves. And indeed, we don't have to decide on this right now. > > > In case you didn't know already, RIDs that have no mapping in the > > RID2SID table map to SID 0. That's why I picked 1 as the SID in the > > iommu-map property for the port. > > I sort-off guessed, as using 0 made everything work by 'magic', while > using your DT prevented the machine from booting. I tend to dislike > magic, hence this patch. Right. I deliberately used SID 1 in the DT to make sure other devices on the bus couldn't accidentally use IOMMU mappings for a device that was mapped to SID 0. > > > > I never checked it myself though and it doesn't make much > > > > of a difference for now since only four different RIDs will > > > > ever be connected anyway. > > > > > > Four? I guess the radios expose more than a single RID? > > > > At this point, on the M1 mini there is the Broadcom BCM4378 WiFi/BT > > device (which has two functions), the Fresco Logic FL1100 xHCI > > controller (single function) and the Broadcom BCM57765 Ethernet > > controller. So yes, there are for RIDs. > > But as far as I can see, the RID-to-SID mapping is per port. So at > most, we have two RIDs per port/DART, not four. Or am I missing > something altogether? No you're not missing anything.