From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D5B14C00140 for ; Mon, 15 Aug 2022 23:09:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Date:Cc:To:From:Subject:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tWEbFEpK0ZbgAqTy1p8oIxK5jMbhc2B9YBhbPrz7yv0=; b=ovAn1AeOu1XPq0 Tcljgy/poQev7AsoK31F4xbkR//RqwSyQRWZyIEBc3hfQ6KXwBXKsfhfyyuEjp+oiNonFcwc19sI3 dOdWk47fJwlDuWLPa09BfIubJe78+LJ1JjJ5oE1qHDjDLTO5+ZM+UTDmFSZL0wxTBhn00H6UWoSWW z3jJS0juXfz8QePfR4smYxva4cVVH+jsrkOAvEG6ufIlnPIWBG9dlAlDFEYLY19ik+2OXEoeFOSMe oICTWvO0jkiKETSEy2O8h1tyUjNISKk8Ss1AZXuIGHvlJu0sDsY6O98pKEkrXsqBZ5Wr8md/eAD1B TmFzY65ZVjI3g9BYYD8A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oNjBP-0079Kv-Mg; Mon, 15 Aug 2022 23:07:44 +0000 Received: from gate.crashing.org ([63.228.1.57]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oNjBG-0079Dh-Js for linux-arm-kernel@lists.infradead.org; Mon, 15 Aug 2022 23:07:39 +0000 Received: from ip6-localhost (localhost.localdomain [127.0.0.1]) by gate.crashing.org (8.14.1/8.14.1) with ESMTP id 27FN28MK014575; Mon, 15 Aug 2022 18:02:09 -0500 Message-ID: <1c83e5b3894a7e5ddb0024e5aa6d29e9f872812d.camel@kernel.crashing.org> Subject: Re: arm64 PCI resource allocation issue From: Benjamin Herrenschmidt To: Lorenzo Pieralisi Cc: linux-pci@vger.kernel.org, bhelgaas@google.com, mark.rutland@arm.com, linux-arm-kernel@lists.infradead.org, Jeremy Linton , Ali Saidi , David Woodhouse Date: Tue, 16 Aug 2022 09:02:08 +1000 In-Reply-To: References: <204dda77248a7c95787e27fc7a382f514341c88e.camel@kernel.crashing.org> User-Agent: Evolution 3.36.5-0ubuntu1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220815_160735_147032_73AE416C X-CRM114-Status: GOOD ( 37.03 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Still hoping for opinions on this ... I'll probably have time to hack on it next week... On Fri, 2022-08-05 at 09:51 +1000, Benjamin Herrenschmidt wrote: > That leaves us with 3 overall routes I can think of (we can figure out > the details next): > > 1) We can try to detect early those devices (easy with SPCR, are > there more on aarch64 ? on x86 there is) and hammer them into place, > flagging them somewhat and forcing them (and all their parents) to keep their > resources. > > Pros: It's rather easy to implement, we can "register" the addresses > early and have the PCI probe code match detected devices againt that > list & flag them (for example IORESOURCE_PCI_FIXES :-) and their > parents. > > Cons: It will force entire bus hierarchies to be fixed, which might > not really help on firmwares that are known to setup sub-optimal > apertures (or even completely b0rked ones). But we don't know who those are > except maybe one or two if we dig down into the previous version of > that discussion from a couple of years ago. > > 2) We can try to "keep track" of them as they move. Variant A. > > We do it the way efifb does it and wrap that in something a bit nicer > as follow: > > - We add a helper to "record" a pci_dev/BAR#/offset combination and > an > other one to do the lookup & fixup of a FW originated address. > > - We make efifb quirk use that instead of its existing global > "bar_resource". > > - We add a similar quirk to the ACPI code that parses SPCR and > (maybe) another one for earlycon (hint they may be the same device, some > deduplication would be useful). > > - We update 8250_pci (I assume pl01x are never PCI ?) to call this > to "fixup" addresses obtained from earlycon. That's the easy bit. SPRC > is trickier, we'd need to fixup addressed parsed from > add_preferred_console() .. I'm not 100% sure there's a case where > such an address would be added post-PCI-remap and we might incorrectly fix > it up. I don't think so but ... > > Pros: It should (hopefully) not be overly complicated and reasonably > self contained, low risk. > > Cons: > > - It's a bit more complicated than other solutions, though not > insanely > > - This doesn't solve the problem of a driver such as earlycon being > "live" accross the remapping (and thus means we'll probably still > have verbose PCI probing with earlycon dying horribly). This is already > partially broken since we temporarily disable decoding during probing > but that's a small window ... We can look at solving that separately > by adding on top of this registration mechanism: We *could* optionally > register in our above helper a pair of callbacks that the PCI code > would call for each registered "early device" before and after > remapping to "suspend access" and "fixup address". Those would be > ideally called around the remapping of the entire host bridge the > device is on. > > 3) Keeping track, Variant B > > (note: the more I think about it, the more I prefer variant A but > let's see what others think) > > We generalize pcibios_save_fw_addr() and for the sake of it, we move > that into pci_dev which simplifies everything and gets rid of that > separate list. > > Then, things like efifb, 8250_pci etc... do a lookup in there for > addresses they obtain from screen_info, earlycon, > add_preferred_console.. and on match, perform the necessary fixup. > Assuming we are confident those addresses originate from before the > PCI remapping that is. > > Pros: It *seems* even simpler than the above other options and maybe > even faster. > > Cons: It's more resource intensive as we now backup original BARs for > everything under the sun. It also doesn't provide a great path to > address the case I mentioned earlier for dealing with "live" devices. > > That's all I came up with ... Any better ideas and any preferences ? > At this point I'm reasonably keen on (2) (tracking variant A). _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel