From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93D10C19F2D for ; Sat, 6 Aug 2022 12:17:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231210AbiHFMRi (ORCPT ); Sat, 6 Aug 2022 08:17:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38458 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230525AbiHFMQW (ORCPT ); Sat, 6 Aug 2022 08:16:22 -0400 Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7914525F9 for ; Sat, 6 Aug 2022 05:16:20 -0700 (PDT) Received: by mail-pl1-x629.google.com with SMTP id w14so4747259plp.9 for ; Sat, 06 Aug 2022 05:16:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=fvZ1gSD4IexYnMpshzD56bFQ2RImKC8UwmCBJy/56WI=; b=JdmiFTeBVJCp4FJzeVo4WD/8bS5P8AvjqoODOw5nuPV3ysbPemfSkpamomnqnwFdiS Jr6Y445TMWmEB7k2F8dY9gfG6wuL7XCOrw3xsLhKThzzQ8nWIuU/R+x0IgdS5tKlz9Xt R1Aid0OMlNEGiT+miGiu7v1Dv8brfDAqoxyNYLLGR/4tGE2EMjCES5Qm9AZnP4ixAcPn nP/YgoMmzK3lsvbJTxvdSato2rRhtJB3ghSIqNutNoYq6+/t+akyfk1qn6xBSQdK9S5e yK5IDjpzV2FCfwFKNA8roa3pGyA6uEBbS/8hrPASVqfMXe38sUH875N/tnIqejsrTpdu x15w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=fvZ1gSD4IexYnMpshzD56bFQ2RImKC8UwmCBJy/56WI=; b=Ic6YwRsNb5e+113YND2QftkSiXkRMZxSSfSsAUv9qPLBUP3yMr9zlzynNKjE2wSE0N LutPhw6PVq3DONuinGpq6CRNAudKU8gZ4QBtMZGV3mqTz8a91l1DdBfj1ikQVZMSXVTj +XURIw1YKvL2Z14PvKIoYGfQGQ5DlQDBhXPtnH0yVRZlBcvhtH1Lz8BhXm5P5w3e6UC5 t0A/rLyNdqfZ6FoQ4sx1OQFoK2GIsJXyVDwfl9EKBy8XVpdBhPj/w+b4r0EScQlYUiqb 5lEfEoHyWxAzkE5qlC/bGSEpmlkDaEtyPMO4RtRLvyTIPK0IZadks1bOjU9IUckjSnZt LEeA== X-Gm-Message-State: ACgBeo2ZBfVoWGxwzGo5tXzB3Zy4iBC2H3/aCUtARAWZfb3K63hx2iNj fi4OUBFHO4XYs7z4DXewFMZC X-Google-Smtp-Source: AA6agR4ymkkCwIsHh4APISh7UVQAFidVJrEGZ+phlaNfxsMAVCIqn7hubi3XSs6fgMIH4+b45wdlwA== X-Received: by 2002:a17:902:7e89:b0:170:94d6:be73 with SMTP id z9-20020a1709027e8900b0017094d6be73mr1062634pla.52.1659788179877; Sat, 06 Aug 2022 05:16:19 -0700 (PDT) Received: from thinkpad ([117.202.188.20]) by smtp.gmail.com with ESMTPSA id y16-20020a17090a16d000b001ef87123615sm4613055pje.37.2022.08.06.05.16.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 06 Aug 2022 05:16:19 -0700 (PDT) Date: Sat, 6 Aug 2022 17:46:14 +0530 From: Manivannan Sadhasivam To: Pali =?iso-8859-1?Q?Roh=E1r?= Cc: Krzysztof Kozlowski , Rob Herring , Mauri Sandberg , devicetree@vger.kernel.org, linux-pci@vger.kernel.org Subject: Re: How to correctly define memory range of PCIe config space Message-ID: <20220806121614.GA11359@thinkpad> References: <20220710225108.bgedria6igtqpz5l@pali> <20220806110613.GB4516@thinkpad> <20220806111702.ezzknr76a4imej4u@pali> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20220806111702.ezzknr76a4imej4u@pali> Precedence: bulk List-ID: X-Mailing-List: devicetree@vger.kernel.org On Sat, Aug 06, 2022 at 01:17:02PM +0200, Pali Rohár wrote: > On Saturday 06 August 2022 16:36:13 Manivannan Sadhasivam wrote: > > Hi Pali, > > > > On Mon, Jul 11, 2022 at 12:51:08AM +0200, Pali Rohár wrote: > > > Hello! > > > > > > Together with Mauri we are working on extending pci-mvebu.c driver to > > > support Orion PCIe controllers as these controllers are same as mvebu > > > controller. > > > > > > There is just one big difference: Config space access on Orion is > > > different. mvebu uses classic Intel CFC/CF8 registers for indirect > > > config space access but Orion has direct memory mapped config space. > > > So Orion DTS files need to have this memory range for config space and > > > pci-mvebu.c driver have to read this range from DTS and properly map it. > > > > > > So my question is: How to properly define config space range in device > > > tree file? In which device tree property and in which format? Please > > > note that this memory range of config space is PCIe root port specific > > > and it requires its own MBUS_ID() like memory range of PCIe MEM and PCIe > > > IO mapping. Please look e.g. at armada-385.dtsi how are MBUS_ID() used: > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/armada-385.dtsi > > > > > > > On most of the platforms, the standard "reg" property is used to specify the > > config space together with other device specific memory regions. For instance, > > on the Qcom platforms based on Designware IP, we have below regions: > > > > reg = <0xfc520000 0x2000>, > > <0xff000000 0x1000>, > > <0xff001000 0x1000>, > > <0xff002000 0x2000>; > > reg-names = "parf", "dbi", "elbi", "config"; > > > > Where "parf" and "elbi" are Qcom controller specific regions, while "dbi" and > > "config" (config space) are common to all Designware IPs. > > > > These properties are documented in: Documentation/devicetree/bindings/pci/qcom,pcie.yaml > > > > Hope this helps! > > Hello! I have already looked at this. But as I pointed in above > armada-385.dtsi file, mvebu is quite complicated. First it does not use > explicit address ranges, but rather macros MBUS_ID() which assign > addresses at kernel runtime by mbus driver. Second issue is that config > space range (like any other resources) are pcie root port specific. So > it cannot be in pcie controller node and in pcie devices is "reg" > property reserved for pci bdf address. > > In last few days, I spent some time on this issue and after reading lot > of pcie dts files, including bindings and other documents (including > open firmware pci2_1.pdf) and I'm proposing following definition: > > soc { > pcie-mem-aperture = <0xe0000000 0x08000000>; /* 128 MiB memory space */ > pcie-cfg-aperture = <0xf0000000 0x01000000>; /* 16 MiB config space */ > pcie-io-aperture = <0xf2000000 0x00100000>; /* 1 MiB I/O space */ > > pcie { > ranges = <0x82000000 0 0x40000 MBUS_ID(0xf0, 0x01) 0x40000 0x0 0x2000>, /* Port 0.0 Internal registers */ > <0x82000000 0 0xf0000000 MBUS_ID(0x04, 0x79) 0 0x0 0x1000000>, /* Port 0.0 Config space */ > <0x82000000 1 0x0 MBUS_ID(0x04, 0x59) 0 0x1 0x0>, /* Port 0.0 Mem */ > <0x81000000 1 0x0 MBUS_ID(0x04, 0x51) 0 0x1 0x0>, /* Port 0.0 I/O */ > > pcie@1,0 { > reg = <0x0800 0 0 0 0>; /* BDF 0:1.0 */ > assigned-addresses = <0x82000800 0 0x40000 0x0 0x2000>, /* Port 0.0 Internal registers */ > <0x82000800 0 0xf0000000 0x0 0x1000000>; /* Port 0.0 Config space */ > ranges = <0x82000000 0 0 0x82000000 1 0 0x1 0x0>, /* Port 0.0 Mem */ > 0x81000000 0 0 0x81000000 1 0 0x1 0x0>; /* Port 0.0 I/O */ > }; > }; > }; > > So the pci config space address range would be defined in > "assigned-addresses" property as the _second_ value. First value is > already used for specifying internal registers (similar what is "parf" > for qcom). > Sounds reasonable to me. Another option would be to introduce a mvebu specific property but that would be the least preferred option I guess. But the fact that "assigned-addresses" property is described as "MMIO registers" also adds up to the justification IMO. Rob/Krzysztof could always correct that during binding review. > config space is currently limited to 16 MB (without extended PCIe), but > after we find free continuous physical address window of size 256MB we > can extend it to full PCIe config space range. > > Any objections to above device tree definition? > Are you also converting the binding to YAML for validation? Thanks, Mani > > Thanks, > > Mani > > > > > Krzysztof, would you be able to help with proper definition of this > > > property, so it would be fine also for schema checkers or other > > > automatic testing tools? > > > > -- > > மணிவண்ணன் சதாசிவம் -- மணிவண்ணன் சதாசிவம்