From: Jason Gunthorpe <jgunthorpe@obsidianresearch.com>
To: Arnd Bergmann <arnd@arndb.de>
Cc: "devicetree@vger.kernel.org" <devicetree@vger.kernel.org>,
"linaro-kernel@lists.linaro.org" <linaro-kernel@lists.linaro.org>,
linux-pci <linux-pci@vger.kernel.org>,
Liviu Dudau <Liviu.Dudau@arm.com>,
LKML <linux-kernel@vger.kernel.org>,
Catalin Marinas <Catalin.Marinas@arm.com>,
Bjorn Helgaas <bhelgaas@google.com>,
LAKML <linux-arm-kernel@lists.infradead.org>
Subject: Re: [PATCH] arm64: Add architecture support for PCI
Date: Tue, 4 Feb 2014 12:10:55 -0700 [thread overview]
Message-ID: <20140204191055.GC25695@obsidianresearch.com> (raw)
In-Reply-To: <8676627.b6SYsazoah@wuerfel>
On Tue, Feb 04, 2014 at 07:34:50PM +0100, Arnd Bergmann wrote:
> Well, the way I see it, we already have support for arbitrary
> PCI domains in the kernel, and that works fine, so we can just
> as well use it. That way we don't have to partition the available
> 256 buses among the host bridges, and anything that needs a separate
> PCI config space can live in its own world. Quite often when you
> have multiple PCI hosts, they actually have different ways to
> get at the config space and don't even share the same driver.
> On x86, any kind of HT/PCI/PCIe/PCI-x bridge is stuffed into a
> single domain so they can support OSs that only know the
> traditional config space access methods, but I don't see
> any real advantage to that for other architectures.
Supporting a standard configration interface is a solid reason, but
there is alot more going on.
For instance to support peer-to-peer IO you need to have a consisent,
non-overlapping set of bus/device/function/tag to uniquely route TLPs
within the chip. Cross domain TLP routing in HW is non-trivial.
IOMMUs (and SR-IOv) rely on the BDF to identify the originating device
for each TLP. Multiple domains means a much more complex IOMMU
environment.
Failure to integrate on-chip devices into the PCI world also means
thing like SR-IOv won't work sanely with on-chip devices.
The only reason we should see multi-domain on a SOC is because the HW
design was lazy. Being lazy misses the Big Picture where PCI is the
cornerstone of many important Server/Enterprise technologies.
> > SOC internal peripherals should all show up in the bus 0 config space
> > of the only domain and SOC PCI-E physical ports should show up on bus
> > 0 as PCI-PCI bridges. This is all covered in the PCI-E specs regarding
> > the root complex.
> >
> > Generally I would expect the internal peripherals to still be
> > internally connected with AXI, but also connected through the ECAM
> > space for configuration, control, power management and address
> > assignment.
>
> That would of course be very nice from a software perspective,
> but I think that is much less likely for any practical
> implementation.
Well, all x86 implementations do this already.. It actually isn't that
big a deal from a HW perspective, you just have to think about it
fully, understand PCI, and position your registers accordingly.
Jason
next prev parent reply other threads:[~2014-02-04 19:10 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-02-03 18:43 [PATCH] [RFC] Add AArch64 support for PCI Liviu Dudau
2014-02-03 18:43 ` [PATCH] arm64: Add architecture " Liviu Dudau
2014-02-03 18:58 ` Arnd Bergmann
2014-02-03 19:18 ` Liviu Dudau
2014-02-03 19:21 ` kodiak furr
2014-02-03 20:05 ` Arnd Bergmann
2014-02-03 21:36 ` Liviu Dudau
2014-02-04 8:44 ` Arnd Bergmann
2014-02-04 11:09 ` Liviu Dudau
2014-02-04 11:54 ` Arnd Bergmann
2014-02-04 16:41 ` Catalin Marinas
2014-02-03 23:07 ` Rob Herring
2014-02-03 23:31 ` Jason Gunthorpe
2014-02-04 9:44 ` Arnd Bergmann
2014-02-04 13:57 ` Rob Herring
2014-02-04 19:50 ` Arnd Bergmann
2014-02-04 18:15 ` Jason Gunthorpe
2014-02-04 18:34 ` Arnd Bergmann
2014-02-04 19:10 ` Jason Gunthorpe [this message]
2014-02-04 19:21 ` Arnd Bergmann
2014-02-04 9:01 ` Arnd Bergmann
2014-02-03 22:34 ` Andrew Murray
2014-02-04 12:29 ` Liviu Dudau
2014-02-04 13:23 ` Andrew Murray
2014-02-04 16:18 ` Arnd Bergmann
2014-02-18 6:33 ` Yijing Wang
2014-02-20 14:38 ` Liviu Dudau
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140204191055.GC25695@obsidianresearch.com \
--to=jgunthorpe@obsidianresearch.com \
--cc=Catalin.Marinas@arm.com \
--cc=Liviu.Dudau@arm.com \
--cc=arnd@arndb.de \
--cc=bhelgaas@google.com \
--cc=devicetree@vger.kernel.org \
--cc=linaro-kernel@lists.linaro.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).