Re: [PATCH v5] PCI: Unify ECAM constants in native PCI Express drivers

Bjorn Helgaas
2020-11-28T22:07:24Z

On Fri, Nov 27, 2020 at 10:46:26AM +0000, Krzysztof Wilczyński wrote:
> Unify ECAM-related constants into a single set of standard constants
> defining memory address shift values for the byte-level address that can
> be used when accessing the PCI Express Configuration Space, and then
> move native PCI Express controller drivers to use newly introduced
> definitions retiring any driver-specific ones.
>
> The ECAM ("Enhanced Configuration Access Mechanism") is defined by the
> PCI Express specification (see PCI Express Base Specification, Revision
> 5.0, Version 1.0, Section 7.2.2, p. 676), thus most hardware should
> implement it the same way. Most of the native PCI Express controller
> drivers define their ECAM-related constants, many of these could be
> shared, or use open-coded values when setting the .bus_shift field of
> the struct pci_ecam_ops.
>
> All of the newly added constants should remove ambiguity and reduce the
> number of open-coded values, and also correlate more strongly with the
> descriptions in the aforementioned specification (see Table 7-1
> "Enhanced Configuration Address Mapping", p. 677).
>
> There is no change to functionality.
>
> Suggested-by: Bjorn Helgaas <bhelgaas@google.com>
> Signed-off-by: Krzysztof Wilczyński <kw@linux.com>

Beautiful. This should probably go via Lorenzo's tree, so he may have
comments, too. Could apply this as-is; I had a few trivial notes
below.

It's ironic that we don't use PCIE_ECAM_OFFSET in drivers/pci/ecam.c.
We could do something like this, which would also let us drop
.bus_shift completely in all the conforming implementations. It also
closes the hole that we didn't limit "where" to 4K for
pci_ecam_map_bus() users.

if (per_bus_mapping) {
base = cfg->winp[busn];
busn = 0;
} else {
base = cfg->win;
}

if (cfg->ops->bus_shift) {
u32 bus_offset = (busn & 0xff) << cfg->ops->bus_shift;
u32 devfn_offset = (devfn & 0xff) << (cfg->ops->bus_shift - 8);

where &= 0xfff;

return base + (bus_offset | devfn_offset | where);
}

return base + PCIE_ECAM_OFFSET(busn, devfn, where);

Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>

> static void __iomem *ppc4xx_pciex_get_config_base(struct ppc4xx_pciex_port *port,
> struct pci_bus *bus,
> - unsigned int devfn)
> + unsigned int devfn,
> + int offset)

The interface change (to add "offset") could be a preparatory patch by
itself.

But I'm actually not sure it's worth even touching this file. This is
the only place outside drivers/pci that includes linux/pci-ecam.h. I
think I might rather put PCIE_ECAM_OFFSET() and related things in
drivers/pci/pci.h and keep it all inside drivers/pci.

> static const struct pci_ecam_ops pci_thunder_pem_ops = {
> - .bus_shift = 24,
> + .bus_shift = THUNDER_PCIE_ECAM_BUS_SHIFT,
> .init = thunder_pem_platform_init,
> .pci_ops = {
> .map_bus = pci_ecam_map_bus,

This could be split to its own patch, no big deal either way.

> const struct pci_ecam_ops xgene_v2_pcie_ecam_ops = {
> - .bus_shift = 16,
> .init = xgene_v2_pcie_ecam_init,
> .pci_ops = {
> .map_bus = xgene_pcie_map_bus,

Thanks for mentioning this change in the cover letter. It could also
be split off to a preparatory patch, since it's not related to
PCIE_ECAM_OFFSET(), which is the main point of this patch.

> static void __iomem *iproc_pcie_map_ep_cfg_reg(struct iproc_pcie *pcie,
> unsigned int busno,
> - unsigned int slot,
> - unsigned int fn,
> + unsigned int devfn,

This interface change *could* be a separate preparatory patch, too,
but I'm starting to feel even more OCD than usual :)

> @@ -94,7 +95,7 @@ struct vmd_dev {
> struct pci_dev *dev;
>
> spinlock_t cfg_lock;
> - char __iomem *cfgbar;
> + void __iomem *cfgbar;

This type change might be worth pushing to a separate patch since the
casting issues are not completely trivial.