Re: [PATCH v9 3/7] PCI: Add parent_bus_offset to resource_entry

[Bjorn Helgaas <helgaas@kernel.org>]

On Tue, Jan 28, 2025 at 05:07:36PM -0500, Frank Li wrote:
> Introduce `parent_bus_offset` in `resource_entry` and a new API,
> `pci_add_resource_parent_bus_offset()`, to provide necessary information
> for PCI controllers with address translation units.
> 
> Typical PCI data flow involves:
>   CPU (CPU address) -> Bus Fabric (Intermediate address) ->
>   PCI Controller (PCI bus address) -> PCI Bus.
> 
> While most bus fabrics preserve address consistency, some modify addresses
> to intermediate values. The `parent_bus_offset` enables PCI controllers to
> translate these intermediate addresses correctly to PCI bus addresses.
> 
> Pave the road to remove hardcoded cpu_addr_fixup() and similar patterns in
> PCI controller drivers.
> ...

> +++ b/drivers/pci/of.c
> @@ -402,7 +402,17 @@ static int devm_of_pci_get_host_bridge_resources(struct device *dev,
>  			res->flags &= ~IORESOURCE_MEM_64;
>  		}
>  
> -		pci_add_resource_offset(resources, res,	res->start - range.pci_addr);
> +		/*
> +		 * IORESOURCE_IO res->start is io space start address.
> +		 * IORESOURCE_MEM res->start is cpu start address, which is the
> +		 * same as range.cpu_addr.
> +		 *
> +		 * Use (range.cpu_addr - range.parent_bus_addr) to align both
> +		 * IO and MEM's parent_bus_offset always offset to cpu address.
> +		 */
> +
> +		pci_add_resource_parent_bus_offset(resources, res, res->start - range.pci_addr,
> +						   range.cpu_addr - range.parent_bus_addr);

I don't know exactly where it needs to go, but I think we can call
.cpu_addr_fixup() once at startup on the base of the region.  This
will tell us the offset that applies to the entire region, i.e.,
parent_bus_offset.

Then we can remove all the .cpu_addr_fixup() calls in
cdns_pcie_host_init_address_translation(),
cdns_pcie_set_outbound_region(), and dw_pcie_prog_outbound_atu().

Until we can get rid of all the .cpu_addr_fixup() implementations,
We'll still have that single call at startup (I guess once for cadence
and another for designware), but it should simplify the current
callers quite a bit.

>  	}
>  
>  	/* Check for dma-ranges property */
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 47b31ad724fa5..0d7e67b47be47 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1510,6 +1510,8 @@ static inline void pci_release_config_region(struct pci_dev *pdev,
>  void pci_add_resource(struct list_head *resources, struct resource *res);
>  void pci_add_resource_offset(struct list_head *resources, struct resource *res,
>  			     resource_size_t offset);
> +void pci_add_resource_parent_bus_offset(struct list_head *resources, struct resource *res,
> +					resource_size_t offset, resource_size_t parent_bus_offset);
>  void pci_free_resource_list(struct list_head *resources);
>  void pci_bus_add_resource(struct pci_bus *bus, struct resource *res);
>  struct resource *pci_bus_resource_n(const struct pci_bus *bus, int n);
> diff --git a/include/linux/resource_ext.h b/include/linux/resource_ext.h
> index ff0339df56afc..b6ec6cc318203 100644
> --- a/include/linux/resource_ext.h
> +++ b/include/linux/resource_ext.h
> @@ -24,6 +24,7 @@ struct resource_entry {
>  	struct list_head	node;
>  	struct resource		*res;	/* In master (CPU) address space */
>  	resource_size_t		offset;	/* Translation offset for bridge */
> +	resource_size_t		parent_bus_offset; /* Parent bus address offset for bridge */
>  	struct resource		__res;	/* Default storage for res */
>  };
>  
> 
> -- 
> 2.34.1
>