* [PATCH v4 1/4] xen/arm: allow PCI host bridge to have private data
2025-04-23 11:08 [PATCH v4 0/4] Add support for R-Car Gen4 PCI host controller Mykyta Poturai
@ 2025-04-23 11:08 ` Mykyta Poturai
2025-04-23 11:08 ` [PATCH v4 3/4] xen/arm: add support for PCI child bus Mykyta Poturai
` (2 subsequent siblings)
3 siblings, 0 replies; 12+ messages in thread
From: Mykyta Poturai @ 2025-04-23 11:08 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Andrushchenko, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk, Mykyta Poturai
From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Some of the PCI host bridges require private data. Add priv field
to struct pci_host_bridge, so such bridges may populate it with
their private data.
Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
v3->v4:
* Added Stefano's RB
v2->v3:
* removed priv allocation from common code
v1->v2:
* no change
---
xen/arch/arm/include/asm/pci.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/xen/arch/arm/include/asm/pci.h b/xen/arch/arm/include/asm/pci.h
index 7f77226c9b..a87672d834 100644
--- a/xen/arch/arm/include/asm/pci.h
+++ b/xen/arch/arm/include/asm/pci.h
@@ -66,6 +66,7 @@ struct pci_host_bridge {
uint16_t segment; /* Segment number */
struct pci_config_window* cfg; /* Pointer to the bridge config window */
const struct pci_ops *ops;
+ void *priv; /* Private data of the bridge. */
};
struct pci_ops {
--
2.34.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH v4 3/4] xen/arm: add support for PCI child bus
2025-04-23 11:08 [PATCH v4 0/4] Add support for R-Car Gen4 PCI host controller Mykyta Poturai
2025-04-23 11:08 ` [PATCH v4 1/4] xen/arm: allow PCI host bridge to have private data Mykyta Poturai
@ 2025-04-23 11:08 ` Mykyta Poturai
2025-04-23 18:34 ` Stewart Hildebrand
` (2 more replies)
2025-04-23 11:08 ` [PATCH v4 2/4] xen/arm: make pci_host_common_probe return the bridge Mykyta Poturai
2025-04-23 11:08 ` [PATCH v4 4/4] xen/arm: add support for R-Car Gen4 PCI host controller Mykyta Poturai
3 siblings, 3 replies; 12+ messages in thread
From: Mykyta Poturai @ 2025-04-23 11:08 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Andrushchenko, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk, Mykyta Poturai
From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
PCI host bridges often have different ways to access the root and child
bus configuration spaces. One of the examples is Designware's host bridge
and its multiple clones [1].
Linux kernel implements this by instantiating a child bus when device
drivers provide not only the usual pci_ops to access ECAM space (this is
the case for the generic host bridge), but also means to access the child
bus which has a dedicated configuration space and own implementation for
accessing the bus, e.g. child_ops.
For Xen it is not feasible to fully implement PCI bus infrastructure as
Linux kernel does, but still child bus can be supported.
Add support for the PCI child bus which includes the following changes:
- introduce bus mapping functions depending on SBDF
- assign bus start and end for the child bus and re-configure the same for
the parent (root) bus
- make pci_find_host_bridge be aware of multiple busses behind the same bridge
- update pci_host_bridge_mappings, so it also doesn't map to guest the memory
spaces belonging to the child bus
- make pci_host_common_probe accept one more pci_ops structure for the child bus
- install MMIO handlers for the child bus
- re-work vpci_mmio_{write|read} with parent and child approach in mind
[1] https://elixir.bootlin.com/linux/v5.15/source/drivers/pci/controller/dwc
Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
---
v3->v4:
* remove changes to pci_ecam_map_bus
* make map_bus inline
* fix logic error in pci_host_bridge_mappings
v2->v3:
* no change
v1->v2:
* fixed compilation issues
---
xen/arch/arm/include/asm/pci.h | 9 ++-
xen/arch/arm/pci/pci-access.c | 37 +++++++++++--
xen/arch/arm/pci/pci-host-common.c | 86 +++++++++++++++++++++++------
xen/arch/arm/pci/pci-host-generic.c | 2 +-
xen/arch/arm/pci/pci-host-zynqmp.c | 2 +-
xen/arch/arm/vpci.c | 83 ++++++++++++++++++++++------
6 files changed, 177 insertions(+), 42 deletions(-)
diff --git a/xen/arch/arm/include/asm/pci.h b/xen/arch/arm/include/asm/pci.h
index 3d2ca9b5b0..94ec63f3c8 100644
--- a/xen/arch/arm/include/asm/pci.h
+++ b/xen/arch/arm/include/asm/pci.h
@@ -67,6 +67,9 @@ struct pci_host_bridge {
uint16_t segment; /* Segment number */
struct pci_config_window* cfg; /* Pointer to the bridge config window */
const struct pci_ops *ops;
+ /* Child bus */
+ struct pci_config_window *child_cfg;
+ const struct pci_ops *child_ops;
void *priv; /* Private data of the bridge. */
};
@@ -96,8 +99,10 @@ struct pci_ecam_ops {
/* Default ECAM ops */
extern const struct pci_ecam_ops pci_generic_ecam_ops;
-struct pci_host_bridge *pci_host_common_probe(struct dt_device_node *dev,
- const struct pci_ecam_ops *ops);
+struct pci_host_bridge *
+pci_host_common_probe(struct dt_device_node *dev,
+ const struct pci_ecam_ops *ops,
+ const struct pci_ecam_ops *child_ops);
int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
uint32_t reg, uint32_t len, uint32_t *value);
int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
index 9f9aac43d7..91271973fb 100644
--- a/xen/arch/arm/pci/pci-access.c
+++ b/xen/arch/arm/pci/pci-access.c
@@ -18,10 +18,31 @@
#define INVALID_VALUE (~0U)
#define PCI_ERR_VALUE(len) GENMASK(0, len * 8)
+static const struct pci_ops *get_ops(struct pci_host_bridge *bridge,
+ pci_sbdf_t sbdf)
+{
+ if ( bridge->child_ops )
+ {
+ struct pci_config_window *cfg = bridge->child_cfg;
+
+ if ( (sbdf.bus >= cfg->busn_start) && (sbdf.bus <= cfg->busn_end) )
+ return bridge->child_ops;
+ }
+ return bridge->ops;
+}
+
+static inline void __iomem *map_bus(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
+ uint32_t reg)
+{
+ const struct pci_ops *ops = get_ops(bridge, sbdf);
+
+ return ops->map_bus(bridge, sbdf, reg);
+}
+
int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
uint32_t reg, uint32_t len, uint32_t *value)
{
- void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);
+ void __iomem *addr = map_bus(bridge, sbdf, reg);
if ( !addr )
{
@@ -50,7 +71,7 @@ int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
uint32_t reg, uint32_t len, uint32_t value)
{
- void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);
+ void __iomem *addr = map_bus(bridge, sbdf, reg);
if ( !addr )
return -ENODEV;
@@ -78,14 +99,16 @@ static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg,
{
uint32_t val = PCI_ERR_VALUE(len);
struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
+ const struct pci_ops *ops;
if ( unlikely(!bridge) )
return val;
- if ( unlikely(!bridge->ops->read) )
+ ops = get_ops(bridge, sbdf);
+ if ( unlikely(!ops->read) )
return val;
- bridge->ops->read(bridge, sbdf, reg, len, &val);
+ ops->read(bridge, sbdf, reg, len, &val);
return val;
}
@@ -94,14 +117,16 @@ static void pci_config_write(pci_sbdf_t sbdf, unsigned int reg,
unsigned int len, uint32_t val)
{
struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
+ const struct pci_ops *ops;
if ( unlikely(!bridge) )
return;
- if ( unlikely(!bridge->ops->write) )
+ ops = get_ops(bridge, sbdf);
+ if ( unlikely(!ops->write) )
return;
- bridge->ops->write(bridge, sbdf, reg, len, val);
+ ops->write(bridge, sbdf, reg, len, val);
}
/*
diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index 53953d4895..1d0dc1b4bd 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -57,17 +57,12 @@ static void pci_ecam_free(struct pci_config_window *cfg)
xfree(cfg);
}
-static struct pci_config_window * __init
-gen_pci_init(struct dt_device_node *dev, const struct pci_ecam_ops *ops)
+static void __init gen_pci_init_bus_range(struct dt_device_node *dev,
+ struct pci_host_bridge *bridge,
+ struct pci_config_window *cfg)
{
- int err, cfg_reg_idx;
u32 bus_range[2];
- paddr_t addr, size;
- struct pci_config_window *cfg;
-
- cfg = xzalloc(struct pci_config_window);
- if ( !cfg )
- return NULL;
+ int err;
err = dt_property_read_u32_array(dev, "bus-range", bus_range,
ARRAY_SIZE(bus_range));
@@ -82,6 +77,36 @@ gen_pci_init(struct dt_device_node *dev, const struct pci_ecam_ops *ops)
if ( cfg->busn_end > cfg->busn_start + 0xff )
cfg->busn_end = cfg->busn_start + 0xff;
}
+}
+
+static void __init gen_pci_init_bus_range_child(struct dt_device_node *dev,
+ struct pci_host_bridge *bridge,
+ struct pci_config_window *cfg)
+{
+ cfg->busn_start = bridge->cfg->busn_start + 1;
+ cfg->busn_end = bridge->cfg->busn_end;
+ bridge->cfg->busn_end = bridge->cfg->busn_start;
+
+ printk(XENLOG_INFO "Root bus end updated: [bus %x-%x]\n",
+ bridge->cfg->busn_start, bridge->cfg->busn_end);
+}
+
+static struct pci_config_window *__init
+gen_pci_init(struct dt_device_node *dev, struct pci_host_bridge *bridge,
+ const struct pci_ecam_ops *ops,
+ void (*init_bus_range)(struct dt_device_node *dev,
+ struct pci_host_bridge *bridge,
+ struct pci_config_window *cfg))
+{
+ int err, cfg_reg_idx;
+ paddr_t addr, size;
+ struct pci_config_window *cfg;
+
+ cfg = xzalloc(struct pci_config_window);
+ if ( !cfg )
+ return NULL;
+
+ init_bus_range(dev, bridge, cfg);
if ( ops->cfg_reg_index )
{
@@ -208,8 +233,10 @@ static int pci_bus_find_domain_nr(struct dt_device_node *dev)
return domain;
}
-struct pci_host_bridge *pci_host_common_probe(struct dt_device_node *dev,
- const struct pci_ecam_ops *ops)
+struct pci_host_bridge *
+pci_host_common_probe(struct dt_device_node *dev,
+ const struct pci_ecam_ops *ops,
+ const struct pci_ecam_ops *child_ops)
{
struct pci_host_bridge *bridge;
struct pci_config_window *cfg;
@@ -224,7 +251,7 @@ struct pci_host_bridge *pci_host_common_probe(struct dt_device_node *dev,
return ERR_PTR(-ENOMEM);
/* Parse and map our Configuration Space windows */
- cfg = gen_pci_init(dev, ops);
+ cfg = gen_pci_init(dev, bridge, ops, gen_pci_init_bus_range);
if ( !cfg )
{
err = -ENOMEM;
@@ -242,10 +269,29 @@ struct pci_host_bridge *pci_host_common_probe(struct dt_device_node *dev,
BUG();
}
bridge->segment = domain;
+
+ if ( child_ops )
+ {
+ /* Parse and map child's Configuration Space windows */
+ cfg = gen_pci_init(dev, bridge, child_ops,
+ gen_pci_init_bus_range_child);
+ if ( !cfg )
+ {
+ err = -ENOMEM;
+ goto err_child;
+ }
+
+ bridge->child_cfg = cfg;
+ bridge->child_ops = &child_ops->pci_ops;
+ }
+
pci_add_host_bridge(bridge);
return bridge;
+err_child:
+ xfree(bridge->cfg);
+
err_exit:
xfree(bridge);
@@ -280,9 +326,12 @@ struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus)
{
if ( bridge->segment != segment )
continue;
- if ( (bus < bridge->cfg->busn_start) || (bus > bridge->cfg->busn_end) )
- continue;
- return bridge;
+ if ( bridge->child_cfg && (bus >= bridge->child_cfg->busn_start) &&
+ (bus <= bridge->child_cfg->busn_end) )
+ return bridge;
+ if ( (bus >= bridge->cfg->busn_start) &&
+ (bus <= bridge->cfg->busn_end) )
+ return bridge;
}
return NULL;
@@ -348,6 +397,7 @@ int __init pci_host_bridge_mappings(struct domain *d)
{
const struct dt_device_node *dev = bridge->dt_node;
unsigned int i;
+ bool need_mapping;
for ( i = 0; i < dt_number_of_address(dev); i++ )
{
@@ -363,7 +413,11 @@ int __init pci_host_bridge_mappings(struct domain *d)
return err;
}
- if ( bridge->ops->need_p2m_hwdom_mapping(d, bridge, addr) )
+ need_mapping = bridge->ops->need_p2m_hwdom_mapping(d, bridge, addr);
+ if ( need_mapping || bridge->child_ops )
+ need_mapping =
+ bridge->child_ops->need_p2m_hwdom_mapping(d, bridge, addr);
+ if ( need_mapping )
{
err = map_range_to_domain(dev, addr, size, &mr_data);
if ( err )
diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c
index a6ffbda174..47cf144831 100644
--- a/xen/arch/arm/pci/pci-host-generic.c
+++ b/xen/arch/arm/pci/pci-host-generic.c
@@ -29,7 +29,7 @@ static const struct dt_device_match __initconstrel gen_pci_dt_match[] =
static int __init pci_host_generic_probe(struct dt_device_node *dev,
const void *data)
{
- return PTR_RET(pci_host_common_probe(dev, &pci_generic_ecam_ops));
+ return PTR_RET(pci_host_common_probe(dev, &pci_generic_ecam_ops, NULL));
}
DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI_HOSTBRIDGE)
diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c
index a38f2e019e..75d1235aaf 100644
--- a/xen/arch/arm/pci/pci-host-zynqmp.c
+++ b/xen/arch/arm/pci/pci-host-zynqmp.c
@@ -47,7 +47,7 @@ static const struct dt_device_match __initconstrel nwl_pcie_dt_match[] =
static int __init pci_host_generic_probe(struct dt_device_node *dev,
const void *data)
{
- return PTR_RET(pci_host_common_probe(dev, &nwl_pcie_ops));
+ return PTR_RET(pci_host_common_probe(dev, &nwl_pcie_ops, NULL));
}
DT_DEVICE_START(pci_gen, "PCI HOST ZYNQMP", DEVICE_PCI_HOSTBRIDGE)
diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
index b63a356bb4..468aee1db7 100644
--- a/xen/arch/arm/vpci.c
+++ b/xen/arch/arm/vpci.c
@@ -8,15 +8,17 @@
#include <asm/mmio.h>
static pci_sbdf_t vpci_sbdf_from_gpa(const struct pci_host_bridge *bridge,
- paddr_t gpa)
+ paddr_t gpa, bool use_root)
{
pci_sbdf_t sbdf;
if ( bridge )
{
- sbdf.sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr);
+ const struct pci_config_window *cfg = use_root ? bridge->cfg
+ : bridge->child_cfg;
+ sbdf.sbdf = VPCI_ECAM_BDF(gpa - cfg->phys_addr);
sbdf.seg = bridge->segment;
- sbdf.bus += bridge->cfg->busn_start;
+ sbdf.bus += cfg->busn_start;
}
else
sbdf.sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE);
@@ -24,11 +26,9 @@ static pci_sbdf_t vpci_sbdf_from_gpa(const struct pci_host_bridge *bridge,
return sbdf;
}
-static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
- register_t *r, void *p)
+static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info, register_t *r,
+ pci_sbdf_t sbdf)
{
- struct pci_host_bridge *bridge = p;
- pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
const unsigned int access_size = (1U << info->dabt.size) * 8;
const register_t invalid = GENMASK_ULL(access_size - 1, 0);
/* data is needed to prevent a pointer cast on 32bit */
@@ -46,31 +46,78 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
return 0;
}
-static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
- register_t r, void *p)
+static int vpci_mmio_read_root(struct vcpu *v, mmio_info_t *info, register_t *r,
+ void *p)
+{
+ struct pci_host_bridge *bridge = p;
+ pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa, true);
+
+ return vpci_mmio_read(v, info, r, sbdf);
+}
+
+static int vpci_mmio_read_child(struct vcpu *v, mmio_info_t *info,
+ register_t *r, void *p)
{
struct pci_host_bridge *bridge = p;
- pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
+ pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa, false);
+
+ return vpci_mmio_read(v, info, r, sbdf);
+}
+static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info, register_t r,
+ pci_sbdf_t sbdf)
+{
return vpci_ecam_write(sbdf, ECAM_REG_OFFSET(info->gpa),
1U << info->dabt.size, r);
}
+static int vpci_mmio_write_root(struct vcpu *v, mmio_info_t *info, register_t r,
+ void *p)
+{
+ struct pci_host_bridge *bridge = p;
+ pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa, true);
+
+ return vpci_mmio_write(v, info, r, sbdf);
+}
+
+static int vpci_mmio_write_child(struct vcpu *v, mmio_info_t *info,
+ register_t r, void *p)
+{
+ struct pci_host_bridge *bridge = p;
+ pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa, false);
+
+ return vpci_mmio_write(v, info, r, sbdf);
+}
+
static const struct mmio_handler_ops vpci_mmio_handler = {
- .read = vpci_mmio_read,
- .write = vpci_mmio_write,
+ .read = vpci_mmio_read_root,
+ .write = vpci_mmio_write_root,
+};
+
+static const struct mmio_handler_ops vpci_mmio_handler_child = {
+ .read = vpci_mmio_read_child,
+ .write = vpci_mmio_write_child,
};
static int vpci_setup_mmio_handler_cb(struct domain *d,
struct pci_host_bridge *bridge)
{
struct pci_config_window *cfg = bridge->cfg;
+ int count = 1;
register_mmio_handler(d, &vpci_mmio_handler,
cfg->phys_addr, cfg->size, bridge);
- /* We have registered a single MMIO handler. */
- return 1;
+ if ( bridge->child_ops )
+ {
+ struct pci_config_window *cfg = bridge->child_cfg;
+
+ register_mmio_handler(d, &vpci_mmio_handler_child, cfg->phys_addr,
+ cfg->size, bridge);
+ count++;
+ }
+
+ return count;
}
int domain_vpci_init(struct domain *d)
@@ -101,8 +148,12 @@ int domain_vpci_init(struct domain *d)
static int vpci_get_num_handlers_cb(struct domain *d,
struct pci_host_bridge *bridge)
{
- /* Each bridge has a single MMIO handler for the configuration space. */
- return 1;
+ int count = 1;
+
+ if ( bridge->child_cfg )
+ count++;
+
+ return count;
}
unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d)
--
2.34.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PATCH v4 3/4] xen/arm: add support for PCI child bus
2025-04-23 11:08 ` [PATCH v4 3/4] xen/arm: add support for PCI child bus Mykyta Poturai
@ 2025-04-23 18:34 ` Stewart Hildebrand
2025-04-23 20:59 ` Stefano Stabellini
2025-04-24 15:16 ` Stewart Hildebrand
2 siblings, 0 replies; 12+ messages in thread
From: Stewart Hildebrand @ 2025-04-23 18:34 UTC (permalink / raw)
To: Mykyta Poturai, xen-devel@lists.xenproject.org
Cc: Oleksandr Andrushchenko, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk
On 4/23/25 07:08, Mykyta Poturai wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>
> PCI host bridges often have different ways to access the root and child
> bus configuration spaces. One of the examples is Designware's host bridge
> and its multiple clones [1].
>
> Linux kernel implements this by instantiating a child bus when device
> drivers provide not only the usual pci_ops to access ECAM space (this is
> the case for the generic host bridge), but also means to access the child
> bus which has a dedicated configuration space and own implementation for
> accessing the bus, e.g. child_ops.
>
> For Xen it is not feasible to fully implement PCI bus infrastructure as
> Linux kernel does, but still child bus can be supported.
>
> Add support for the PCI child bus which includes the following changes:
> - introduce bus mapping functions depending on SBDF
> - assign bus start and end for the child bus and re-configure the same for
> the parent (root) bus
> - make pci_find_host_bridge be aware of multiple busses behind the same bridge
> - update pci_host_bridge_mappings, so it also doesn't map to guest the memory
> spaces belonging to the child bus
> - make pci_host_common_probe accept one more pci_ops structure for the child bus
> - install MMIO handlers for the child bus
Nit: perhaps clarify that they are only installed for the hardware domain
> - re-work vpci_mmio_{write|read} with parent and child approach in mind
>
> [1] https://elixir.bootlin.com/linux/v5.15/source/drivers/pci/controller/dwc
>
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
> ---
> v3->v4:
> * remove changes to pci_ecam_map_bus
> * make map_bus inline
> * fix logic error in pci_host_bridge_mappings
>
> v2->v3:
> * no change
>
> v1->v2:
> * fixed compilation issues
> ---
> xen/arch/arm/include/asm/pci.h | 9 ++-
> xen/arch/arm/pci/pci-access.c | 37 +++++++++++--
> xen/arch/arm/pci/pci-host-common.c | 86 +++++++++++++++++++++++------
> xen/arch/arm/pci/pci-host-generic.c | 2 +-
> xen/arch/arm/pci/pci-host-zynqmp.c | 2 +-
> xen/arch/arm/vpci.c | 83 ++++++++++++++++++++++------
> 6 files changed, 177 insertions(+), 42 deletions(-)
>
> diff --git a/xen/arch/arm/include/asm/pci.h b/xen/arch/arm/include/asm/pci.h
> index 3d2ca9b5b0..94ec63f3c8 100644
> --- a/xen/arch/arm/include/asm/pci.h
> +++ b/xen/arch/arm/include/asm/pci.h
> @@ -67,6 +67,9 @@ struct pci_host_bridge {
> uint16_t segment; /* Segment number */
> struct pci_config_window* cfg; /* Pointer to the bridge config window */
> const struct pci_ops *ops;
> + /* Child bus */
> + struct pci_config_window *child_cfg;
> + const struct pci_ops *child_ops;
> void *priv; /* Private data of the bridge. */
> };
>
> @@ -96,8 +99,10 @@ struct pci_ecam_ops {
> /* Default ECAM ops */
> extern const struct pci_ecam_ops pci_generic_ecam_ops;
>
> -struct pci_host_bridge *pci_host_common_probe(struct dt_device_node *dev,
> - const struct pci_ecam_ops *ops);
> +struct pci_host_bridge *
> +pci_host_common_probe(struct dt_device_node *dev,
> + const struct pci_ecam_ops *ops,
> + const struct pci_ecam_ops *child_ops);
> int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> uint32_t reg, uint32_t len, uint32_t *value);
> int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
> index 9f9aac43d7..91271973fb 100644
> --- a/xen/arch/arm/pci/pci-access.c
> +++ b/xen/arch/arm/pci/pci-access.c
> @@ -18,10 +18,31 @@
> #define INVALID_VALUE (~0U)
> #define PCI_ERR_VALUE(len) GENMASK(0, len * 8)
>
> +static const struct pci_ops *get_ops(struct pci_host_bridge *bridge,
> + pci_sbdf_t sbdf)
> +{
> + if ( bridge->child_ops )
> + {
> + struct pci_config_window *cfg = bridge->child_cfg;
> +
> + if ( (sbdf.bus >= cfg->busn_start) && (sbdf.bus <= cfg->busn_end) )
> + return bridge->child_ops;
> + }
> + return bridge->ops;
> +}
> +
> +static inline void __iomem *map_bus(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> + uint32_t reg)
> +{
> + const struct pci_ops *ops = get_ops(bridge, sbdf);
> +
> + return ops->map_bus(bridge, sbdf, reg);
> +}
> +
> int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> uint32_t reg, uint32_t len, uint32_t *value)
> {
> - void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);
> + void __iomem *addr = map_bus(bridge, sbdf, reg);
>
> if ( !addr )
> {
> @@ -50,7 +71,7 @@ int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> uint32_t reg, uint32_t len, uint32_t value)
> {
> - void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg);
> + void __iomem *addr = map_bus(bridge, sbdf, reg);
>
> if ( !addr )
> return -ENODEV;
> @@ -78,14 +99,16 @@ static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg,
> {
> uint32_t val = PCI_ERR_VALUE(len);
> struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
> + const struct pci_ops *ops;
>
> if ( unlikely(!bridge) )
> return val;
>
> - if ( unlikely(!bridge->ops->read) )
> + ops = get_ops(bridge, sbdf);
> + if ( unlikely(!ops->read) )
> return val;
>
> - bridge->ops->read(bridge, sbdf, reg, len, &val);
> + ops->read(bridge, sbdf, reg, len, &val);
>
> return val;
> }
> @@ -94,14 +117,16 @@ static void pci_config_write(pci_sbdf_t sbdf, unsigned int reg,
> unsigned int len, uint32_t val)
> {
> struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus);
> + const struct pci_ops *ops;
>
> if ( unlikely(!bridge) )
> return;
>
> - if ( unlikely(!bridge->ops->write) )
> + ops = get_ops(bridge, sbdf);
> + if ( unlikely(!ops->write) )
> return;
>
> - bridge->ops->write(bridge, sbdf, reg, len, val);
> + ops->write(bridge, sbdf, reg, len, val);
> }
>
> /*
> diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
> index 53953d4895..1d0dc1b4bd 100644
> --- a/xen/arch/arm/pci/pci-host-common.c
> +++ b/xen/arch/arm/pci/pci-host-common.c
> @@ -57,17 +57,12 @@ static void pci_ecam_free(struct pci_config_window *cfg)
> xfree(cfg);
> }
>
> -static struct pci_config_window * __init
> -gen_pci_init(struct dt_device_node *dev, const struct pci_ecam_ops *ops)
> +static void __init gen_pci_init_bus_range(struct dt_device_node *dev,
> + struct pci_host_bridge *bridge,
> + struct pci_config_window *cfg)
> {
> - int err, cfg_reg_idx;
> u32 bus_range[2];
> - paddr_t addr, size;
> - struct pci_config_window *cfg;
> -
> - cfg = xzalloc(struct pci_config_window);
> - if ( !cfg )
> - return NULL;
> + int err;
>
> err = dt_property_read_u32_array(dev, "bus-range", bus_range,
> ARRAY_SIZE(bus_range));
> @@ -82,6 +77,36 @@ gen_pci_init(struct dt_device_node *dev, const struct pci_ecam_ops *ops)
> if ( cfg->busn_end > cfg->busn_start + 0xff )
> cfg->busn_end = cfg->busn_start + 0xff;
> }
> +}
> +
> +static void __init gen_pci_init_bus_range_child(struct dt_device_node *dev,
> + struct pci_host_bridge *bridge,
> + struct pci_config_window *cfg)
> +{
> + cfg->busn_start = bridge->cfg->busn_start + 1;
I read this as "the child bus number starts at the parent bus number
plus one". I'm not sure we want to hardcode this type of information in
a common file. Perhaps it would be better suited in pci-designware.c or
pci-host-rcar4.c. Could you pass the bus range init function pointer via
struct pci_ops?
> + cfg->busn_end = bridge->cfg->busn_end;
> + bridge->cfg->busn_end = bridge->cfg->busn_start;
> +
> + printk(XENLOG_INFO "Root bus end updated: [bus %x-%x]\n",
> + bridge->cfg->busn_start, bridge->cfg->busn_end);
> +}
> +
> +static struct pci_config_window *__init
> +gen_pci_init(struct dt_device_node *dev, struct pci_host_bridge *bridge,
> + const struct pci_ecam_ops *ops,
> + void (*init_bus_range)(struct dt_device_node *dev,
> + struct pci_host_bridge *bridge,
> + struct pci_config_window *cfg))
> +{
> + int err, cfg_reg_idx;
> + paddr_t addr, size;
> + struct pci_config_window *cfg;
> +
> + cfg = xzalloc(struct pci_config_window);
> + if ( !cfg )
> + return NULL;
> +
> + init_bus_range(dev, bridge, cfg);
>
> if ( ops->cfg_reg_index )
> {
> @@ -208,8 +233,10 @@ static int pci_bus_find_domain_nr(struct dt_device_node *dev)
> return domain;
> }
>
> -struct pci_host_bridge *pci_host_common_probe(struct dt_device_node *dev,
> - const struct pci_ecam_ops *ops)
> +struct pci_host_bridge *
> +pci_host_common_probe(struct dt_device_node *dev,
> + const struct pci_ecam_ops *ops,
> + const struct pci_ecam_ops *child_ops)
> {
> struct pci_host_bridge *bridge;
> struct pci_config_window *cfg;
> @@ -224,7 +251,7 @@ struct pci_host_bridge *pci_host_common_probe(struct dt_device_node *dev,
> return ERR_PTR(-ENOMEM);
>
> /* Parse and map our Configuration Space windows */
> - cfg = gen_pci_init(dev, ops);
> + cfg = gen_pci_init(dev, bridge, ops, gen_pci_init_bus_range);
> if ( !cfg )
> {
> err = -ENOMEM;
> @@ -242,10 +269,29 @@ struct pci_host_bridge *pci_host_common_probe(struct dt_device_node *dev,
> BUG();
> }
> bridge->segment = domain;
> +
> + if ( child_ops )
> + {
> + /* Parse and map child's Configuration Space windows */
> + cfg = gen_pci_init(dev, bridge, child_ops,
> + gen_pci_init_bus_range_child);
> + if ( !cfg )
> + {
> + err = -ENOMEM;
> + goto err_child;
> + }
> +
> + bridge->child_cfg = cfg;
> + bridge->child_ops = &child_ops->pci_ops;
> + }
> +
> pci_add_host_bridge(bridge);
>
> return bridge;
>
> +err_child:
> + xfree(bridge->cfg);
> +
> err_exit:
> xfree(bridge);
>
> @@ -280,9 +326,12 @@ struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus)
> {
> if ( bridge->segment != segment )
> continue;
> - if ( (bus < bridge->cfg->busn_start) || (bus > bridge->cfg->busn_end) )
> - continue;
> - return bridge;
> + if ( bridge->child_cfg && (bus >= bridge->child_cfg->busn_start) &&
> + (bus <= bridge->child_cfg->busn_end) )
> + return bridge;
> + if ( (bus >= bridge->cfg->busn_start) &&
> + (bus <= bridge->cfg->busn_end) )
> + return bridge;
> }
>
> return NULL;
> @@ -348,6 +397,7 @@ int __init pci_host_bridge_mappings(struct domain *d)
> {
> const struct dt_device_node *dev = bridge->dt_node;
> unsigned int i;
> + bool need_mapping;
>
> for ( i = 0; i < dt_number_of_address(dev); i++ )
> {
> @@ -363,7 +413,11 @@ int __init pci_host_bridge_mappings(struct domain *d)
> return err;
> }
>
> - if ( bridge->ops->need_p2m_hwdom_mapping(d, bridge, addr) )
> + need_mapping = bridge->ops->need_p2m_hwdom_mapping(d, bridge, addr);
> + if ( need_mapping || bridge->child_ops )
> + need_mapping =
> + bridge->child_ops->need_p2m_hwdom_mapping(d, bridge, addr);
You're dereferencing child_ops without NULL-checking it.
> + if ( need_mapping )
> {
> err = map_range_to_domain(dev, addr, size, &mr_data);
> if ( err )
> diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c
> index a6ffbda174..47cf144831 100644
> --- a/xen/arch/arm/pci/pci-host-generic.c
> +++ b/xen/arch/arm/pci/pci-host-generic.c
> @@ -29,7 +29,7 @@ static const struct dt_device_match __initconstrel gen_pci_dt_match[] =
> static int __init pci_host_generic_probe(struct dt_device_node *dev,
> const void *data)
> {
> - return PTR_RET(pci_host_common_probe(dev, &pci_generic_ecam_ops));
> + return PTR_RET(pci_host_common_probe(dev, &pci_generic_ecam_ops, NULL));
> }
>
> DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI_HOSTBRIDGE)
> diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c
> index a38f2e019e..75d1235aaf 100644
> --- a/xen/arch/arm/pci/pci-host-zynqmp.c
> +++ b/xen/arch/arm/pci/pci-host-zynqmp.c
> @@ -47,7 +47,7 @@ static const struct dt_device_match __initconstrel nwl_pcie_dt_match[] =
> static int __init pci_host_generic_probe(struct dt_device_node *dev,
> const void *data)
> {
> - return PTR_RET(pci_host_common_probe(dev, &nwl_pcie_ops));
> + return PTR_RET(pci_host_common_probe(dev, &nwl_pcie_ops, NULL));
> }
>
> DT_DEVICE_START(pci_gen, "PCI HOST ZYNQMP", DEVICE_PCI_HOSTBRIDGE)
> diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c
> index b63a356bb4..468aee1db7 100644
> --- a/xen/arch/arm/vpci.c
> +++ b/xen/arch/arm/vpci.c
> @@ -8,15 +8,17 @@
> #include <asm/mmio.h>
>
> static pci_sbdf_t vpci_sbdf_from_gpa(const struct pci_host_bridge *bridge,
> - paddr_t gpa)
> + paddr_t gpa, bool use_root)
> {
> pci_sbdf_t sbdf;
>
> if ( bridge )
> {
> - sbdf.sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr);
> + const struct pci_config_window *cfg = use_root ? bridge->cfg
> + : bridge->child_cfg;
> + sbdf.sbdf = VPCI_ECAM_BDF(gpa - cfg->phys_addr);
> sbdf.seg = bridge->segment;
> - sbdf.bus += bridge->cfg->busn_start;
> + sbdf.bus += cfg->busn_start;
> }
> else
> sbdf.sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE);
> @@ -24,11 +26,9 @@ static pci_sbdf_t vpci_sbdf_from_gpa(const struct pci_host_bridge *bridge,
> return sbdf;
> }
>
> -static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
> - register_t *r, void *p)
> +static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info, register_t *r,
> + pci_sbdf_t sbdf)
> {
> - struct pci_host_bridge *bridge = p;
> - pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
> const unsigned int access_size = (1U << info->dabt.size) * 8;
> const register_t invalid = GENMASK_ULL(access_size - 1, 0);
> /* data is needed to prevent a pointer cast on 32bit */
> @@ -46,31 +46,78 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info,
> return 0;
> }
>
> -static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info,
> - register_t r, void *p)
> +static int vpci_mmio_read_root(struct vcpu *v, mmio_info_t *info, register_t *r,
> + void *p)
> +{
> + struct pci_host_bridge *bridge = p;
> + pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa, true);
> +
> + return vpci_mmio_read(v, info, r, sbdf);
> +}
> +
> +static int vpci_mmio_read_child(struct vcpu *v, mmio_info_t *info,
> + register_t *r, void *p)
> {
> struct pci_host_bridge *bridge = p;
> - pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa);
> + pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa, false);
> +
> + return vpci_mmio_read(v, info, r, sbdf);
> +}
>
> +static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info, register_t r,
> + pci_sbdf_t sbdf)
> +{
> return vpci_ecam_write(sbdf, ECAM_REG_OFFSET(info->gpa),
> 1U << info->dabt.size, r);
> }
>
> +static int vpci_mmio_write_root(struct vcpu *v, mmio_info_t *info, register_t r,
> + void *p)
> +{
> + struct pci_host_bridge *bridge = p;
> + pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa, true);
> +
> + return vpci_mmio_write(v, info, r, sbdf);
> +}
> +
> +static int vpci_mmio_write_child(struct vcpu *v, mmio_info_t *info,
> + register_t r, void *p)
> +{
> + struct pci_host_bridge *bridge = p;
> + pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa, false);
> +
> + return vpci_mmio_write(v, info, r, sbdf);
> +}
> +
> static const struct mmio_handler_ops vpci_mmio_handler = {
> - .read = vpci_mmio_read,
> - .write = vpci_mmio_write,
> + .read = vpci_mmio_read_root,
> + .write = vpci_mmio_write_root,
> +};
> +
> +static const struct mmio_handler_ops vpci_mmio_handler_child = {
> + .read = vpci_mmio_read_child,
> + .write = vpci_mmio_write_child,
> };
>
> static int vpci_setup_mmio_handler_cb(struct domain *d,
> struct pci_host_bridge *bridge)
> {
> struct pci_config_window *cfg = bridge->cfg;
> + int count = 1;
>
> register_mmio_handler(d, &vpci_mmio_handler,
> cfg->phys_addr, cfg->size, bridge);
>
> - /* We have registered a single MMIO handler. */
> - return 1;
> + if ( bridge->child_ops )
> + {
> + struct pci_config_window *cfg = bridge->child_cfg;
This variable shadows the cfg variable of the same name at the top of
the function.
> +
> + register_mmio_handler(d, &vpci_mmio_handler_child, cfg->phys_addr,
> + cfg->size, bridge);
> + count++;
> + }
> +
> + return count;
> }
>
> int domain_vpci_init(struct domain *d)
> @@ -101,8 +148,12 @@ int domain_vpci_init(struct domain *d)
> static int vpci_get_num_handlers_cb(struct domain *d,
> struct pci_host_bridge *bridge)
> {
> - /* Each bridge has a single MMIO handler for the configuration space. */
> - return 1;
> + int count = 1;
> +
> + if ( bridge->child_cfg )
> + count++;
> +
> + return count;
> }
>
> unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d)
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH v4 3/4] xen/arm: add support for PCI child bus
2025-04-23 11:08 ` [PATCH v4 3/4] xen/arm: add support for PCI child bus Mykyta Poturai
2025-04-23 18:34 ` Stewart Hildebrand
@ 2025-04-23 20:59 ` Stefano Stabellini
2025-04-24 15:16 ` Stewart Hildebrand
2 siblings, 0 replies; 12+ messages in thread
From: Stefano Stabellini @ 2025-04-23 20:59 UTC (permalink / raw)
To: Mykyta Poturai
Cc: xen-devel@lists.xenproject.org, Oleksandr Andrushchenko,
Stefano Stabellini, Julien Grall, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk
On Wed, 23 Apr 2025, Mykyta Poturai wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>
> PCI host bridges often have different ways to access the root and child
> bus configuration spaces. One of the examples is Designware's host bridge
> and its multiple clones [1].
>
> Linux kernel implements this by instantiating a child bus when device
> drivers provide not only the usual pci_ops to access ECAM space (this is
> the case for the generic host bridge), but also means to access the child
> bus which has a dedicated configuration space and own implementation for
> accessing the bus, e.g. child_ops.
>
> For Xen it is not feasible to fully implement PCI bus infrastructure as
> Linux kernel does, but still child bus can be supported.
>
> Add support for the PCI child bus which includes the following changes:
> - introduce bus mapping functions depending on SBDF
> - assign bus start and end for the child bus and re-configure the same for
> the parent (root) bus
> - make pci_find_host_bridge be aware of multiple busses behind the same bridge
> - update pci_host_bridge_mappings, so it also doesn't map to guest the memory
> spaces belonging to the child bus
> - make pci_host_common_probe accept one more pci_ops structure for the child bus
> - install MMIO handlers for the child bus
> - re-work vpci_mmio_{write|read} with parent and child approach in mind
>
> [1] https://elixir.bootlin.com/linux/v5.15/source/drivers/pci/controller/dwc
>
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
I just wanted to say that pending Stewart's comment, this patch looks
good to me
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH v4 3/4] xen/arm: add support for PCI child bus
2025-04-23 11:08 ` [PATCH v4 3/4] xen/arm: add support for PCI child bus Mykyta Poturai
2025-04-23 18:34 ` Stewart Hildebrand
2025-04-23 20:59 ` Stefano Stabellini
@ 2025-04-24 15:16 ` Stewart Hildebrand
2 siblings, 0 replies; 12+ messages in thread
From: Stewart Hildebrand @ 2025-04-24 15:16 UTC (permalink / raw)
To: Mykyta Poturai, xen-devel@lists.xenproject.org
Cc: Oleksandr Andrushchenko, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk
On 4/23/25 07:08, Mykyta Poturai wrote:
> diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c
> index 9f9aac43d7..91271973fb 100644
> --- a/xen/arch/arm/pci/pci-access.c
> +++ b/xen/arch/arm/pci/pci-access.c
> @@ -18,10 +18,31 @@
> #define INVALID_VALUE (~0U)
> #define PCI_ERR_VALUE(len) GENMASK(0, len * 8)
>
> +static const struct pci_ops *get_ops(struct pci_host_bridge *bridge,
> + pci_sbdf_t sbdf)
> +{
> + if ( bridge->child_ops )
> + {
> + struct pci_config_window *cfg = bridge->child_cfg;
> +
> + if ( (sbdf.bus >= cfg->busn_start) && (sbdf.bus <= cfg->busn_end) )
> + return bridge->child_ops;
> + }
> + return bridge->ops;
> +}
> +
> +static inline void __iomem *map_bus(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
Line too long
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v4 2/4] xen/arm: make pci_host_common_probe return the bridge
2025-04-23 11:08 [PATCH v4 0/4] Add support for R-Car Gen4 PCI host controller Mykyta Poturai
2025-04-23 11:08 ` [PATCH v4 1/4] xen/arm: allow PCI host bridge to have private data Mykyta Poturai
2025-04-23 11:08 ` [PATCH v4 3/4] xen/arm: add support for PCI child bus Mykyta Poturai
@ 2025-04-23 11:08 ` Mykyta Poturai
2025-04-23 20:56 ` Stefano Stabellini
2025-04-23 11:08 ` [PATCH v4 4/4] xen/arm: add support for R-Car Gen4 PCI host controller Mykyta Poturai
3 siblings, 1 reply; 12+ messages in thread
From: Mykyta Poturai @ 2025-04-23 11:08 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Andrushchenko, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk, Mykyta Poturai
From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Some of the PCI host bridges require additional processing during the
probe phase. For that they need to access struct bridge of the probed
host, so return pointer to the new bridge from pci_host_common_probe.
Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
---
v3->v4:
* change return 0 to return NULL
v2->v3:
* no change
v1->v2:
* no change
---
xen/arch/arm/include/asm/pci.h | 5 +++--
xen/arch/arm/pci/pci-host-common.c | 12 ++++++------
xen/arch/arm/pci/pci-host-generic.c | 2 +-
xen/arch/arm/pci/pci-host-zynqmp.c | 2 +-
4 files changed, 11 insertions(+), 10 deletions(-)
diff --git a/xen/arch/arm/include/asm/pci.h b/xen/arch/arm/include/asm/pci.h
index a87672d834..3d2ca9b5b0 100644
--- a/xen/arch/arm/include/asm/pci.h
+++ b/xen/arch/arm/include/asm/pci.h
@@ -18,6 +18,7 @@
#ifdef CONFIG_HAS_PCI
#include <asm/p2m.h>
+#include <xen/err.h>
#define pci_to_dev(pcidev) (&(pcidev)->arch.dev)
@@ -95,8 +96,8 @@ struct pci_ecam_ops {
/* Default ECAM ops */
extern const struct pci_ecam_ops pci_generic_ecam_ops;
-int pci_host_common_probe(struct dt_device_node *dev,
- const struct pci_ecam_ops *ops);
+struct pci_host_bridge *pci_host_common_probe(struct dt_device_node *dev,
+ const struct pci_ecam_ops *ops);
int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
uint32_t reg, uint32_t len, uint32_t *value);
int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index c0faf0f436..53953d4895 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -208,8 +208,8 @@ static int pci_bus_find_domain_nr(struct dt_device_node *dev)
return domain;
}
-int pci_host_common_probe(struct dt_device_node *dev,
- const struct pci_ecam_ops *ops)
+struct pci_host_bridge *pci_host_common_probe(struct dt_device_node *dev,
+ const struct pci_ecam_ops *ops)
{
struct pci_host_bridge *bridge;
struct pci_config_window *cfg;
@@ -217,11 +217,11 @@ int pci_host_common_probe(struct dt_device_node *dev,
int domain;
if ( dt_device_for_passthrough(dev) )
- return 0;
+ return NULL;
bridge = pci_alloc_host_bridge();
if ( !bridge )
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
/* Parse and map our Configuration Space windows */
cfg = gen_pci_init(dev, ops);
@@ -244,12 +244,12 @@ int pci_host_common_probe(struct dt_device_node *dev,
bridge->segment = domain;
pci_add_host_bridge(bridge);
- return 0;
+ return bridge;
err_exit:
xfree(bridge);
- return err;
+ return ERR_PTR(err);
}
/*
diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c
index 46de6e43cc..a6ffbda174 100644
--- a/xen/arch/arm/pci/pci-host-generic.c
+++ b/xen/arch/arm/pci/pci-host-generic.c
@@ -29,7 +29,7 @@ static const struct dt_device_match __initconstrel gen_pci_dt_match[] =
static int __init pci_host_generic_probe(struct dt_device_node *dev,
const void *data)
{
- return pci_host_common_probe(dev, &pci_generic_ecam_ops);
+ return PTR_RET(pci_host_common_probe(dev, &pci_generic_ecam_ops));
}
DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI_HOSTBRIDGE)
diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c
index 101edb8593..a38f2e019e 100644
--- a/xen/arch/arm/pci/pci-host-zynqmp.c
+++ b/xen/arch/arm/pci/pci-host-zynqmp.c
@@ -47,7 +47,7 @@ static const struct dt_device_match __initconstrel nwl_pcie_dt_match[] =
static int __init pci_host_generic_probe(struct dt_device_node *dev,
const void *data)
{
- return pci_host_common_probe(dev, &nwl_pcie_ops);
+ return PTR_RET(pci_host_common_probe(dev, &nwl_pcie_ops));
}
DT_DEVICE_START(pci_gen, "PCI HOST ZYNQMP", DEVICE_PCI_HOSTBRIDGE)
--
2.34.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PATCH v4 2/4] xen/arm: make pci_host_common_probe return the bridge
2025-04-23 11:08 ` [PATCH v4 2/4] xen/arm: make pci_host_common_probe return the bridge Mykyta Poturai
@ 2025-04-23 20:56 ` Stefano Stabellini
0 siblings, 0 replies; 12+ messages in thread
From: Stefano Stabellini @ 2025-04-23 20:56 UTC (permalink / raw)
To: Mykyta Poturai
Cc: xen-devel@lists.xenproject.org, Oleksandr Andrushchenko,
Stefano Stabellini, Julien Grall, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk
On Wed, 23 Apr 2025, Mykyta Poturai wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>
> Some of the PCI host bridges require additional processing during the
> probe phase. For that they need to access struct bridge of the probed
> host, so return pointer to the new bridge from pci_host_common_probe.
>
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH v4 4/4] xen/arm: add support for R-Car Gen4 PCI host controller
2025-04-23 11:08 [PATCH v4 0/4] Add support for R-Car Gen4 PCI host controller Mykyta Poturai
` (2 preceding siblings ...)
2025-04-23 11:08 ` [PATCH v4 2/4] xen/arm: make pci_host_common_probe return the bridge Mykyta Poturai
@ 2025-04-23 11:08 ` Mykyta Poturai
2025-04-23 21:06 ` Stefano Stabellini
2025-04-24 19:22 ` Stewart Hildebrand
3 siblings, 2 replies; 12+ messages in thread
From: Mykyta Poturai @ 2025-04-23 11:08 UTC (permalink / raw)
To: xen-devel@lists.xenproject.org
Cc: Oleksandr Andrushchenko, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk, Mykyta Poturai
From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Add support for Renesas R-Car Gen4 PCI host controller, specifically
targeting the S4 and V4H SoCs. The implementation includes configuration
read/write operations for both root and child buses. For accessing the
child bus, iATU is used for address translation.
Code common to all DesignWare PCI host controllers is located in a
separate file to allow for easy reuse in other DesignWare-based PCI
host controllers.
Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
---
v3->v4:
* no changes
v2->v3:
* move priv allocation to dw_pcie_host_probe
v1->v2:
* move designware code in a separate file
---
xen/arch/arm/pci/Makefile | 2 +
xen/arch/arm/pci/pci-designware.c | 403 ++++++++++++++++++++++++++++++
xen/arch/arm/pci/pci-designware.h | 102 ++++++++
xen/arch/arm/pci/pci-host-rcar4.c | 103 ++++++++
4 files changed, 610 insertions(+)
create mode 100644 xen/arch/arm/pci/pci-designware.c
create mode 100644 xen/arch/arm/pci/pci-designware.h
create mode 100644 xen/arch/arm/pci/pci-host-rcar4.c
diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
index 1d045ade01..ca6135e282 100644
--- a/xen/arch/arm/pci/Makefile
+++ b/xen/arch/arm/pci/Makefile
@@ -4,3 +4,5 @@ obj-y += pci-host-generic.o
obj-y += pci-host-common.o
obj-y += ecam.o
obj-y += pci-host-zynqmp.o
+obj-y += pci-designware.o
+obj-y += pci-host-rcar4.o
diff --git a/xen/arch/arm/pci/pci-designware.c b/xen/arch/arm/pci/pci-designware.c
new file mode 100644
index 0000000000..dcbd07ced3
--- /dev/null
+++ b/xen/arch/arm/pci/pci-designware.c
@@ -0,0 +1,403 @@
+/*
+ * Based on Linux drivers/pci/controller/pci-host-common.c
+ * Based on Linux drivers/pci/controller/pci-host-generic.c
+ * Based on xen/arch/arm/pci/pci-host-generic.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/delay.h>
+#include <asm/io.h>
+
+#include "pci-designware.h"
+/**
+ * upper_32_bits - return bits 32-63 of a number
+ * @n: the number we're accessing
+ *
+ * A basic shift-right of a 64- or 32-bit quantity. Use this to suppress
+ * the "right shift count >= width of type" warning when that quantity is
+ * 32-bits.
+ */
+#define upper_32_bits(n) ((uint32_t)(((n) >> 16) >> 16))
+
+/**
+ * lower_32_bits - return bits 0-31 of a number
+ * @n: the number we're accessing
+ */
+#define lower_32_bits(n) ((uint32_t)((n) & 0xffffffff))
+
+static int dw_pcie_read(void __iomem *addr, int size, uint32_t *val)
+{
+ if ( !IS_ALIGNED((uintptr_t)addr, size) )
+ {
+ *val = 0;
+ return PCIBIOS_BAD_REGISTER_NUMBER;
+ }
+
+ if ( size == 4 )
+ *val = readl(addr);
+ else if ( size == 2 )
+ *val = readw(addr);
+ else if ( size == 1 )
+ *val = readb(addr);
+ else
+ {
+ *val = 0;
+ return PCIBIOS_BAD_REGISTER_NUMBER;
+ }
+
+ return PCIBIOS_SUCCESSFUL;
+}
+
+static int dw_pcie_write(void __iomem *addr, int size, uint32_t val)
+{
+ if ( !IS_ALIGNED((uintptr_t)addr, size) )
+ return PCIBIOS_BAD_REGISTER_NUMBER;
+
+ if ( size == 4 )
+ writel(val, addr);
+ else if ( size == 2 )
+ writew(val, addr);
+ else if ( size == 1 )
+ writeb(val, addr);
+ else
+ return PCIBIOS_BAD_REGISTER_NUMBER;
+
+ return PCIBIOS_SUCCESSFUL;
+}
+
+static uint32_t dw_pcie_read_dbi(struct pci_host_bridge *bridge, uint32_t reg,
+ size_t size)
+{
+ void __iomem *addr = bridge->cfg->win + reg;
+ uint32_t val;
+
+ dw_pcie_read(addr, size, &val);
+ return val;
+}
+
+static void dw_pcie_write_dbi(struct pci_host_bridge *bridge, uint32_t reg,
+ size_t size, uint32_t val)
+{
+ void __iomem *addr = bridge->cfg->win + reg;
+
+ dw_pcie_write(addr, size, val);
+}
+
+static uint32_t dw_pcie_readl_dbi(struct pci_host_bridge *bridge, uint32_t reg)
+{
+ return dw_pcie_read_dbi(bridge, reg, sizeof(uint32_t));
+}
+
+static void dw_pcie_writel_dbi(struct pci_host_bridge *pci, uint32_t reg,
+ uint32_t val)
+{
+ dw_pcie_write_dbi(pci, reg, sizeof(uint32_t), val);
+}
+
+static void dw_pcie_read_iatu_unroll_enabled(struct pci_host_bridge *bridge)
+{
+ struct dw_pcie_priv *priv = bridge->priv;
+ uint32_t val;
+
+ val = dw_pcie_readl_dbi(bridge, PCIE_ATU_VIEWPORT);
+ if ( val == 0xffffffff )
+ priv->iatu_unroll_enabled = true;
+
+ printk(XENLOG_DEBUG "%s iATU unroll: %sabled\n",
+ dt_node_full_name(bridge->dt_node),
+ priv->iatu_unroll_enabled ? "en" : "dis");
+}
+
+static uint32_t dw_pcie_readl_atu(struct pci_host_bridge *pci, uint32_t reg)
+{
+ struct dw_pcie_priv *priv = pci->priv;
+ int ret;
+ uint32_t val;
+
+ ret = dw_pcie_read(priv->atu_base + reg, 4, &val);
+ if ( ret )
+ printk(XENLOG_ERR "Read ATU address failed\n");
+
+ return val;
+}
+
+static void dw_pcie_writel_atu(struct pci_host_bridge *pci, uint32_t reg,
+ uint32_t val)
+{
+ struct dw_pcie_priv *priv = pci->priv;
+ int ret;
+
+ ret = dw_pcie_write(priv->atu_base + reg, 4, val);
+ if ( ret )
+ printk(XENLOG_ERR "Write ATU address failed\n");
+}
+
+static uint32_t dw_pcie_readl_ob_unroll(struct pci_host_bridge *pci,
+ uint32_t index, uint32_t reg)
+{
+ uint32_t offset = PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(index);
+
+ return dw_pcie_readl_atu(pci, offset + reg);
+}
+
+static void dw_pcie_writel_ob_unroll(struct pci_host_bridge *pci,
+ uint32_t index, uint32_t reg, uint32_t val)
+{
+ uint32_t offset = PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(index);
+
+ dw_pcie_writel_atu(pci, offset + reg, val);
+}
+
+static uint32_t dw_pcie_enable_ecrc(uint32_t val)
+{
+ ASSERT_UNREACHABLE();
+ return 0;
+}
+
+static void dw_pcie_prog_outbound_atu_unroll(struct pci_host_bridge *pci,
+ uint8_t func_no, int index,
+ int type, uint64_t cpu_addr,
+ uint64_t pci_addr, uint64_t size)
+{
+ struct dw_pcie_priv *priv = pci->priv;
+ uint32_t retries, val;
+ uint64_t limit_addr = cpu_addr + size - 1;
+
+ dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_BASE,
+ lower_32_bits(cpu_addr));
+ dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_BASE,
+ upper_32_bits(cpu_addr));
+ dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_LIMIT,
+ lower_32_bits(limit_addr));
+ dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_LIMIT,
+ upper_32_bits(limit_addr));
+ dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_TARGET,
+ lower_32_bits(pci_addr));
+ dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
+ upper_32_bits(pci_addr));
+ val = type | PCIE_ATU_FUNC_NUM(func_no);
+ val = upper_32_bits(size - 1) ? val | PCIE_ATU_INCREASE_REGION_SIZE : val;
+ if ( priv->version == 0x490A )
+ val = dw_pcie_enable_ecrc(val);
+ dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, val);
+ dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
+ PCIE_ATU_ENABLE);
+
+ /*
+ * Make sure ATU enable takes effect before any subsequent config
+ * and I/O accesses.
+ */
+ for ( retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++ )
+ {
+ val = dw_pcie_readl_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2);
+ if ( val & PCIE_ATU_ENABLE )
+ return;
+
+ mdelay(LINK_WAIT_IATU);
+ }
+ printk(XENLOG_ERR "Outbound iATU is not being enabled\n");
+}
+
+static void __dw_pcie_prog_outbound_atu(struct pci_host_bridge *pci,
+ uint8_t func_no, int index, int type,
+ uint64_t cpu_addr, uint64_t pci_addr,
+ uint64_t size)
+{
+ struct dw_pcie_priv *priv = pci->priv;
+ uint32_t retries, val;
+
+ if ( priv->iatu_unroll_enabled )
+ {
+ dw_pcie_prog_outbound_atu_unroll(pci, func_no, index, type, cpu_addr,
+ pci_addr, size);
+ return;
+ }
+
+ dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT,
+ PCIE_ATU_REGION_OUTBOUND | index);
+ dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_BASE, lower_32_bits(cpu_addr));
+ dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_BASE, upper_32_bits(cpu_addr));
+ dw_pcie_writel_dbi(pci, PCIE_ATU_LIMIT, lower_32_bits(cpu_addr + size - 1));
+ if ( priv->version >= 0x460A )
+ dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_LIMIT,
+ upper_32_bits(cpu_addr + size - 1));
+ dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, lower_32_bits(pci_addr));
+ dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET, upper_32_bits(pci_addr));
+ val = type | PCIE_ATU_FUNC_NUM(func_no);
+ val = ((upper_32_bits(size - 1)) && (priv->version >= 0x460A))
+ ? val | PCIE_ATU_INCREASE_REGION_SIZE
+ : val;
+ if ( priv->version == 0x490A )
+ val = dw_pcie_enable_ecrc(val);
+ dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, val);
+ dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE);
+
+ /*
+ * Make sure ATU enable takes effect before any subsequent config
+ * and I/O accesses.
+ */
+ for ( retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++ )
+ {
+ val = dw_pcie_readl_dbi(pci, PCIE_ATU_CR2);
+ if ( val & PCIE_ATU_ENABLE )
+ return;
+
+ mdelay(LINK_WAIT_IATU);
+ }
+ printk(XENLOG_ERR "Outbound iATU is not being enabled\n");
+}
+
+static void dw_pcie_prog_outbound_atu(struct pci_host_bridge *pci, int index,
+ int type, uint64_t cpu_addr,
+ uint64_t pci_addr, uint64_t size)
+{
+ __dw_pcie_prog_outbound_atu(pci, 0, index, type, cpu_addr, pci_addr, size);
+}
+
+void dw_pcie_set_version(struct pci_host_bridge *bridge, unsigned int version)
+{
+ struct dw_pcie_priv *priv = bridge->priv;
+
+ priv->version = version;
+}
+
+void __iomem *dw_pcie_child_map_bus(struct pci_host_bridge *bridge,
+ pci_sbdf_t sbdf, uint32_t where)
+{
+ uint32_t busdev;
+
+ busdev = PCIE_ATU_BUS(sbdf.bus) | PCIE_ATU_DEV(PCI_SLOT(sbdf.devfn)) |
+ PCIE_ATU_FUNC(PCI_FUNC(sbdf.devfn));
+
+ /* FIXME: Parent is the root bus, so use PCIE_ATU_TYPE_CFG0. */
+ dw_pcie_prog_outbound_atu(bridge, PCIE_ATU_REGION_INDEX1,
+ PCIE_ATU_TYPE_CFG0, bridge->child_cfg->phys_addr,
+ busdev, bridge->child_cfg->size);
+
+ return bridge->child_cfg->win + where;
+}
+
+int dw_pcie_child_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
+ uint32_t reg, uint32_t len, uint32_t *value)
+{
+ struct dw_pcie_priv *priv = bridge->priv;
+ int ret;
+
+ /*
+ * FIXME: we cannot read iATU settings at the early initialization
+ * (probe) as the host's HW is not yet initialized at that phase.
+ * This read operation is the very first thing Domain-0 will do
+ * during its initialization, so take this opportunity and read
+ * iATU setting now.
+ */
+ if ( unlikely(!priv->iatu_unroll_initilized) )
+ {
+ dw_pcie_read_iatu_unroll_enabled(bridge);
+ priv->iatu_unroll_initilized = true;
+ }
+
+ ret = pci_generic_config_read(bridge, sbdf, reg, len, value);
+ if ( !ret && (priv->num_viewport <= 2) )
+ dw_pcie_prog_outbound_atu(bridge, PCIE_ATU_REGION_INDEX1,
+ PCIE_ATU_TYPE_IO,
+ bridge->child_cfg->phys_addr, 0,
+ bridge->child_cfg->size);
+
+ return ret;
+}
+
+int dw_pcie_child_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
+ uint32_t reg, uint32_t len, uint32_t value)
+{
+ struct dw_pcie_priv *priv = bridge->priv;
+ int ret;
+
+ ret = pci_generic_config_write(bridge, sbdf, reg, len, value);
+ if ( !ret && (priv->num_viewport <= 2) )
+ dw_pcie_prog_outbound_atu(bridge, PCIE_ATU_REGION_INDEX1,
+ PCIE_ATU_TYPE_IO,
+ bridge->child_cfg->phys_addr, 0,
+ bridge->child_cfg->size);
+ return ret;
+}
+
+bool __init dw_pcie_child_need_p2m_hwdom_mapping(struct domain *d,
+ struct pci_host_bridge *bridge,
+ uint64_t addr)
+{
+ struct pci_config_window *cfg = bridge->child_cfg;
+
+ /*
+ * We do not want ECAM address space to be mapped in Domain-0's p2m,
+ * so we can trap access to it.
+ */
+ return cfg->phys_addr != addr;
+}
+
+struct pci_host_bridge *__init
+dw_pcie_host_probe(struct dt_device_node *dev, const void *data,
+ const struct pci_ecam_ops *ops,
+ const struct pci_ecam_ops *child_ops)
+{
+ struct pci_host_bridge *bridge;
+ struct dw_pcie_priv *priv;
+
+ paddr_t atu_phys_addr;
+ paddr_t atu_size;
+ int atu_idx, ret;
+
+ bridge = pci_host_common_probe(dev, ops, child_ops);
+ if ( IS_ERR(bridge) )
+ return bridge;
+
+ priv = xzalloc_bytes(sizeof(struct dw_pcie_priv));
+ if ( !priv )
+ return ERR_PTR(-ENOMEM);
+
+ bridge->priv = priv;
+
+ atu_idx = dt_property_match_string(dev, "reg-names", "atu");
+ if ( atu_idx < 0 )
+ {
+ printk(XENLOG_ERR "Cannot find \"atu\" range index in device tree\n");
+ return ERR_PTR(atu_idx);
+ }
+ ret = dt_device_get_address(dev, atu_idx, &atu_phys_addr, &atu_size);
+ if ( ret )
+ {
+ printk(XENLOG_ERR "Cannot find \"atu\" range in device tree\n");
+ return ERR_PTR(ret);
+ }
+ printk("iATU at [mem 0x%" PRIpaddr "-0x%" PRIpaddr "]\n", atu_phys_addr,
+ atu_phys_addr + atu_size - 1);
+ priv->atu_base = ioremap_nocache(atu_phys_addr, atu_size);
+ if ( !priv->atu_base )
+ {
+ printk(XENLOG_ERR "iATU ioremap failed\n");
+ return ERR_PTR(ENXIO);
+ }
+
+ if ( !dt_property_read_u32(dev, "num-viewport", &priv->num_viewport) )
+ priv->num_viewport = 2;
+
+ /*
+ * FIXME: we cannot read iATU unroll enable now as the host bridge's
+ * HW is not yet initialized by Domain-0: leave it for later.
+ */
+
+ printk(XENLOG_INFO "%s number of view ports: %d\n", dt_node_full_name(dev),
+ priv->num_viewport);
+
+ return bridge;
+}
diff --git a/xen/arch/arm/pci/pci-designware.h b/xen/arch/arm/pci/pci-designware.h
new file mode 100644
index 0000000000..a15ba9a329
--- /dev/null
+++ b/xen/arch/arm/pci/pci-designware.h
@@ -0,0 +1,102 @@
+/*
+ * Based on Linux drivers/pci/controller/pci-host-common.c
+ * Based on Linux drivers/pci/controller/pci-host-generic.c
+ * Based on xen/arch/arm/pci/pci-host-generic.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/pci.h>
+#include <xen/init.h>
+
+#ifndef __PCI_DESIGNWARE_H__
+#define __PCI_DESIGNWARE_H__
+
+#define PCIBIOS_SUCCESSFUL 0x00
+#define PCIBIOS_BAD_REGISTER_NUMBER 0x87
+
+#define PCIE_ATU_VIEWPORT 0x900
+#define PCIE_ATU_REGION_OUTBOUND 0
+#define PCIE_ATU_CR1 0x904
+#define PCIE_ATU_INCREASE_REGION_SIZE BIT(13, UL)
+#define PCIE_ATU_CR2 0x908
+#define PCIE_ATU_ENABLE BIT(31, UL)
+#define PCIE_ATU_LOWER_BASE 0x90C
+#define PCIE_ATU_UPPER_BASE 0x910
+#define PCIE_ATU_LIMIT 0x914
+#define PCIE_ATU_LOWER_TARGET 0x918
+#define PCIE_ATU_UPPER_TARGET 0x91C
+#define PCIE_ATU_UPPER_LIMIT 0x924
+
+#define PCIE_ATU_REGION_INDEX1 0x1
+#define PCIE_ATU_TYPE_IO 0x2
+#define PCIE_ATU_TYPE_CFG0 0x4
+
+#define FIELD_PREP(_mask, _val) \
+ (((typeof(_mask))(_val) << (ffs64(_mask) - 1)) & (_mask))
+
+#define PCIE_ATU_BUS(x) FIELD_PREP(GENMASK(31, 24), x)
+#define PCIE_ATU_DEV(x) FIELD_PREP(GENMASK(23, 19), x)
+#define PCIE_ATU_FUNC(x) FIELD_PREP(GENMASK(18, 16), x)
+
+/* Register address builder */
+#define PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(region) \
+ ((region) << 9)
+
+/*
+ * iATU Unroll-specific register definitions
+ * From 4.80 core version the address translation will be made by unroll
+ */
+#define PCIE_ATU_UNR_REGION_CTRL1 0x00
+#define PCIE_ATU_UNR_REGION_CTRL2 0x04
+#define PCIE_ATU_UNR_LOWER_BASE 0x08
+#define PCIE_ATU_UNR_UPPER_BASE 0x0C
+#define PCIE_ATU_UNR_LOWER_LIMIT 0x10
+#define PCIE_ATU_UNR_LOWER_TARGET 0x14
+#define PCIE_ATU_UNR_UPPER_TARGET 0x18
+#define PCIE_ATU_UNR_UPPER_LIMIT 0x20
+
+#define PCIE_ATU_FUNC_NUM(pf) ((pf) << 20)
+
+/* Parameters for the waiting for iATU enabled routine */
+#define LINK_WAIT_MAX_IATU_RETRIES 5
+#define LINK_WAIT_IATU 9
+
+struct dw_pcie_priv {
+ uint32_t num_viewport;
+ bool iatu_unroll_initilized;
+ bool iatu_unroll_enabled;
+ void __iomem *atu_base;
+ unsigned int version;
+};
+
+void dw_pcie_set_version(struct pci_host_bridge *bridge, unsigned int version);
+
+void __iomem *dw_pcie_child_map_bus(struct pci_host_bridge *bridge,
+ pci_sbdf_t sbdf, uint32_t where);
+
+int dw_pcie_child_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
+ uint32_t reg, uint32_t len, uint32_t *value);
+
+int dw_pcie_child_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
+ uint32_t reg, uint32_t len, uint32_t value);
+
+bool __init dw_pcie_child_need_p2m_hwdom_mapping(struct domain *d,
+ struct pci_host_bridge *bridge,
+ uint64_t addr);
+
+struct pci_host_bridge *__init
+dw_pcie_host_probe(struct dt_device_node *dev, const void *data,
+ const struct pci_ecam_ops *ops,
+ const struct pci_ecam_ops *child_ops);
+#endif /* __PCI_DESIGNWARE_H__ */
diff --git a/xen/arch/arm/pci/pci-host-rcar4.c b/xen/arch/arm/pci/pci-host-rcar4.c
new file mode 100644
index 0000000000..cae1b4254d
--- /dev/null
+++ b/xen/arch/arm/pci/pci-host-rcar4.c
@@ -0,0 +1,103 @@
+/*
+ * Based on Linux drivers/pci/controller/pci-host-common.c
+ * Based on Linux drivers/pci/controller/pci-host-generic.c
+ * Based on xen/arch/arm/pci/pci-host-generic.c
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/init.h>
+#include <xen/pci.h>
+
+#include <asm/device.h>
+#include <asm/io.h>
+#include <asm/pci.h>
+
+#include "pci-designware.h"
+
+#define RCAR4_DWC_VERSION 0x520A
+
+/*
+ * PCI host bridges often have different ways to access the root and child
+ * bus config spaces:
+ * "dbi" : the aperture where root port's own configuration registers
+ * are available.
+ * "config": child's configuration space
+ * "atu" : iATU registers for DWC version 4.80 or later
+ */
+static int __init rcar4_cfg_reg_index(struct dt_device_node *np)
+{
+ return dt_property_match_string(np, "reg-names", "dbi");
+}
+
+static int __init rcar4_child_cfg_reg_index(struct dt_device_node *np)
+{
+ return dt_property_match_string(np, "reg-names", "config");
+}
+
+/* ECAM ops */
+const struct pci_ecam_ops rcar4_pcie_ops = {
+ .bus_shift = 20,
+ .cfg_reg_index = rcar4_cfg_reg_index,
+ .pci_ops = {
+ .map_bus = pci_ecam_map_bus,
+ .read = pci_generic_config_read,
+ .write = pci_generic_config_write,
+ .need_p2m_hwdom_mapping = pci_ecam_need_p2m_hwdom_mapping,
+ }
+};
+
+const struct pci_ecam_ops rcar4_pcie_child_ops = {
+ .bus_shift = 20,
+ .cfg_reg_index = rcar4_child_cfg_reg_index,
+ .pci_ops = {
+ .map_bus = dw_pcie_child_map_bus,
+ .read = dw_pcie_child_config_read,
+ .write = dw_pcie_child_config_write,
+ .need_p2m_hwdom_mapping = dw_pcie_child_need_p2m_hwdom_mapping,
+ }
+};
+
+static const struct dt_device_match __initconstrel rcar4_pcie_dt_match[] = {
+ { .compatible = "renesas,r8a779f0-pcie" },
+ { .compatible = "renesas,r8a779g0-pcie" },
+ {},
+};
+
+static int __init pci_host_generic_probe(struct dt_device_node *dev,
+ const void *data)
+{
+ struct pci_host_bridge *bridge;
+
+ bridge = dw_pcie_host_probe(dev, data, &rcar4_pcie_ops,
+ &rcar4_pcie_child_ops);
+
+ dw_pcie_set_version(bridge, RCAR4_DWC_VERSION);
+
+ return 0;
+}
+
+DT_DEVICE_START(pci_gen, "PCI HOST R-CAR GEN4", DEVICE_PCI_HOSTBRIDGE)
+.dt_match = rcar4_pcie_dt_match,
+.init = pci_host_generic_probe,
+DT_DEVICE_END
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
--
2.34.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PATCH v4 4/4] xen/arm: add support for R-Car Gen4 PCI host controller
2025-04-23 11:08 ` [PATCH v4 4/4] xen/arm: add support for R-Car Gen4 PCI host controller Mykyta Poturai
@ 2025-04-23 21:06 ` Stefano Stabellini
2025-04-24 19:22 ` Stewart Hildebrand
1 sibling, 0 replies; 12+ messages in thread
From: Stefano Stabellini @ 2025-04-23 21:06 UTC (permalink / raw)
To: Mykyta Poturai
Cc: xen-devel@lists.xenproject.org, Oleksandr Andrushchenko,
Stefano Stabellini, Julien Grall, Bertrand Marquis, Michal Orzel,
Volodymyr Babchuk
On Wed, 23 Apr 2025, Mykyta Poturai wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>
> Add support for Renesas R-Car Gen4 PCI host controller, specifically
> targeting the S4 and V4H SoCs. The implementation includes configuration
> read/write operations for both root and child buses. For accessing the
> child bus, iATU is used for address translation.
>
> Code common to all DesignWare PCI host controllers is located in a
> separate file to allow for easy reuse in other DesignWare-based PCI
> host controllers.
>
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
At a glance, this looks OKish. Especially if you would be willing to
maintain the driver going forward :-)
Would you be willing to add entry to the MAINTAINERS file to cover
pci-designware* and pci-host-rcar4.c ?
> ---
> v3->v4:
> * no changes
>
> v2->v3:
> * move priv allocation to dw_pcie_host_probe
>
> v1->v2:
> * move designware code in a separate file
> ---
> xen/arch/arm/pci/Makefile | 2 +
> xen/arch/arm/pci/pci-designware.c | 403 ++++++++++++++++++++++++++++++
> xen/arch/arm/pci/pci-designware.h | 102 ++++++++
> xen/arch/arm/pci/pci-host-rcar4.c | 103 ++++++++
> 4 files changed, 610 insertions(+)
> create mode 100644 xen/arch/arm/pci/pci-designware.c
> create mode 100644 xen/arch/arm/pci/pci-designware.h
> create mode 100644 xen/arch/arm/pci/pci-host-rcar4.c
>
> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> index 1d045ade01..ca6135e282 100644
> --- a/xen/arch/arm/pci/Makefile
> +++ b/xen/arch/arm/pci/Makefile
> @@ -4,3 +4,5 @@ obj-y += pci-host-generic.o
> obj-y += pci-host-common.o
> obj-y += ecam.o
> obj-y += pci-host-zynqmp.o
> +obj-y += pci-designware.o
> +obj-y += pci-host-rcar4.o
> diff --git a/xen/arch/arm/pci/pci-designware.c b/xen/arch/arm/pci/pci-designware.c
> new file mode 100644
> index 0000000000..dcbd07ced3
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-designware.c
> @@ -0,0 +1,403 @@
> +/*
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + * Based on xen/arch/arm/pci/pci-host-generic.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/delay.h>
> +#include <asm/io.h>
> +
> +#include "pci-designware.h"
> +/**
> + * upper_32_bits - return bits 32-63 of a number
> + * @n: the number we're accessing
> + *
> + * A basic shift-right of a 64- or 32-bit quantity. Use this to suppress
> + * the "right shift count >= width of type" warning when that quantity is
> + * 32-bits.
> + */
> +#define upper_32_bits(n) ((uint32_t)(((n) >> 16) >> 16))
> +
> +/**
> + * lower_32_bits - return bits 0-31 of a number
> + * @n: the number we're accessing
> + */
> +#define lower_32_bits(n) ((uint32_t)((n) & 0xffffffff))
Please run this by MISRA using ECLAIR's checker in Gitlab CI.
> +static int dw_pcie_read(void __iomem *addr, int size, uint32_t *val)
> +{
> + if ( !IS_ALIGNED((uintptr_t)addr, size) )
> + {
> + *val = 0;
> + return PCIBIOS_BAD_REGISTER_NUMBER;
> + }
> +
> + if ( size == 4 )
> + *val = readl(addr);
> + else if ( size == 2 )
> + *val = readw(addr);
> + else if ( size == 1 )
> + *val = readb(addr);
> + else
> + {
> + *val = 0;
> + return PCIBIOS_BAD_REGISTER_NUMBER;
> + }
> +
> + return PCIBIOS_SUCCESSFUL;
> +}
> +
> +static int dw_pcie_write(void __iomem *addr, int size, uint32_t val)
> +{
> + if ( !IS_ALIGNED((uintptr_t)addr, size) )
> + return PCIBIOS_BAD_REGISTER_NUMBER;
> +
> + if ( size == 4 )
> + writel(val, addr);
> + else if ( size == 2 )
> + writew(val, addr);
> + else if ( size == 1 )
> + writeb(val, addr);
> + else
> + return PCIBIOS_BAD_REGISTER_NUMBER;
> +
> + return PCIBIOS_SUCCESSFUL;
> +}
> +
> +static uint32_t dw_pcie_read_dbi(struct pci_host_bridge *bridge, uint32_t reg,
> + size_t size)
> +{
> + void __iomem *addr = bridge->cfg->win + reg;
> + uint32_t val;
> +
> + dw_pcie_read(addr, size, &val);
> + return val;
> +}
> +
> +static void dw_pcie_write_dbi(struct pci_host_bridge *bridge, uint32_t reg,
> + size_t size, uint32_t val)
> +{
> + void __iomem *addr = bridge->cfg->win + reg;
> +
> + dw_pcie_write(addr, size, val);
> +}
> +
> +static uint32_t dw_pcie_readl_dbi(struct pci_host_bridge *bridge, uint32_t reg)
> +{
> + return dw_pcie_read_dbi(bridge, reg, sizeof(uint32_t));
> +}
> +
> +static void dw_pcie_writel_dbi(struct pci_host_bridge *pci, uint32_t reg,
> + uint32_t val)
> +{
> + dw_pcie_write_dbi(pci, reg, sizeof(uint32_t), val);
> +}
> +
> +static void dw_pcie_read_iatu_unroll_enabled(struct pci_host_bridge *bridge)
> +{
> + struct dw_pcie_priv *priv = bridge->priv;
> + uint32_t val;
> +
> + val = dw_pcie_readl_dbi(bridge, PCIE_ATU_VIEWPORT);
> + if ( val == 0xffffffff )
> + priv->iatu_unroll_enabled = true;
> +
> + printk(XENLOG_DEBUG "%s iATU unroll: %sabled\n",
> + dt_node_full_name(bridge->dt_node),
> + priv->iatu_unroll_enabled ? "en" : "dis");
> +}
> +
> +static uint32_t dw_pcie_readl_atu(struct pci_host_bridge *pci, uint32_t reg)
> +{
> + struct dw_pcie_priv *priv = pci->priv;
> + int ret;
> + uint32_t val;
> +
> + ret = dw_pcie_read(priv->atu_base + reg, 4, &val);
> + if ( ret )
> + printk(XENLOG_ERR "Read ATU address failed\n");
> +
> + return val;
> +}
> +
> +static void dw_pcie_writel_atu(struct pci_host_bridge *pci, uint32_t reg,
> + uint32_t val)
> +{
> + struct dw_pcie_priv *priv = pci->priv;
> + int ret;
> +
> + ret = dw_pcie_write(priv->atu_base + reg, 4, val);
> + if ( ret )
> + printk(XENLOG_ERR "Write ATU address failed\n");
> +}
> +
> +static uint32_t dw_pcie_readl_ob_unroll(struct pci_host_bridge *pci,
> + uint32_t index, uint32_t reg)
> +{
> + uint32_t offset = PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(index);
> +
> + return dw_pcie_readl_atu(pci, offset + reg);
> +}
> +
> +static void dw_pcie_writel_ob_unroll(struct pci_host_bridge *pci,
> + uint32_t index, uint32_t reg, uint32_t val)
> +{
> + uint32_t offset = PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(index);
> +
> + dw_pcie_writel_atu(pci, offset + reg, val);
> +}
> +
> +static uint32_t dw_pcie_enable_ecrc(uint32_t val)
> +{
> + ASSERT_UNREACHABLE();
> + return 0;
> +}
> +
> +static void dw_pcie_prog_outbound_atu_unroll(struct pci_host_bridge *pci,
> + uint8_t func_no, int index,
> + int type, uint64_t cpu_addr,
> + uint64_t pci_addr, uint64_t size)
> +{
> + struct dw_pcie_priv *priv = pci->priv;
> + uint32_t retries, val;
> + uint64_t limit_addr = cpu_addr + size - 1;
> +
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_BASE,
> + lower_32_bits(cpu_addr));
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_BASE,
> + upper_32_bits(cpu_addr));
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_LIMIT,
> + lower_32_bits(limit_addr));
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_LIMIT,
> + upper_32_bits(limit_addr));
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_TARGET,
> + lower_32_bits(pci_addr));
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
> + upper_32_bits(pci_addr));
> + val = type | PCIE_ATU_FUNC_NUM(func_no);
> + val = upper_32_bits(size - 1) ? val | PCIE_ATU_INCREASE_REGION_SIZE : val;
> + if ( priv->version == 0x490A )
> + val = dw_pcie_enable_ecrc(val);
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, val);
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
> + PCIE_ATU_ENABLE);
> +
> + /*
> + * Make sure ATU enable takes effect before any subsequent config
> + * and I/O accesses.
> + */
> + for ( retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++ )
> + {
> + val = dw_pcie_readl_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2);
> + if ( val & PCIE_ATU_ENABLE )
> + return;
> +
> + mdelay(LINK_WAIT_IATU);
> + }
> + printk(XENLOG_ERR "Outbound iATU is not being enabled\n");
> +}
> +
> +static void __dw_pcie_prog_outbound_atu(struct pci_host_bridge *pci,
> + uint8_t func_no, int index, int type,
> + uint64_t cpu_addr, uint64_t pci_addr,
> + uint64_t size)
> +{
> + struct dw_pcie_priv *priv = pci->priv;
> + uint32_t retries, val;
> +
> + if ( priv->iatu_unroll_enabled )
> + {
> + dw_pcie_prog_outbound_atu_unroll(pci, func_no, index, type, cpu_addr,
> + pci_addr, size);
> + return;
> + }
> +
> + dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT,
> + PCIE_ATU_REGION_OUTBOUND | index);
> + dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_BASE, lower_32_bits(cpu_addr));
> + dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_BASE, upper_32_bits(cpu_addr));
> + dw_pcie_writel_dbi(pci, PCIE_ATU_LIMIT, lower_32_bits(cpu_addr + size - 1));
> + if ( priv->version >= 0x460A )
> + dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_LIMIT,
> + upper_32_bits(cpu_addr + size - 1));
> + dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, lower_32_bits(pci_addr));
> + dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET, upper_32_bits(pci_addr));
> + val = type | PCIE_ATU_FUNC_NUM(func_no);
> + val = ((upper_32_bits(size - 1)) && (priv->version >= 0x460A))
> + ? val | PCIE_ATU_INCREASE_REGION_SIZE
> + : val;
> + if ( priv->version == 0x490A )
> + val = dw_pcie_enable_ecrc(val);
> + dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, val);
> + dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE);
> +
> + /*
> + * Make sure ATU enable takes effect before any subsequent config
> + * and I/O accesses.
> + */
> + for ( retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++ )
> + {
> + val = dw_pcie_readl_dbi(pci, PCIE_ATU_CR2);
> + if ( val & PCIE_ATU_ENABLE )
> + return;
> +
> + mdelay(LINK_WAIT_IATU);
> + }
> + printk(XENLOG_ERR "Outbound iATU is not being enabled\n");
> +}
> +
> +static void dw_pcie_prog_outbound_atu(struct pci_host_bridge *pci, int index,
> + int type, uint64_t cpu_addr,
> + uint64_t pci_addr, uint64_t size)
> +{
> + __dw_pcie_prog_outbound_atu(pci, 0, index, type, cpu_addr, pci_addr, size);
> +}
> +
> +void dw_pcie_set_version(struct pci_host_bridge *bridge, unsigned int version)
> +{
> + struct dw_pcie_priv *priv = bridge->priv;
> +
> + priv->version = version;
> +}
> +
> +void __iomem *dw_pcie_child_map_bus(struct pci_host_bridge *bridge,
> + pci_sbdf_t sbdf, uint32_t where)
> +{
> + uint32_t busdev;
> +
> + busdev = PCIE_ATU_BUS(sbdf.bus) | PCIE_ATU_DEV(PCI_SLOT(sbdf.devfn)) |
> + PCIE_ATU_FUNC(PCI_FUNC(sbdf.devfn));
> +
> + /* FIXME: Parent is the root bus, so use PCIE_ATU_TYPE_CFG0. */
> + dw_pcie_prog_outbound_atu(bridge, PCIE_ATU_REGION_INDEX1,
> + PCIE_ATU_TYPE_CFG0, bridge->child_cfg->phys_addr,
> + busdev, bridge->child_cfg->size);
> +
> + return bridge->child_cfg->win + where;
> +}
> +
> +int dw_pcie_child_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> + uint32_t reg, uint32_t len, uint32_t *value)
> +{
> + struct dw_pcie_priv *priv = bridge->priv;
> + int ret;
> +
> + /*
> + * FIXME: we cannot read iATU settings at the early initialization
> + * (probe) as the host's HW is not yet initialized at that phase.
> + * This read operation is the very first thing Domain-0 will do
> + * during its initialization, so take this opportunity and read
> + * iATU setting now.
> + */
> + if ( unlikely(!priv->iatu_unroll_initilized) )
> + {
> + dw_pcie_read_iatu_unroll_enabled(bridge);
> + priv->iatu_unroll_initilized = true;
> + }
> +
> + ret = pci_generic_config_read(bridge, sbdf, reg, len, value);
> + if ( !ret && (priv->num_viewport <= 2) )
> + dw_pcie_prog_outbound_atu(bridge, PCIE_ATU_REGION_INDEX1,
> + PCIE_ATU_TYPE_IO,
> + bridge->child_cfg->phys_addr, 0,
> + bridge->child_cfg->size);
> +
> + return ret;
> +}
> +
> +int dw_pcie_child_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> + uint32_t reg, uint32_t len, uint32_t value)
> +{
> + struct dw_pcie_priv *priv = bridge->priv;
> + int ret;
> +
> + ret = pci_generic_config_write(bridge, sbdf, reg, len, value);
> + if ( !ret && (priv->num_viewport <= 2) )
> + dw_pcie_prog_outbound_atu(bridge, PCIE_ATU_REGION_INDEX1,
> + PCIE_ATU_TYPE_IO,
> + bridge->child_cfg->phys_addr, 0,
> + bridge->child_cfg->size);
> + return ret;
> +}
> +
> +bool __init dw_pcie_child_need_p2m_hwdom_mapping(struct domain *d,
> + struct pci_host_bridge *bridge,
> + uint64_t addr)
> +{
> + struct pci_config_window *cfg = bridge->child_cfg;
> +
> + /*
> + * We do not want ECAM address space to be mapped in Domain-0's p2m,
> + * so we can trap access to it.
> + */
> + return cfg->phys_addr != addr;
> +}
> +
> +struct pci_host_bridge *__init
> +dw_pcie_host_probe(struct dt_device_node *dev, const void *data,
> + const struct pci_ecam_ops *ops,
> + const struct pci_ecam_ops *child_ops)
> +{
> + struct pci_host_bridge *bridge;
> + struct dw_pcie_priv *priv;
> +
> + paddr_t atu_phys_addr;
> + paddr_t atu_size;
> + int atu_idx, ret;
> +
> + bridge = pci_host_common_probe(dev, ops, child_ops);
> + if ( IS_ERR(bridge) )
> + return bridge;
> +
> + priv = xzalloc_bytes(sizeof(struct dw_pcie_priv));
> + if ( !priv )
> + return ERR_PTR(-ENOMEM);
> +
> + bridge->priv = priv;
> +
> + atu_idx = dt_property_match_string(dev, "reg-names", "atu");
> + if ( atu_idx < 0 )
> + {
> + printk(XENLOG_ERR "Cannot find \"atu\" range index in device tree\n");
> + return ERR_PTR(atu_idx);
> + }
> + ret = dt_device_get_address(dev, atu_idx, &atu_phys_addr, &atu_size);
> + if ( ret )
> + {
> + printk(XENLOG_ERR "Cannot find \"atu\" range in device tree\n");
> + return ERR_PTR(ret);
> + }
> + printk("iATU at [mem 0x%" PRIpaddr "-0x%" PRIpaddr "]\n", atu_phys_addr,
> + atu_phys_addr + atu_size - 1);
> + priv->atu_base = ioremap_nocache(atu_phys_addr, atu_size);
> + if ( !priv->atu_base )
> + {
> + printk(XENLOG_ERR "iATU ioremap failed\n");
> + return ERR_PTR(ENXIO);
> + }
> +
> + if ( !dt_property_read_u32(dev, "num-viewport", &priv->num_viewport) )
> + priv->num_viewport = 2;
> +
> + /*
> + * FIXME: we cannot read iATU unroll enable now as the host bridge's
> + * HW is not yet initialized by Domain-0: leave it for later.
> + */
> +
> + printk(XENLOG_INFO "%s number of view ports: %d\n", dt_node_full_name(dev),
> + priv->num_viewport);
> +
> + return bridge;
> +}
> diff --git a/xen/arch/arm/pci/pci-designware.h b/xen/arch/arm/pci/pci-designware.h
> new file mode 100644
> index 0000000000..a15ba9a329
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-designware.h
> @@ -0,0 +1,102 @@
> +/*
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + * Based on xen/arch/arm/pci/pci-host-generic.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/pci.h>
> +#include <xen/init.h>
> +
> +#ifndef __PCI_DESIGNWARE_H__
> +#define __PCI_DESIGNWARE_H__
> +
> +#define PCIBIOS_SUCCESSFUL 0x00
> +#define PCIBIOS_BAD_REGISTER_NUMBER 0x87
> +
> +#define PCIE_ATU_VIEWPORT 0x900
> +#define PCIE_ATU_REGION_OUTBOUND 0
> +#define PCIE_ATU_CR1 0x904
> +#define PCIE_ATU_INCREASE_REGION_SIZE BIT(13, UL)
> +#define PCIE_ATU_CR2 0x908
> +#define PCIE_ATU_ENABLE BIT(31, UL)
> +#define PCIE_ATU_LOWER_BASE 0x90C
> +#define PCIE_ATU_UPPER_BASE 0x910
> +#define PCIE_ATU_LIMIT 0x914
> +#define PCIE_ATU_LOWER_TARGET 0x918
> +#define PCIE_ATU_UPPER_TARGET 0x91C
> +#define PCIE_ATU_UPPER_LIMIT 0x924
> +
> +#define PCIE_ATU_REGION_INDEX1 0x1
> +#define PCIE_ATU_TYPE_IO 0x2
> +#define PCIE_ATU_TYPE_CFG0 0x4
> +
> +#define FIELD_PREP(_mask, _val) \
> + (((typeof(_mask))(_val) << (ffs64(_mask) - 1)) & (_mask))
> +
> +#define PCIE_ATU_BUS(x) FIELD_PREP(GENMASK(31, 24), x)
> +#define PCIE_ATU_DEV(x) FIELD_PREP(GENMASK(23, 19), x)
> +#define PCIE_ATU_FUNC(x) FIELD_PREP(GENMASK(18, 16), x)
> +
> +/* Register address builder */
> +#define PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(region) \
> + ((region) << 9)
> +
> +/*
> + * iATU Unroll-specific register definitions
> + * From 4.80 core version the address translation will be made by unroll
> + */
> +#define PCIE_ATU_UNR_REGION_CTRL1 0x00
> +#define PCIE_ATU_UNR_REGION_CTRL2 0x04
> +#define PCIE_ATU_UNR_LOWER_BASE 0x08
> +#define PCIE_ATU_UNR_UPPER_BASE 0x0C
> +#define PCIE_ATU_UNR_LOWER_LIMIT 0x10
> +#define PCIE_ATU_UNR_LOWER_TARGET 0x14
> +#define PCIE_ATU_UNR_UPPER_TARGET 0x18
> +#define PCIE_ATU_UNR_UPPER_LIMIT 0x20
> +
> +#define PCIE_ATU_FUNC_NUM(pf) ((pf) << 20)
> +
> +/* Parameters for the waiting for iATU enabled routine */
> +#define LINK_WAIT_MAX_IATU_RETRIES 5
> +#define LINK_WAIT_IATU 9
> +
> +struct dw_pcie_priv {
> + uint32_t num_viewport;
> + bool iatu_unroll_initilized;
> + bool iatu_unroll_enabled;
> + void __iomem *atu_base;
> + unsigned int version;
> +};
> +
> +void dw_pcie_set_version(struct pci_host_bridge *bridge, unsigned int version);
> +
> +void __iomem *dw_pcie_child_map_bus(struct pci_host_bridge *bridge,
> + pci_sbdf_t sbdf, uint32_t where);
> +
> +int dw_pcie_child_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> + uint32_t reg, uint32_t len, uint32_t *value);
> +
> +int dw_pcie_child_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> + uint32_t reg, uint32_t len, uint32_t value);
> +
> +bool __init dw_pcie_child_need_p2m_hwdom_mapping(struct domain *d,
> + struct pci_host_bridge *bridge,
> + uint64_t addr);
> +
> +struct pci_host_bridge *__init
> +dw_pcie_host_probe(struct dt_device_node *dev, const void *data,
> + const struct pci_ecam_ops *ops,
> + const struct pci_ecam_ops *child_ops);
> +#endif /* __PCI_DESIGNWARE_H__ */
> diff --git a/xen/arch/arm/pci/pci-host-rcar4.c b/xen/arch/arm/pci/pci-host-rcar4.c
> new file mode 100644
> index 0000000000..cae1b4254d
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-host-rcar4.c
> @@ -0,0 +1,103 @@
> +/*
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + * Based on xen/arch/arm/pci/pci-host-generic.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/init.h>
> +#include <xen/pci.h>
> +
> +#include <asm/device.h>
> +#include <asm/io.h>
> +#include <asm/pci.h>
> +
> +#include "pci-designware.h"
> +
> +#define RCAR4_DWC_VERSION 0x520A
> +
> +/*
> + * PCI host bridges often have different ways to access the root and child
> + * bus config spaces:
> + * "dbi" : the aperture where root port's own configuration registers
> + * are available.
> + * "config": child's configuration space
> + * "atu" : iATU registers for DWC version 4.80 or later
> + */
> +static int __init rcar4_cfg_reg_index(struct dt_device_node *np)
> +{
> + return dt_property_match_string(np, "reg-names", "dbi");
> +}
> +
> +static int __init rcar4_child_cfg_reg_index(struct dt_device_node *np)
> +{
> + return dt_property_match_string(np, "reg-names", "config");
> +}
> +
> +/* ECAM ops */
> +const struct pci_ecam_ops rcar4_pcie_ops = {
> + .bus_shift = 20,
> + .cfg_reg_index = rcar4_cfg_reg_index,
> + .pci_ops = {
> + .map_bus = pci_ecam_map_bus,
> + .read = pci_generic_config_read,
> + .write = pci_generic_config_write,
> + .need_p2m_hwdom_mapping = pci_ecam_need_p2m_hwdom_mapping,
> + }
> +};
> +
> +const struct pci_ecam_ops rcar4_pcie_child_ops = {
> + .bus_shift = 20,
> + .cfg_reg_index = rcar4_child_cfg_reg_index,
> + .pci_ops = {
> + .map_bus = dw_pcie_child_map_bus,
> + .read = dw_pcie_child_config_read,
> + .write = dw_pcie_child_config_write,
> + .need_p2m_hwdom_mapping = dw_pcie_child_need_p2m_hwdom_mapping,
> + }
> +};
> +
> +static const struct dt_device_match __initconstrel rcar4_pcie_dt_match[] = {
> + { .compatible = "renesas,r8a779f0-pcie" },
> + { .compatible = "renesas,r8a779g0-pcie" },
> + {},
> +};
> +
> +static int __init pci_host_generic_probe(struct dt_device_node *dev,
> + const void *data)
> +{
> + struct pci_host_bridge *bridge;
> +
> + bridge = dw_pcie_host_probe(dev, data, &rcar4_pcie_ops,
> + &rcar4_pcie_child_ops);
> +
> + dw_pcie_set_version(bridge, RCAR4_DWC_VERSION);
> +
> + return 0;
> +}
> +
> +DT_DEVICE_START(pci_gen, "PCI HOST R-CAR GEN4", DEVICE_PCI_HOSTBRIDGE)
> +.dt_match = rcar4_pcie_dt_match,
> +.init = pci_host_generic_probe,
> +DT_DEVICE_END
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH v4 4/4] xen/arm: add support for R-Car Gen4 PCI host controller
2025-04-23 11:08 ` [PATCH v4 4/4] xen/arm: add support for R-Car Gen4 PCI host controller Mykyta Poturai
2025-04-23 21:06 ` Stefano Stabellini
@ 2025-04-24 19:22 ` Stewart Hildebrand
2025-05-13 14:44 ` Mykyta Poturai
1 sibling, 1 reply; 12+ messages in thread
From: Stewart Hildebrand @ 2025-04-24 19:22 UTC (permalink / raw)
To: Mykyta Poturai, xen-devel@lists.xenproject.org
Cc: Oleksandr Andrushchenko, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk
On 4/23/25 07:08, Mykyta Poturai wrote:
> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>
> Add support for Renesas R-Car Gen4 PCI host controller, specifically
> targeting the S4 and V4H SoCs. The implementation includes configuration
> read/write operations for both root and child buses. For accessing the
> child bus, iATU is used for address translation.
In a system with dom0, I assume dom0 programs the iATU, and would do so
before issuing any PHYSDEVOP_pci_device_add hypercalls or PCI config
space accesses. Is that a correct assumption? If so, what is the use
case for iATU initialization in Xen? E.g. is it to guard against a
misconfigured or broken dom0? Do different iATU settings need to be
applied when running as a Xen guest vs not?
Further, is this driver sufficient to initialize the PCI controller
without dom0 involvement (e.g. to enable dom0less PCI passthrough)?
These would all be valid use cases IMO, but I would just like to
understand the rationale.
>
> Code common to all DesignWare PCI host controllers is located in a
> separate file to allow for easy reuse in other DesignWare-based PCI
> host controllers.
>
> Signed-off-by: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
> Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
> ---
> v3->v4:
> * no changes
>
> v2->v3:
> * move priv allocation to dw_pcie_host_probe
>
> v1->v2:
> * move designware code in a separate file
> ---
> xen/arch/arm/pci/Makefile | 2 +
> xen/arch/arm/pci/pci-designware.c | 403 ++++++++++++++++++++++++++++++
> xen/arch/arm/pci/pci-designware.h | 102 ++++++++
> xen/arch/arm/pci/pci-host-rcar4.c | 103 ++++++++
> 4 files changed, 610 insertions(+)
> create mode 100644 xen/arch/arm/pci/pci-designware.c
> create mode 100644 xen/arch/arm/pci/pci-designware.h
> create mode 100644 xen/arch/arm/pci/pci-host-rcar4.c
>
> diff --git a/xen/arch/arm/pci/Makefile b/xen/arch/arm/pci/Makefile
> index 1d045ade01..ca6135e282 100644
> --- a/xen/arch/arm/pci/Makefile
> +++ b/xen/arch/arm/pci/Makefile
> @@ -4,3 +4,5 @@ obj-y += pci-host-generic.o
> obj-y += pci-host-common.o
> obj-y += ecam.o
> obj-y += pci-host-zynqmp.o
> +obj-y += pci-designware.o
> +obj-y += pci-host-rcar4.o
> diff --git a/xen/arch/arm/pci/pci-designware.c b/xen/arch/arm/pci/pci-designware.c
> new file mode 100644
> index 0000000000..dcbd07ced3
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-designware.c
> @@ -0,0 +1,403 @@
> +/*
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + * Based on xen/arch/arm/pci/pci-host-generic.c
Linux drivers/pci/controller/dwc/pcie-designware.c ?
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
Please use SPDX (see CODING_STYLE) here and in the other two new files.
> +
> +#include <xen/delay.h>
> +#include <asm/io.h>
> +
> +#include "pci-designware.h"
> +/**
> + * upper_32_bits - return bits 32-63 of a number
> + * @n: the number we're accessing
> + *
> + * A basic shift-right of a 64- or 32-bit quantity. Use this to suppress
> + * the "right shift count >= width of type" warning when that quantity is
> + * 32-bits.
> + */
> +#define upper_32_bits(n) ((uint32_t)(((n) >> 16) >> 16))
> +
> +/**
> + * lower_32_bits - return bits 0-31 of a number
> + * @n: the number we're accessing
> + */
> +#define lower_32_bits(n) ((uint32_t)((n) & 0xffffffff))
This constant needs a U suffix
> +
> +static int dw_pcie_read(void __iomem *addr, int size, uint32_t *val)
> +{
> + if ( !IS_ALIGNED((uintptr_t)addr, size) )
> + {
> + *val = 0;
> + return PCIBIOS_BAD_REGISTER_NUMBER;
The callers of dw_pcie_{read,write} below don't appear to do much (other
than printk) about the error returned. Please see
docs/process/coding-best-practices.pandoc regarding error handling.
> + }
> +
> + if ( size == 4 )
> + *val = readl(addr);
> + else if ( size == 2 )
> + *val = readw(addr);
> + else if ( size == 1 )
> + *val = readb(addr);
> + else
> + {
> + *val = 0;
> + return PCIBIOS_BAD_REGISTER_NUMBER;
> + }
> +
> + return PCIBIOS_SUCCESSFUL;
> +}
> +
> +static int dw_pcie_write(void __iomem *addr, int size, uint32_t val)
> +{
> + if ( !IS_ALIGNED((uintptr_t)addr, size) )
> + return PCIBIOS_BAD_REGISTER_NUMBER;
> +
> + if ( size == 4 )
> + writel(val, addr);
> + else if ( size == 2 )
> + writew(val, addr);
> + else if ( size == 1 )
> + writeb(val, addr);
> + else
> + return PCIBIOS_BAD_REGISTER_NUMBER;
> +
> + return PCIBIOS_SUCCESSFUL;
> +}
> +
> +static uint32_t dw_pcie_read_dbi(struct pci_host_bridge *bridge, uint32_t reg,
> + size_t size)
> +{
> + void __iomem *addr = bridge->cfg->win + reg;
> + uint32_t val;
> +
> + dw_pcie_read(addr, size, &val);
> + return val;
> +}
> +
> +static void dw_pcie_write_dbi(struct pci_host_bridge *bridge, uint32_t reg,
> + size_t size, uint32_t val)
> +{
> + void __iomem *addr = bridge->cfg->win + reg;
> +
> + dw_pcie_write(addr, size, val);
> +}
> +
> +static uint32_t dw_pcie_readl_dbi(struct pci_host_bridge *bridge, uint32_t reg)
> +{
> + return dw_pcie_read_dbi(bridge, reg, sizeof(uint32_t));
> +}
> +
> +static void dw_pcie_writel_dbi(struct pci_host_bridge *pci, uint32_t reg,
> + uint32_t val)
> +{
> + dw_pcie_write_dbi(pci, reg, sizeof(uint32_t), val);
> +}
> +
> +static void dw_pcie_read_iatu_unroll_enabled(struct pci_host_bridge *bridge)
> +{
> + struct dw_pcie_priv *priv = bridge->priv;
> + uint32_t val;
> +
> + val = dw_pcie_readl_dbi(bridge, PCIE_ATU_VIEWPORT);
> + if ( val == 0xffffffff )
This constant needs a U suffix
> + priv->iatu_unroll_enabled = true;
> +
> + printk(XENLOG_DEBUG "%s iATU unroll: %sabled\n",
XENLOG_G_DEBUG
> + dt_node_full_name(bridge->dt_node),
> + priv->iatu_unroll_enabled ? "en" : "dis");
> +}
> +
> +static uint32_t dw_pcie_readl_atu(struct pci_host_bridge *pci, uint32_t reg)
> +{
> + struct dw_pcie_priv *priv = pci->priv;
> + int ret;
> + uint32_t val;
> +
> + ret = dw_pcie_read(priv->atu_base + reg, 4, &val);
> + if ( ret )
> + printk(XENLOG_ERR "Read ATU address failed\n");
XENLOG_G_ERR
> +
> + return val;
> +}
> +
> +static void dw_pcie_writel_atu(struct pci_host_bridge *pci, uint32_t reg,
> + uint32_t val)
> +{
> + struct dw_pcie_priv *priv = pci->priv;
> + int ret;
> +
> + ret = dw_pcie_write(priv->atu_base + reg, 4, val);
> + if ( ret )
> + printk(XENLOG_ERR "Write ATU address failed\n");
XENLOG_G_ERR
> +}
> +
> +static uint32_t dw_pcie_readl_ob_unroll(struct pci_host_bridge *pci,
> + uint32_t index, uint32_t reg)
> +{
> + uint32_t offset = PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(index);
> +
> + return dw_pcie_readl_atu(pci, offset + reg);
> +}
> +
> +static void dw_pcie_writel_ob_unroll(struct pci_host_bridge *pci,
> + uint32_t index, uint32_t reg, uint32_t val)
> +{
> + uint32_t offset = PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(index);
> +
> + dw_pcie_writel_atu(pci, offset + reg, val);
> +}
> +
> +static uint32_t dw_pcie_enable_ecrc(uint32_t val)
> +{
> + ASSERT_UNREACHABLE();
> + return 0;
> +}
> +
> +static void dw_pcie_prog_outbound_atu_unroll(struct pci_host_bridge *pci,
> + uint8_t func_no, int index,
> + int type, uint64_t cpu_addr,
> + uint64_t pci_addr, uint64_t size)
> +{
> + struct dw_pcie_priv *priv = pci->priv;
> + uint32_t retries, val;
> + uint64_t limit_addr = cpu_addr + size - 1;
> +
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_BASE,
> + lower_32_bits(cpu_addr));
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_BASE,
> + upper_32_bits(cpu_addr));
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_LIMIT,
> + lower_32_bits(limit_addr));
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_LIMIT,
> + upper_32_bits(limit_addr));
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_LOWER_TARGET,
> + lower_32_bits(pci_addr));
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_UPPER_TARGET,
> + upper_32_bits(pci_addr));
> + val = type | PCIE_ATU_FUNC_NUM(func_no);
> + val = upper_32_bits(size - 1) ? val | PCIE_ATU_INCREASE_REGION_SIZE : val;
> + if ( priv->version == 0x490A )
> + val = dw_pcie_enable_ecrc(val);
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL1, val);
> + dw_pcie_writel_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2,
> + PCIE_ATU_ENABLE);
> +
> + /*
> + * Make sure ATU enable takes effect before any subsequent config
> + * and I/O accesses.
> + */
> + for ( retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++ )
> + {
> + val = dw_pcie_readl_ob_unroll(pci, index, PCIE_ATU_UNR_REGION_CTRL2);
> + if ( val & PCIE_ATU_ENABLE )
> + return;
> +
> + mdelay(LINK_WAIT_IATU);
> + }
dw_pcie_child_{map_bus,config_read,config_write}, and thus
dw_pcie_prog_outbound_atu{_unroll}, would potentially be called on every
guest PCI access, where the d->pci_lock may be held. It doesn't seem
appropriate to have a busy-wait such as this in that code path.
If the busy-wait is necessary, then perhaps iATU configuration should
happen during init/probe (as your FIXME note below suggests), not as a
consequence of guest PCI access.
Same comment applies below.
> + printk(XENLOG_ERR "Outbound iATU is not being enabled\n");
XENLOG_G_ERR
> +}
> +
> +static void __dw_pcie_prog_outbound_atu(struct pci_host_bridge *pci,
> + uint8_t func_no, int index, int type,
> + uint64_t cpu_addr, uint64_t pci_addr,
> + uint64_t size)
> +{
> + struct dw_pcie_priv *priv = pci->priv;
> + uint32_t retries, val;
> +
> + if ( priv->iatu_unroll_enabled )
> + {
> + dw_pcie_prog_outbound_atu_unroll(pci, func_no, index, type, cpu_addr,
> + pci_addr, size);
> + return;
> + }
> +
> + dw_pcie_writel_dbi(pci, PCIE_ATU_VIEWPORT,
> + PCIE_ATU_REGION_OUTBOUND | index);
> + dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_BASE, lower_32_bits(cpu_addr));
> + dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_BASE, upper_32_bits(cpu_addr));
> + dw_pcie_writel_dbi(pci, PCIE_ATU_LIMIT, lower_32_bits(cpu_addr + size - 1));
> + if ( priv->version >= 0x460A )
> + dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_LIMIT,
> + upper_32_bits(cpu_addr + size - 1));
> + dw_pcie_writel_dbi(pci, PCIE_ATU_LOWER_TARGET, lower_32_bits(pci_addr));
> + dw_pcie_writel_dbi(pci, PCIE_ATU_UPPER_TARGET, upper_32_bits(pci_addr));
> + val = type | PCIE_ATU_FUNC_NUM(func_no);
> + val = ((upper_32_bits(size - 1)) && (priv->version >= 0x460A))
> + ? val | PCIE_ATU_INCREASE_REGION_SIZE
> + : val;
> + if ( priv->version == 0x490A )
> + val = dw_pcie_enable_ecrc(val);
> + dw_pcie_writel_dbi(pci, PCIE_ATU_CR1, val);
> + dw_pcie_writel_dbi(pci, PCIE_ATU_CR2, PCIE_ATU_ENABLE);
> +
> + /*
> + * Make sure ATU enable takes effect before any subsequent config
> + * and I/O accesses.
> + */
> + for ( retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++ )
> + {
> + val = dw_pcie_readl_dbi(pci, PCIE_ATU_CR2);
> + if ( val & PCIE_ATU_ENABLE )
> + return;
> +
> + mdelay(LINK_WAIT_IATU);
> + }> + printk(XENLOG_ERR "Outbound iATU is not being enabled\n");
XENLOG_G_ERR
> +}
> +
> +static void dw_pcie_prog_outbound_atu(struct pci_host_bridge *pci, int index,
> + int type, uint64_t cpu_addr,
> + uint64_t pci_addr, uint64_t size)
> +{
> + __dw_pcie_prog_outbound_atu(pci, 0, index, type, cpu_addr, pci_addr, size);
> +}
> +
> +void dw_pcie_set_version(struct pci_host_bridge *bridge, unsigned int version)
> +{
> + struct dw_pcie_priv *priv = bridge->priv;
> +
> + priv->version = version;
> +}
> +
> +void __iomem *dw_pcie_child_map_bus(struct pci_host_bridge *bridge,
> + pci_sbdf_t sbdf, uint32_t where)
> +{
> + uint32_t busdev;
> +
> + busdev = PCIE_ATU_BUS(sbdf.bus) | PCIE_ATU_DEV(PCI_SLOT(sbdf.devfn)) |
> + PCIE_ATU_FUNC(PCI_FUNC(sbdf.devfn));
> +
> + /* FIXME: Parent is the root bus, so use PCIE_ATU_TYPE_CFG0. */
> + dw_pcie_prog_outbound_atu(bridge, PCIE_ATU_REGION_INDEX1,
> + PCIE_ATU_TYPE_CFG0, bridge->child_cfg->phys_addr,
> + busdev, bridge->child_cfg->size);
Does iATU configuration really need to happen on every guest PCI access?
> +
> + return bridge->child_cfg->win + where;
> +}
> +
> +int dw_pcie_child_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> + uint32_t reg, uint32_t len, uint32_t *value)
> +{
> + struct dw_pcie_priv *priv = bridge->priv;
> + int ret;
> +
> + /*
> + * FIXME: we cannot read iATU settings at the early initialization
> + * (probe) as the host's HW is not yet initialized at that phase.
> + * This read operation is the very first thing Domain-0 will do
> + * during its initialization, so take this opportunity and read
> + * iATU setting now.
> + */
> + if ( unlikely(!priv->iatu_unroll_initilized) )
> + {
> + dw_pcie_read_iatu_unroll_enabled(bridge);
> + priv->iatu_unroll_initilized = true;
> + }
> +
> + ret = pci_generic_config_read(bridge, sbdf, reg, len, value);
> + if ( !ret && (priv->num_viewport <= 2) )
> + dw_pcie_prog_outbound_atu(bridge, PCIE_ATU_REGION_INDEX1,
> + PCIE_ATU_TYPE_IO,
> + bridge->child_cfg->phys_addr, 0,
> + bridge->child_cfg->size);
> +
> + return ret;
> +}
> +
> +int dw_pcie_child_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> + uint32_t reg, uint32_t len, uint32_t value)
> +{
> + struct dw_pcie_priv *priv = bridge->priv;
> + int ret;
> +
> + ret = pci_generic_config_write(bridge, sbdf, reg, len, value);
> + if ( !ret && (priv->num_viewport <= 2) )
> + dw_pcie_prog_outbound_atu(bridge, PCIE_ATU_REGION_INDEX1,
> + PCIE_ATU_TYPE_IO,
> + bridge->child_cfg->phys_addr, 0,
> + bridge->child_cfg->size);
> + return ret;
> +}
> +
> +bool __init dw_pcie_child_need_p2m_hwdom_mapping(struct domain *d,
> + struct pci_host_bridge *bridge,
> + uint64_t addr)
> +{
> + struct pci_config_window *cfg = bridge->child_cfg;
> +
> + /*
> + * We do not want ECAM address space to be mapped in Domain-0's p2m,
> + * so we can trap access to it.
> + */
> + return cfg->phys_addr != addr;
> +}
> +
> +struct pci_host_bridge *__init
> +dw_pcie_host_probe(struct dt_device_node *dev, const void *data,
> + const struct pci_ecam_ops *ops,
> + const struct pci_ecam_ops *child_ops)
> +{
> + struct pci_host_bridge *bridge;
> + struct dw_pcie_priv *priv;
> +
> + paddr_t atu_phys_addr;
> + paddr_t atu_size;
> + int atu_idx, ret;
> +
> + bridge = pci_host_common_probe(dev, ops, child_ops);
> + if ( IS_ERR(bridge) )
> + return bridge;
> +
> + priv = xzalloc_bytes(sizeof(struct dw_pcie_priv));
xzalloc(struct dw_pcie_priv) ?
> + if ( !priv )
> + return ERR_PTR(-ENOMEM);
> +
> + bridge->priv = priv;
> +
> + atu_idx = dt_property_match_string(dev, "reg-names", "atu");
> + if ( atu_idx < 0 )
> + {
> + printk(XENLOG_ERR "Cannot find \"atu\" range index in device tree\n");
> + return ERR_PTR(atu_idx);
> + }
> + ret = dt_device_get_address(dev, atu_idx, &atu_phys_addr, &atu_size);
> + if ( ret )
> + {
> + printk(XENLOG_ERR "Cannot find \"atu\" range in device tree\n");
> + return ERR_PTR(ret);
> + }
> + printk("iATU at [mem 0x%" PRIpaddr "-0x%" PRIpaddr "]\n", atu_phys_addr,
> + atu_phys_addr + atu_size - 1);
> + priv->atu_base = ioremap_nocache(atu_phys_addr, atu_size);
> + if ( !priv->atu_base )
> + {
> + printk(XENLOG_ERR "iATU ioremap failed\n");
> + return ERR_PTR(ENXIO);
> + }
> +
> + if ( !dt_property_read_u32(dev, "num-viewport", &priv->num_viewport) )
> + priv->num_viewport = 2;
> +
> + /*
> + * FIXME: we cannot read iATU unroll enable now as the host bridge's
> + * HW is not yet initialized by Domain-0: leave it for later.
> + */
Is there a plan to address this?
> +
> + printk(XENLOG_INFO "%s number of view ports: %d\n", dt_node_full_name(dev),
> + priv->num_viewport);
> +
> + return bridge;
> +}
> diff --git a/xen/arch/arm/pci/pci-designware.h b/xen/arch/arm/pci/pci-designware.h
> new file mode 100644
> index 0000000000..a15ba9a329
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-designware.h
> @@ -0,0 +1,102 @@
> +/*
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + * Based on xen/arch/arm/pci/pci-host-generic.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/pci.h>
> +#include <xen/init.h>
> +
> +#ifndef __PCI_DESIGNWARE_H__
> +#define __PCI_DESIGNWARE_H__
> +
> +#define PCIBIOS_SUCCESSFUL 0x00
> +#define PCIBIOS_BAD_REGISTER_NUMBER 0x87
Could you use existing error code definitions instead of introducing new
ones?
> +
> +#define PCIE_ATU_VIEWPORT 0x900
> +#define PCIE_ATU_REGION_OUTBOUND 0
> +#define PCIE_ATU_CR1 0x904
> +#define PCIE_ATU_INCREASE_REGION_SIZE BIT(13, UL)
> +#define PCIE_ATU_CR2 0x908
> +#define PCIE_ATU_ENABLE BIT(31, UL)
> +#define PCIE_ATU_LOWER_BASE 0x90C
> +#define PCIE_ATU_UPPER_BASE 0x910
> +#define PCIE_ATU_LIMIT 0x914
> +#define PCIE_ATU_LOWER_TARGET 0x918
> +#define PCIE_ATU_UPPER_TARGET 0x91C
> +#define PCIE_ATU_UPPER_LIMIT 0x924
> +
> +#define PCIE_ATU_REGION_INDEX1 0x1
> +#define PCIE_ATU_TYPE_IO 0x2
> +#define PCIE_ATU_TYPE_CFG0 0x4
> +
> +#define FIELD_PREP(_mask, _val) \
> + (((typeof(_mask))(_val) << (ffs64(_mask) - 1)) & (_mask))
> +
> +#define PCIE_ATU_BUS(x) FIELD_PREP(GENMASK(31, 24), x)
> +#define PCIE_ATU_DEV(x) FIELD_PREP(GENMASK(23, 19), x)
> +#define PCIE_ATU_FUNC(x) FIELD_PREP(GENMASK(18, 16), x)
Parenthesis around (x)
> +
> +/* Register address builder */
> +#define PCIE_GET_ATU_OUTB_UNR_REG_OFFSET(region) \
> + ((region) << 9)
> +
> +/*
> + * iATU Unroll-specific register definitions
> + * From 4.80 core version the address translation will be made by unroll
> + */
> +#define PCIE_ATU_UNR_REGION_CTRL1 0x00
> +#define PCIE_ATU_UNR_REGION_CTRL2 0x04
> +#define PCIE_ATU_UNR_LOWER_BASE 0x08
> +#define PCIE_ATU_UNR_UPPER_BASE 0x0C
> +#define PCIE_ATU_UNR_LOWER_LIMIT 0x10
> +#define PCIE_ATU_UNR_LOWER_TARGET 0x14
> +#define PCIE_ATU_UNR_UPPER_TARGET 0x18
> +#define PCIE_ATU_UNR_UPPER_LIMIT 0x20
> +
> +#define PCIE_ATU_FUNC_NUM(pf) ((pf) << 20)
> +
> +/* Parameters for the waiting for iATU enabled routine */
> +#define LINK_WAIT_MAX_IATU_RETRIES 5
> +#define LINK_WAIT_IATU 9
> +
> +struct dw_pcie_priv {
> + uint32_t num_viewport;
> + bool iatu_unroll_initilized;
> + bool iatu_unroll_enabled;
> + void __iomem *atu_base;
> + unsigned int version;
> +};
> +
> +void dw_pcie_set_version(struct pci_host_bridge *bridge, unsigned int version);
> +
> +void __iomem *dw_pcie_child_map_bus(struct pci_host_bridge *bridge,
> + pci_sbdf_t sbdf, uint32_t where);
> +
> +int dw_pcie_child_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> + uint32_t reg, uint32_t len, uint32_t *value);
> +
> +int dw_pcie_child_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf,
> + uint32_t reg, uint32_t len, uint32_t value);
> +
> +bool __init dw_pcie_child_need_p2m_hwdom_mapping(struct domain *d,
> + struct pci_host_bridge *bridge,
> + uint64_t addr);
> +
> +struct pci_host_bridge *__init
> +dw_pcie_host_probe(struct dt_device_node *dev, const void *data,
> + const struct pci_ecam_ops *ops,
> + const struct pci_ecam_ops *child_ops);
> +#endif /* __PCI_DESIGNWARE_H__ */
> diff --git a/xen/arch/arm/pci/pci-host-rcar4.c b/xen/arch/arm/pci/pci-host-rcar4.c
> new file mode 100644
> index 0000000000..cae1b4254d
> --- /dev/null
> +++ b/xen/arch/arm/pci/pci-host-rcar4.c
> @@ -0,0 +1,103 @@
> +/*
> + * Based on Linux drivers/pci/controller/pci-host-common.c
> + * Based on Linux drivers/pci/controller/pci-host-generic.c
> + * Based on xen/arch/arm/pci/pci-host-generic.c
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */> +
> +#include <xen/init.h>
> +#include <xen/pci.h>
> +
> +#include <asm/device.h>
> +#include <asm/io.h>
> +#include <asm/pci.h>
> +
> +#include "pci-designware.h"
> +
> +#define RCAR4_DWC_VERSION 0x520A
> +
> +/*
> + * PCI host bridges often have different ways to access the root and child
> + * bus config spaces:
> + * "dbi" : the aperture where root port's own configuration registers
> + * are available.
> + * "config": child's configuration space
> + * "atu" : iATU registers for DWC version 4.80 or later
> + */
> +static int __init rcar4_cfg_reg_index(struct dt_device_node *np)
> +{
> + return dt_property_match_string(np, "reg-names", "dbi");
> +}
> +
> +static int __init rcar4_child_cfg_reg_index(struct dt_device_node *np)
> +{
> + return dt_property_match_string(np, "reg-names", "config");
> +}
> +
> +/* ECAM ops */
> +const struct pci_ecam_ops rcar4_pcie_ops = {
> + .bus_shift = 20,
> + .cfg_reg_index = rcar4_cfg_reg_index,
> + .pci_ops = {
> + .map_bus = pci_ecam_map_bus,
> + .read = pci_generic_config_read,
> + .write = pci_generic_config_write,
> + .need_p2m_hwdom_mapping = pci_ecam_need_p2m_hwdom_mapping,
> + }
> +};
> +
> +const struct pci_ecam_ops rcar4_pcie_child_ops = {
> + .bus_shift = 20,
> + .cfg_reg_index = rcar4_child_cfg_reg_index,
> + .pci_ops = {
> + .map_bus = dw_pcie_child_map_bus,
> + .read = dw_pcie_child_config_read,
> + .write = dw_pcie_child_config_write,
> + .need_p2m_hwdom_mapping = dw_pcie_child_need_p2m_hwdom_mapping,
> + }
> +};
> +
> +static const struct dt_device_match __initconstrel rcar4_pcie_dt_match[] = {
> + { .compatible = "renesas,r8a779f0-pcie" },
> + { .compatible = "renesas,r8a779g0-pcie" },
> + {},
> +};
> +
> +static int __init pci_host_generic_probe(struct dt_device_node *dev,
> + const void *data)
It's not really a "generic" probe... How about naming the function
pci_host_rcar4_probe ?
> +{
> + struct pci_host_bridge *bridge;
> +
> + bridge = dw_pcie_host_probe(dev, data, &rcar4_pcie_ops,
> + &rcar4_pcie_child_ops);
> +
> + dw_pcie_set_version(bridge, RCAR4_DWC_VERSION);
> +
> + return 0;
> +}
> +
> +DT_DEVICE_START(pci_gen, "PCI HOST R-CAR GEN4", DEVICE_PCI_HOSTBRIDGE)
> +.dt_match = rcar4_pcie_dt_match,
> +.init = pci_host_generic_probe,
> +DT_DEVICE_END
> +
> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * tab-width: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH v4 4/4] xen/arm: add support for R-Car Gen4 PCI host controller
2025-04-24 19:22 ` Stewart Hildebrand
@ 2025-05-13 14:44 ` Mykyta Poturai
0 siblings, 0 replies; 12+ messages in thread
From: Mykyta Poturai @ 2025-05-13 14:44 UTC (permalink / raw)
To: Stewart Hildebrand, xen-devel@lists.xenproject.org
Cc: Oleksandr Andrushchenko, Stefano Stabellini, Julien Grall,
Bertrand Marquis, Michal Orzel, Volodymyr Babchuk
On 24.04.25 22:22, Stewart Hildebrand wrote:
> On 4/23/25 07:08, Mykyta Poturai wrote:
>> From: Oleksandr Andrushchenko <oleksandr_andrushchenko@epam.com>
>>
>> Add support for Renesas R-Car Gen4 PCI host controller, specifically
>> targeting the S4 and V4H SoCs. The implementation includes configuration
>> read/write operations for both root and child buses. For accessing the
>> child bus, iATU is used for address translation.
>
> In a system with dom0, I assume dom0 programs the iATU, and would do so
> before issuing any PHYSDEVOP_pci_device_add hypercalls or PCI config
> space accesses. Is that a correct assumption? If so, what is the use
> case for iATU initialization in Xen? E.g. is it to guard against a
> misconfigured or broken dom0? Do different iATU settings need to be
> applied when running as a Xen guest vs not?
dom0 can program iATU, but for guests, the programming should be done by
Xen.
> Further, is this driver sufficient to initialize the PCI controller
> without dom0 involvement (e.g. to enable dom0less PCI passthrough)?
In the current state no. We are planning to add support for it in the
future together with Xen device enumeration but it would also require
changes to the bootloaders to deal with the initial host controller
setup (clocks, resets, pincontrol, etc).
> These would all be valid use cases IMO, but I would just like to
> understand the rationale.
>
>
> dw_pcie_child_{map_bus,config_read,config_write}, and thus
> dw_pcie_prog_outbound_atu{_unroll}, would potentially be called on every
> guest PCI access, where the d->pci_lock may be held. It doesn't seem
> appropriate to have a busy-wait such as this in that code path.
>
> If the busy-wait is necessary, then perhaps iATU configuration should
> happen during init/probe (as your FIXME note below suggests), not as a
> consequence of guest PCI access.
>
> Same comment applies below.
>
>
> Does iATU configuration really need to happen on every guest PCI access?
It is possible to implement smarter allocation of iATU windows, but it
would require at least a full control over the PCI host. For now we are
doing the same approach as in Linux, of using window 0 for dynamic
config space access and the rest for static mappings(configured by
dom0). But overall there will always be a need for some dynamic
configuration because there can be more devices than free ATU windows.
In practice with all of the hardware we tested on the ATU always
configured in one pass of the wait loop, and the config space accesses
are not that frequent so it shouldn't have a big impact on performance.
>> +
>> + /*
>> + * FIXME: we cannot read iATU unroll enable now as the host bridge's
>> + * HW is not yet initialized by Domain-0: leave it for later.
>> + */
>
> Is there a plan to address this?
>
We will need to address this to enable PCI scan by Xen.
--
Mykyta
^ permalink raw reply [flat|nested] 12+ messages in thread