Re: [kvm-unit-tests PATCH v7 06/13] pci: Rework pci_bar_addr()

[Peter Xu <peterx@redhat.com>]

On Fri, Oct 14, 2016 at 02:37:25PM +0200, Alexander Gordeev wrote:

[...]

> So pci-host-generic.c is "BIOS or something" in this context. There is
> no contradiction with the your excerpt. Please note "Figure 1-2: PCI
> System Block Diagram" in the spec. The CPU and PCI busses are two
> physically different busses interconnected by a bridge.
> 
> A CPU data access to PCI devices initiates data access on a PCI bus,
> but adresses seen by the CPU and PCI devices do not necessarily match.
> 
> So in case of x86 they do (lib/x86/asm/pci.h):
> 
> static inline
> phys_addr_t pci_translate_addr(pcidevaddr_t dev __unused, uint64_t addr)
> {
> 	return addr;
> }
> 
> But in case of ARM they might not (lib/asm-generic/pci-host-bridge.h):
> 
> static inline
> phys_addr_t pci_translate_addr(pcidevaddr_t dev __unused, uint64_t addr)
> {
> 	/*
> 	 * Assume we only have single PCI host bridge in a system.
> 	 */
> 	return pci_host_bridge_get_paddr(addr);
> }
> 
> > If you see PCI Local Bus Spec Rev 3.0, chap 6.2.5.1:
> > 
> >     Power-up software needs to build a consistent address map before
> >     booting the machine to an operating system. This means it has to
> >     determine how much memory is in the system, and how much address
> >     space the I/O controllers in the system require. After determining
> >     this information, power-up software can map the I/O controllers
> >     into reasonable locations and proceed with system boot. In order
> >     to do this mapping in a device independent manner, the base
> >     registers for this mapping are placed in the predefined header
> >     portion of Configuration Space.
> > 
> > BARs for each PCI devices should be pre-allocated during power-up
> > software, and a consistent map is built with the knowledge of existing
> > RAM in the system.
> 
> Yep, see how pci_probe() / pci_alloc_resource() allocate and assign
> BARs to each device. It is the case for PPC/ARM but in case of x86
> it is BIOS who does it.
> 
> > If you boot a VM with/without IOMMU, you'll see that BAR addresses
> > won't change before/after enabling IOMMU.
> 
> BAR addresses are in PCI bus address space. It sounds quite logical.
> (I am not experienced with IOMMU address translation, though).

Thank you for the thorough explanation. I am still not sure I fully
understand the whole thing, but looks like the translation here is not
the same as x86 IOMMU address translations, and x86 is special in this
case (BAR addresses should be the same as physical addresses in x86,
while not for ARM/PPC).

I am sorry for the noise. Looking forward to your next version. Thanks!

-- peterx