[Qemu-devel] PCIe Transaction handling in Qemu

[Qemu-devel] PCIe Transaction handling in Qemu

[Adnan Khaleel]

[-- Attachment #1: Type: text/plain, Size: 3276 bytes --]

Hello, 


I have a question regarding how Qemu PCIe devices handle Config Transactions vs Memory Transactions (assuming the PCI device is setup to act as PCI_BASE_ADDRESS_SPACE_MEMORY).


I'm using portions of hw/cirrus_vga.c to make my point,



static PCIDeviceInfo cirrus_vga_info = {
    .qdev.name    = "cirrus-vga",
    .qdev.desc    = "Cirrus CLGD 54xx VGA",
    .qdev.size    = sizeof(PCICirrusVGAState),
    .qdev.vmsd    = &vmstate_pci_cirrus_vga,
    .init         = pci_cirrus_vga_initfn,
    .romfile      = VGABIOS_CIRRUS_FILENAME,
    .config_write = pci_cirrus_write_config,
};


PCIDeviceInfo allows for custom .config_write (& config_read) handler as shown above. Any pci config operations operations initiated via legacy I/O operations will use these config handlers.


The MMIO regions and handlers are mapped as shown below:



static uint32_t cirrus_vga_mem_readb(void *opaque, target_phys_addr_t addr)
{
:
} and so on for the other mmio handlers




static CPUReadMemoryFunc * const cirrus_vga_mem_read[3] = {
    cirrus_vga_mem_readb,
    cirrus_vga_mem_readw,
    cirrus_vga_mem_readl,
};


static CPUWriteMemoryFunc * const cirrus_vga_mem_write[3] = {
    cirrus_vga_mem_writeb,
    cirrus_vga_mem_writew,
    cirrus_vga_mem_writel,
};


static void cirrus_init_common(CirrusVGAState * s, int device_id, int is_pci)

{
:
:

    s->vga.vga_io_memory = cpu_register_io_memory(cirrus_vga_mem_read,
                                                  cirrus_vga_mem_write, s);
:
}



static void cirrus_pci_mmio_map(PCIDevice *d, int region_num,
    pcibus_t addr, pcibus_t size, int type)
{
    CirrusVGAState *s = &DO_UPCAST(PCICirrusVGAState, dev, d)->cirrus_vga;


    cpu_register_physical_memory(addr, CIRRUS_PNPMMIO_SIZE,
     s->cirrus_mmio_io_addr);
}



static int pci_cirrus_vga_initfn(PCIDevice *dev)
{
:
:
     cirrus_init_common(..)
:
     if (device_id == CIRRUS_ID_CLGD5446) {
         pci_register_bar((PCIDevice *)d, 1, CIRRUS_PNPMMIO_SIZE,
                          PCI_BASE_ADDRESS_SPACE_MEMORY, cirrus_pci_mmio_map);
     }
     return 0;
}


I have some questions about PCIe operations sssuming the device has MMIO handlers involved (as shown above).
1. Will all PCIe config operations ALWAYS use the installed config handlers? Or can PCIe config operations use the MMIO handlers?
2. Assuming that both PCI config and MMIO operations can use the MMIO handlers, is there any way I can identify if a transaction is a config or a memory transaction?
3.a. What address is passed on the MMIO handlers for config and MMIO operations? From pci_data_write in pci_host.c, it appears that config operations send only the offset into the config region. I couldn't determine what address is passed for MMIO operations.
   b. Is it an offset from the BAR for MMIO operations?
   c. How do I get the full physical address?
   d. What address does a PCIe device expect to see - physical or offset for?
   e. Is there anyway I can find out what the bus and device numbers are once inside the config and MMIO handlers? i.e once the execution has reached the pci_cirrus_write_config() or cirrus_vga_mem_readb(..) from the code above?


Thanks


Adnan

[-- Attachment #2: Type: text/html, Size: 7964 bytes --]

[Qemu-devel] Re: PCIe Transaction handling in Qemu

[Isaku Yamahata]

On Tue, Dec 21, 2010 at 02:24:29PM -0600, Adnan Khaleel wrote:
> Hello, 

Hi.


> I have a question regarding how Qemu PCIe devices handle Config Transactions vs
> Memory Transactions (assuming the PCI device is setup to act
> as PCI_BASE_ADDRESS_SPACE_MEMORY).
> 
> I'm using portions of hw/cirrus_vga.c to make my point,

If you can send out what you have instead of mimicked
example, it would help to figure out what you are trying to do.


> I have some questions about PCIe operations sssuming the device has MMIO
> handlers involved (as shown above).
> 1. Will all PCIe config operations ALWAYS use the installed config handlers? Or
> can PCIe config operations use the MMIO handlers?

MMIO on MMCONFIG area are routed to write/read config handler.
On the other hand MMIO on memory BAR is routed to mmio hanlder you pictured.
NOTE: the upstream qemu lacks q35 chipset support, so guest can NOT do
      MMIO on MMCONFIG area.

> 2. Assuming that both PCI config and MMIO operations can use the MMIO handlers,
> is there any way I can identify if a transaction is a config or a memory
> transaction?
> 3.a. What address is passed on the MMIO handlers for config and MMIO
> operations? From pci_data_write in pci_host.c, it appears that config
> operations send only the offset into the config region. I couldn't determine
> what address is passed for MMIO operations.
>    b. Is it an offset from the BAR for MMIO operations?
>    c. How do I get the full physical address?
>    d. What address does a PCIe device expect to see - physical or offset for?
>    e. Is there anyway I can find out what the bus and device numbers are once
> inside the config and MMIO handlers? i.e once the execution has reached
> the pci_cirrus_write_config() or cirrus_vga_mem_readb(..) from the code above?

offset in configuration space of each pcie function is passed to
write/read config handler
physical address is passed to mmio handler of memory BAR.
-- 
yamahata

Re: [Qemu-devel] PCIe Transaction handling in Qemu

[Paul Brook]

> I have some questions about PCIe operations sssuming the device has MMIO
> handlers involved (as shown above).

> 1. Will all PCIe config operations
> ALWAYS use the installed config handlers? Or can PCIe config operations
> use the MMIO handlers? 

Access to PCI config space is provided by the PCI host bridge. It has nothing 
to do with any memory BARs the device may have.  The host bridge may expose 
this in any way it chooses, including but not limited to ISA IO ports or a 
memory mapped region of its own.  Ether way the device doesn't care.

> 2. Assuming that both PCI config and MMIO
> operations can use the MMIO handlers, is there any way I can identify if a
> transaction is a config or a memory transaction?

Incorrect assumption. Memory and Config accesses ae completely separate.

> 3.a. What address is
> passed on the MMIO handlers for config and MMIO operations? From
> pci_data_write in pci_host.c, it appears that config operations send only
> the offset into the config region. I couldn't determine what address is
> passed for MMIO operations. b. Is it an offset from the BAR for MMIO
> operations?

Th offset from the start of the region.

>    c. How do I get the full physical address?

You don't.  "Full physical address" is a fairly ill defined term. Physical 
addresses are local to a particular bus. It's common for CPU/ram and each PCI 
bus to have completely independent physical address spaces, with the host 
bridge providing mapping between the two.

>    d. What address does a PCIe device expect to see - physical or offset
> for? 

Offset. Old versions of qemu used to pass the cpu physical address. This was a 
bug.

> e. Is there anyway I can find out what the bus and device numbers are
> once inside the config and MMIO handlers? i.e once the execution has
> reached the pci_cirrus_write_config() or cirrus_vga_mem_readb(..) from the
> code above?

No. The device does not, and should not know this. 

Paul