Re: [Qemu-devel] [edk2] Could not add PCI device with big memory to aarch64 VMs

[Laszlo Ersek <lersek@redhat.com>]

On 12/01/15 01:46, liang yan wrote:
> Hello, Laszlo,
> 
> On 11/30/2015 03:05 PM, Laszlo Ersek wrote:

[snip]

>> If you need more room (with large alignments), then there's no way
>> around supporting QEMU's 64 bit aperture, VIRT_PCIE_MMIO_HIGH (see my
>> earlier email).
> I checked the function create_pcie form pathtoqemu/hw/arm/virt.c, it has
> a flag value use_highmem(which has default "true" value).
> 
> It set base_mmio_high and size_mmio_high to device tree by function below,
> 
>         qemu_fdt_setprop_sized_cells(vbi->fdt, nodename, "ranges",
>                                      1, FDT_PCI_RANGE_IOPORT, 2, 0,
>                                      2, base_pio, 2, size_pio,
>                                      1, FDT_PCI_RANGE_MMIO, 2, base_mmio,
>                                      2, base_mmio, 2, size_mmio,
>                                      1, FDT_PCI_RANGE_MMIO_64BIT,
>                                      2, base_mmio_high,
>                                      2, base_mmio_high, 2, size_mmio_high);
> 
> So basically, I need to add two UINT64 variables like mmio_high_base and
> mmio_high_size to PCD under function ProcessPciHost(VirtFdtDxe.c),
> and try to use this high base address and size as new aperture.
> 
> Is this correct?

It is correct, but that's only part of the story.

Parsing the 64-bit aperture from the DTB into new PCDs in
ArmVirtPkg/VirtFdtDxe is the easy part.

The hard part is modifying ArmVirtPkg/PciHostBridgeDxe, so that BAR
allocation requests (submitted by the platform independent PCI bus
driver that resides in "MdeModulePkg/Bus/Pci/PciBusDxe") are actually
serviced from this high aperture too.

> 
>> Unfortunately I can't readily help with that in the
>> "ArmVirtPkg/PciHostBridgeDxe" driver; there's no such (open-source)
>> example in the edk2 tree. Of course, I could experiment with it myself
>> -- only not right now.
> If possible, I do want to finish this part or help you finish it. I just
> work on UEFI recently, and thank you so much for your patient and detail
> explanation. I really appreciate it.
>> I guess copying and adapting the TypeMem32 logic to TypeMem64 (currently
>> short-circuited with EFI_ABORTED) could work.
> 
> Is the 32 or 64 bit determined by BAR(2-3bit) or by the PCI device
> memory size? Is there an option from QEMU?

I can't tell. :)

> Does TypeMem32 still keep  "VIRT_PCIE_MMIO" aperture and TypeMem64 use
> "VIRT_PCIE_MMIO_HIGH" aperture? or It's more like device property
> controlled from QEMU device simulation?

Good question. I don't know. I think in order to answer this question,
we should understand the whole dance between the PCI root bridge / host
bridge driver and the generic PCI bus driver.

The documentation I know of is in the Platform Init spec, version 1.4,
Volume 5, Chapter 10 "PCI Host Bridge". I've taken multiple stabs at
that chapter earlier, but I've always given up.

Sorry I can't help more, but this is new area for me as well.

Laszlo