Re: [Qemu-devel] PXB fixes for QEMU, and extra root buses for OVMF

[Marcel Apfelbaum <marcel@redhat.com>]

On 06/06/2015 02:45 AM, Laszlo Ersek wrote:
> Following up on this cross-posted message, I will send two patch sets,
> one for QEMU (to qemu-devel) and another for OVMF (to edk2-devel). With
> both in place, OVMF supports multiple PCI root buses.
Hi Laszlo,
Thank you for the patches.

>
> Below I'm writing up the way I tested the feature, plus a few random
> notes.
>
> (1) Interrupt line assignments. I patched SeaBIOS (temporarily) to print
>      interrupt line assignments, and I also patched OVMF (permanently) to
>      print the same. (See both patches in the OVMF patch
>
>        OvmfPkg: PlatformBdsLib: debug log interrupt line assignments
>
>      in one of the followup series; the commit message contains the
>      ad-hoc SeaBIOS patch.) The QEMU command line was:
>
>      qemu-system-x86_64 \
>        -m 2048 \
>        -M pc \
>        -enable-kvm \
>        -device qxl-vga \
>        \
>        $FIRMWARE_OPTIONS \
>        \
>        -drive id=cdrom,if=none,readonly,format=raw,file=$ISO \
>        -device virtio-scsi-pci,id=scsi0 \
>        -device scsi-cd,bus=scsi0.0,drive=cdrom,bootindex=0 \
>        \
>        -debugcon file:debug.log \
>        -global isa-debugcon.iobase=0x402 \
>        \
>        -monitor stdio \
>        \
>        -device pxb,id=bridge1,bus_nr=4 \
>        \
>        -netdev user,id=netdev0,hostfwd=tcp:127.0.0.1:2222-:22 \
>        -device e1000,netdev=netdev0,bus=bridge1,addr=2,romfile= \
>        \
>        -netdev user,id=netdev1,hostfwd=tcp:127.0.0.1:2223-:22 \
>        -device e1000,netdev=netdev1,bus=bridge1,addr=3,romfile= \
>        \
>        -netdev user,id=netdev2,hostfwd=tcp:127.0.0.1:2224-:22 \
>        -device e1000,netdev=netdev2,bus=bridge1,addr=4,romfile= \
>        \
>        -netdev user,id=netdev3,hostfwd=tcp:127.0.0.1:2225-:22 \
>        -device e1000,netdev=netdev3,bus=bridge1,addr=5,romfile=
>
>      The ISO variable pointed to a Fedora 20 LiveCD (although it is not
>      relevant for this test). FIRMWARE_OPTIONS were
>
>        -bios .../out/bios.bin
>
>      for the (debug-patched, see above) SeaBIOS test case, and
>
>        -drive if=pflash,readonly,format=raw,file=.../OVMF_CODE.fd \
>        -drive if=pflash,format=raw,file=.../COPY_OF_OVMF_VARS.fd
>
>      for the OVMF test case.
>
>      As you can see the QEMU command line places four e1000 NICs on one
>      PXB (I used e1000 because that's what Marcel recommended for all
>      testing). SeaBIOS then printed
>
>        PCI: init bdf=00:01.3 id=8086:7113
>        assigned irq line 10
>
>        PCI: init bdf=00:02.0 id=1b36:0100
>        assigned irq line 10
>
>        PCI: init bdf=00:03.0 id=1af4:1004
>        assigned irq line 11
>
>        PCI: init bdf=04:00.0 id=1b36:0001
>        assigned irq line 11
>
>        PCI: init bdf=05:02.0 id=8086:100e
>        assigned irq line 10
>
>        PCI: init bdf=05:03.0 id=8086:100e
>        assigned irq line 11
>
>        PCI: init bdf=05:04.0 id=8086:100e
>        assigned irq line 11
>
>        PCI: init bdf=05:05.0 id=8086:100e
>        assigned irq line 10
>
>      whereas OVMF printed
>
>        SetPciIntLine: PciRoot(0x0)/Pci(0x1,0x3) -> 0x0A
>        SetPciIntLine: PciRoot(0x0)/Pci(0x2,0x0) -> 0x0A
>        SetPciIntLine: PciRoot(0x0)/Pci(0x3,0x0) -> 0x0B
>        SetPciIntLine: PciRoot(0x4)/Pci(0x0,0x0) -> 0x0B
>        SetPciIntLine: PciRoot(0x4)/Pci(0x0,0x0)/Pci(0x2,0x0) -> 0x0A
>        SetPciIntLine: PciRoot(0x4)/Pci(0x0,0x0)/Pci(0x3,0x0) -> 0x0B
>        SetPciIntLine: PciRoot(0x4)/Pci(0x0,0x0)/Pci(0x4,0x0) -> 0x0B
>        SetPciIntLine: PciRoot(0x4)/Pci(0x0,0x0)/Pci(0x5,0x0) -> 0x0A
>
>      (Note that the bus number assigned to the devices on the pxb is 5 in
>      the OVMF case too. This is not visible from the device paths logged
>      above, but it is clear from edk2's PCI bus driver's log, and the
>      output of the "PCI" UEFI shell command.)
>
>      Therefore I concluded that the IRQ line assignment logic in OVMF
>      that Gabriel had contributed earlier (and that I updated very
>      slightly in the OVMF patchset here) continued to work correctly.
>
> (2) Marcel, no new fw_cfg file is necessary for exposing the bus numbers
>      of the extra root buses to OVMF. OVMF probes all buses and all
>      devices (function 0) to locate the extra root buses. This procedure
>      is shortened by adhering to the "etc/extra-pci-roots" fw_cfg file,
>      which is already there.
Sure. It would not be a problem to add it, but if there is no need for it...

>
>      This recommendation came from both Michael & Marcel, and it was
>      "surprisingly easy" to implement with edk2's PciLib.
>
> (3) We discussed earlier the idea to flip the PXB's _HID and _UID
>      objects to numeric values in QEMU; I implemented and tested that
>      (see more test cases below).
>
> (4) Ping test was successful from the OVMF-booted Fedora 20 LivecD
>      environment, using one e1000 NIC on a PXB.
>
> (5) Ping test was successful from the UEFI shell, using Intel's
>      proprietary UEFI driver for the e1000 NIC (called PROEFI). The NIC
>      was located on the same PXB as in the previous point.
>
> (6) Successfully loaded public web pages with Firefox in an OVMF-booted
>      Windows Server 2012 R2 guest. The NIC was located on the same PXB as
>      in the previous point.
>
> (7) At least one issue remains to be solved (designed) in QEMU, for both
>      SeaBIOS's and OVMF's sake: booting off devices that are located on
>      the PXB. The problem is with the "bootorder" fw_cfg file. Consider
>      the following example:
>
>        /pci@i0cf8/scsi@3/channel@0/disk@0,0
>        /pci/pci-bridge@0/ethernet@2/ethernet-phy@0
>
>      which is generated for the options
>
>        -device virtio-scsi-pci,id=scsi0 \
>        -device scsi-cd,bus=scsi0.0,drive=cdrom,bootindex=0 \
>        \
>        -device pxb,id=bridge1,bus_nr=4 \
>        -device e1000,netdev=netdev0,bus=bridge1,addr=2,romfile=,bootindex=1
>
>      While the first entry is recognized by both SeaBIOS and OVMF, the
>      second entry (generated for the NIC hanging off the PXB, see above)
>      is recognized by neither. (I tested OVMF, and investigated the
>      SeaBIOS source, for this claim.)
>
>      For the SeaBIOS explanation, grep the source code for FW_PCI_DOMAIN.
Thanks for bringing it to my attention.

>
>      The OVMF explanation is that OVMF simply rejects the initial
>      OpenFirmware device path node "/pci" with a controlled parse error
>      (as opposed to the "/pci@i0cf8" node, which it recognizes and
>      translates to UEFI in combination with the rest of that OFW device
>      path).
>
>      The "/pci" node comes from QEMU's sysbus_get_fw_dev_path() function,
>      file "hw/core/sysbus.c", where *neither* of the (s->num_mmio) and
>      (s->num_pio) branches apply. (The (s->num_pio) branch applies for
>      the first entry, ie. "/pci@i0cf8".)
>
>      Something has to be invented here to clue in both firmwares as to
>      the root bus number (here bus_nr=4), in a format that is compliant
>      with the "OpenFirmware unit address" concept. (Note that
>      "/pci-bridge@0" only gives away the slot number *on* the extra root
>      bus, not the number of the root bus itself.) For example:
>
>        /pci@rootbus4/pci-bridge@0/ethernet@2/ethernet-phy@0
>
>      would be acceptable. However, I don't know how to implement this in
>      sysbus_get_fw_dev_path().
I'll look into it. What is the OpenFirmware unit address" concept ? :)


Thanks,
Marcel

>
> Thanks
> Laszlo
>