Re: [RFC] Proposal of QEMU PCI Endpoint test environment

Manivannan Sadhasivam
2023-08-23T06:09:45Z

On Fri, Aug 18, 2023 at 10:46:02PM +0900, Shunsuke Mie wrote:
> Hi all,
>
> We are proposing to add a new test syste to Linux for PCIe Endpoint. That
> can be run on QEMU without real hardware. At present, partially we have
> confirmed that pci-epf-test is working, but it is not yet complete.
> However, we would appreciate your comments on the architecture design.
>
> # Background
> The background is as follows.
>
> PCI Endpoint function driver is implemented using the PCIe Endpoint
> framework, but it requires physical boards for testing, and it is difficult
> to test sufficiently. In order to find bugs and hardware-dependent
> implementations early, continuous testing is required. Since it is
> difficult to automate tests that require hardware, this RFC proposes a
> virtual environment for testing PCI endpoint function drivers.
>

This sounds exciting to me and yes, it is going to be really helpful for
validating EP framework as a whole.

> # Architecture
> The overview of the architecture is as follows.
>
> Guest 1 Guest 2
> +-------------------------+ +----------------------------+
> | Linux kernel | | Linux kernel |
> | | | |
> | PCI EP function driver | | |
> | (e.g. pci-epf-test) | | |
> |-------------------------| | PCI Device Driver |
> | (2) QEMU EPC Driver | | (e.g. pci_endpoint_test) |
> +-------------------------+ +----------------------------+
> +-------------------------+ +----------------------------+
> | QEMU | | QEMU |
> |-------------------------| |----------------------------|
> | (1) QEMU PCI EPC Device *----* (3) QEMU EPF Bridge Device |
> +-------------------------+ +----------------------------+
>
> At present, it is designed to work guests only on the same host, and
> communication is done through Unix domain sockets.
>
> The three parts shown in the figure were introduced this time.
>
> (1) QEMU PCI Endpoint Controller(EPC) Device
> PCI Endpoint Controller implemented as QEMU PCI device.
> (2) QEMU PCI Endpoint Controller(EPC) Driver
> Linux kernel driver that drives the device (1). It registers a epc device
> to linux kernel and handling each operations for the epc device.
> (3) QEMU PCI Endpoint function(EPF) Bridge Device
> QEMU PCI device that cooperates with (1) and performs accesses to pci
> configuration space, BAR and memory space to communicate each guests, and
> generates interruptions to the guest 1.
>

I'm not very familiar with Qemu, but why can't the existing Qemu PCIe host
controller devices used for EP communication? I mean, what is the need for a
dedicated EPF bridge device (3) in host? (Guest 2 as per your diagram).

Is that because you use socket communication between EP and host?

- Mani

> Each projects are:
> (1), (3) https://github.com/ShunsukeMie/qemu/tree/epf-bridge/v1
> files: hw/misc/{qemu-epc.{c,h}, epf-bridge.c}
> (2) https://github.com/ShunsukeMie/linux-virtio-rdma/tree/qemu-epc
> files: drivers/pci/controller/pcie-qemu-ep.c
>
> # Protocol
>
> PCI, PCIe has a layer structure that includes Physical, Data Lane and
> Transaction. The communicates between the bridge(3) and controller (1)
> mimic the Transaction. Specifically, a protocol is implemented for
> exchanging fd for communication protocol version check and communication,
> in addition to the interaction equivalent to PCIe Transaction Layer Packet
> (Read and Write of I/O, Memory, Configuration space and Message). In my
> mind, we need to discuss the communication mor.
>
> We also are planning to post the patch set after the code is organized and
> the protocol discussion is matured.
>
> Best regards,
> Shunsuke

--
மணிவண்ணன் சதாசிவம்