Re: [PATCH v5 07/18] vfio-user: set qdev bus callbacks for remote machine

[Stefan Hajnoczi <stefanha@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 3875 bytes --]

On Tue, Jan 25, 2022 at 09:12:28PM +0000, Jag Raman wrote:
> 
> 
> > On Jan 25, 2022, at 5:44 AM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> > 
> > On Wed, Jan 19, 2022 at 04:41:56PM -0500, Jagannathan Raman wrote:
> >> Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
> >> Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
> >> Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
> >> ---
> >> hw/remote/machine.c | 57 +++++++++++++++++++++++++++++++++++++++++++++
> >> 1 file changed, 57 insertions(+)
> >> 
> >> diff --git a/hw/remote/machine.c b/hw/remote/machine.c
> >> index 220ff01aa9..221a8430c1 100644
> >> --- a/hw/remote/machine.c
> >> +++ b/hw/remote/machine.c
> >> @@ -22,6 +22,60 @@
> >> #include "hw/pci/pci_host.h"
> >> #include "hw/remote/iohub.h"
> >> 
> >> +static bool remote_machine_get_bus(const char *type, BusState **bus,
> >> +                                   Error **errp)
> >> +{
> >> +    ERRP_GUARD();
> >> +    RemoteMachineState *s = REMOTE_MACHINE(current_machine);
> >> +    BusState *root_bus = NULL;
> >> +    PCIBus *new_pci_bus = NULL;
> >> +
> >> +    if (!bus) {
> >> +        error_setg(errp, "Invalid argument");
> >> +        return false;
> >> +    }
> >> +
> >> +    if (strcmp(type, TYPE_PCI_BUS) && strcmp(type, TYPE_PCI_BUS)) {
> >> +        return true;
> >> +    }
> >> +
> >> +    root_bus = qbus_find_recursive(sysbus_get_default(), NULL, TYPE_PCIE_BUS);
> >> +    if (!root_bus) {
> >> +        error_setg(errp, "Unable to find root PCI device");
> >> +        return false;
> >> +    }
> >> +
> >> +    new_pci_bus = pci_isol_bus_new(root_bus, type, errp);
> >> +    if (!new_pci_bus) {
> >> +        return false;
> >> +    }
> >> +
> >> +    *bus = BUS(new_pci_bus);
> >> +
> >> +    pci_bus_irqs(new_pci_bus, remote_iohub_set_irq, remote_iohub_map_irq,
> >> +                 &s->iohub, REMOTE_IOHUB_NB_PIRQS);
> >> +
> >> +    return true;
> >> +}
> > 
> > Can the user create the same PCI bus via QMP commands? If so, then this
> 
> I think there is a way we could achieve it.
> 
> When I looked around, both the command line and the QMP didn’t have a direct
> way to create a bus. However, there are some indirect ways. For example, the
> TYPE_LSI53C895A device creates a SCSI bus to attach SCSI devices. Similarly,
> there are some special PCI devices like TYPE_PCI_BRIDGE which create a
> secondary PCI bus.
> 
> Similarly, we could implement a PCI device that creates a PCI bus with
> isolated address spaces.

Exactly. device_add instantiates DeviceStates, not busses, so there
needs to be a parent device like a SCSI controller, a PCI bridge, etc
that owns and creates the bus.

> > is just a convenience that saves the extra step. Or is there some magic
> > that cannot be done via QMP device_add?
> > 
> > I'm asking because there are 3 objects involved and I'd like to
> > understand the lifecycle/dependencies:
> > 1. The PCIDevice we wish to export.
> > 2. The PCIBus with isolated address spaces that contains the PCIDevice.
> > 3. The vfio-user server that exports a given PCIDevice.
> > 
> > Users can already create the PCIDevice via hotplug and the vfio-user
> > server via object-add. So if there's no magic they could also create the
> > PCI bus:
> > 1. device_add ...some PCI bus stuff here...,id=isol-pci-bus0
> > 2. device_add ...the PCIDevice...,bus=isol-pci-bus0,id=mydev
> > 3. object-add x-vfio-user-server,device=mydev
> 
> We are able to do 2 & 3 already. We could introduce a PCI device that
> creates an isolated PCI bus. That would cover step 1 outlined above.

I wonder if a new device is needed or whether it's possible to add an
isol_as=on|off (default: off) option to an existing PCI bridge/expander?
Hopefully most of the code is already there.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]