Re: [RFC PATCH 00/13] Add a plugin to support out-of-band live migration for VFIO pass-through device

[Alex Williamson <alex.williamson@redhat.com>]

On Wed, 1 Jun 2022 17:09:25 +0000
"Dong, Eddie" <eddie.dong@intel.com> wrote:

> > -----Original Message-----
> > From: Qemu-devel <qemu-devel-  
> > bounces+eddie.dong=intel.com@nongnu.org> On Behalf Of Alex Williamson  
> > On Tue, 24 May 2022 14:18:35 +0800
> > Lei Rao <lei.rao@intel.com> wrote:
> > > This proposal adopts a plugin mechanism (an example can be found in
> > > [1]) given that IPU/DPU vendors usually implement proprietary
> > > migration interfaces without a standard. But we are also open if an
> > > alternative option makes better sense, e.g. via loadable modules (with
> > > Qemu supporting gRPC or JSON-RPC support) or an IPC mechanism similar  
> > to vhost-user.
> > 
> > AFAIU, QEMU is not interested in supporting plugin modules, especially
> > proprietary ones.  I don't see that a case has really been made that this
> > cannot be done in-band, through a vfio-pci variant driver, possibly making
> > use of proprietary interfaces to a userspace agent if necessary (though
> > please don't ask such to be accepted in-tree for the kernel either).  A vfio-
> > user device server might also host such proprietary, device specific agents
> > while supporting the standard, in-band migration interface.  Thanks,
> >   
> 
> Thanks Alex. Removing plug-in module is not a problem.
> 
> Do you mean to implement the migration and protocol handling inside
> Qemu-client (probably vfio-client after the VFIO-user is merged)? Or
> to build as part of libvfio-user? We can also build it as a separate
> process of Qemu-server as part of Multi-Process Qemu.

AIUI, the QEMU "client" in a vfio-user configuration is simply QEMU
itself.  The vfio-user "server" provides the actual device
implementation, which could support different license models, depending
on what libraries or existing code is incorporated to implement that
server.  The QEMU remote machine type is simply a QEMU-based
implementation of a vfio-user server.  The vfio-user server is analogous
to a vfio-pci variant driver in the kernel/ioctl interface model.  The
vfio-user client should be device agnostic, possibly with similar
exceptions we have today via device specific quirk handling for the
vfio kernel interface.

> In here, the protocol between host CPU and SoC is based on gRPC,
> which support Rust code in client (Host CPU side here) more than
> C/C++. Do you have any suggestion to better support Rust code with
> Qemu C/C++ code? 

I'm not qualified to provide suggestions regarding Rust code
integration with QEMU, but I think that's only required if the device
specific migration support is on the "client".  As above, I don't think
that's the correct model, the vfio migration protocol is meant to
support any device specific requirements on the device end of the
connection, ie. the "server" end for vfio-user, which can be an
entirely separate, non-QEMU based process.  I think there are also ways
to write kernel drivers in Rust, so possibly a kernel interface
vfio-pci variant driver could also work.  Thanks,

Alex