Re: [Qemu-devel] vhost-pci and virtio-vhost-user

[Stefan Hajnoczi <stefanha@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 4228 bytes --]

On Thu, Jan 11, 2018 at 06:57:03PM +0800, Jason Wang wrote:
> 
> 
> On 2018年01月11日 00:14, Stefan Hajnoczi wrote:
> > Hi Wei,
> > I wanted to summarize the differences between the vhost-pci and
> > virtio-vhost-user approaches because previous discussions may have been
> > confusing.
> > 
> > vhost-pci defines a new virtio device type for each vhost device type
> > (net, scsi, blk).  It therefore requires a virtio device driver for each
> > device type inside the slave VM.
> > 
> > Adding a new device type requires:
> > 1. Defining a new virtio device type in the VIRTIO specification.
> > 3. Implementing a new QEMU device model.
> > 2. Implementing a new virtio driver.
> > 
> > virtio-vhost-user is a single virtio device that acts as a vhost-user
> > protocol transport for any vhost device type.  It requires one virtio
> > driver inside the slave VM and device types are implemented using
> > existing vhost-user slave libraries (librte_vhost in DPDK and
> > libvhost-user in QEMU).
> > 
> > Adding a new device type to virtio-vhost-user involves:
> > 1. Adding any new vhost-user protocol messages to the QEMU
> >     virtio-vhost-user device model.
> > 2. Adding any new vhost-user protocol messages to the vhost-user slave
> >     library.
> > 3. Implementing the new device slave.
> > 
> > The simplest case is when no new vhost-user protocol messages are
> > required for the new device.  Then all that's needed for
> > virtio-vhost-user is a device slave implementation (#3).  That slave
> > implementation will also work with AF_UNIX because the vhost-user slave
> > library hides the transport (AF_UNIX vs virtio-vhost-user).  Even
> > better, if another person has already implemented that device slave to
> > use with AF_UNIX then no new code is needed for virtio-vhost-user
> > support at all!
> > 
> > If you compare this to vhost-pci, it would be necessary to design a new
> > virtio device, implement it in QEMU, and implement the virtio driver.
> > Much of virtio driver is more or less the same thing as the vhost-user
> > device slave but it cannot be reused because the vhost-user protocol
> > isn't being used by the virtio device.  The result is a lot of
> > duplication in DPDK and other codebases that implement vhost-user
> > slaves.
> > 
> > The way that vhost-pci is designed means that anyone wishing to support
> > a new device type has to become a virtio device designer.  They need to
> > map vhost-user protocol concepts to a new virtio device type.  This will
> > be time-consuming for everyone involved (e.g. the developer, the VIRTIO
> > community, etc).
> > 
> > The virtio-vhost-user approach stays at the vhost-user protocol level as
> > much as possible.  This way there are fewer concepts that need to be
> > mapped by people adding new device types.  As a result, it will allow
> > virtio-vhost-user to keep up with AF_UNIX vhost-user and grow because
> > it's easier to work with.
> > 
> > What do you think?
> > 
> > Stefan
> 
> So a question is what's the motivation here?
> 
> Form what I'm understanding, vhost-pci tries to build a scalable V2V private
> datapath. But according to what you describe here, virito-vhost-user tries
> to make it possible to implement the device inside another VM. I understand
> the goal of vhost-pci could be done on top, but it looks to me it would then
> rather similar to the design of Xen driver domain. So I can not figure out
> how it can be done in a high performance way.

vhost-pci and virtio-vhost-user both have the same goal.  They allow
a VM to implement a vhost device (net, scsi, blk, etc).  This allows
software defined network or storage appliances running inside a VM to
provide I/O services to other VMs.  To the other VMs the devices look
like regular virtio devices.

I'm not sure I understand your reference to the Xen driver domain or
performance.  Both vhost-pci and virtio-vhost-user work using shared
memory access to the guest RAM of the other VM.  Therefore they can poll
virtqueues and avoid vmexit.  They do also support cross-VM interrupts,
thanks to QEMU setting up irqfd/ioeventfd appropriately on the host.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]