Re: RFC: New device for zero-copy VM memory access

[geoff@hostfission.com]

Another update to this that adds support for unmap notification. The 
device
has also been renamed to `porthole` and now resides here:

https://github.com/gnif/qemu/blob/master/hw/misc/porthole.c

And here is the updated Linux test client.

https://gist.github.com/gnif/77e7fb54604b42a1a98ecb8bf3d2cf46

-Geoff

On 2019-10-31 13:55, geoff@hostfission.com wrote:
> Hi Dave,
> 
> On 2019-10-31 05:52, Dr. David Alan Gilbert wrote:
>> * geoff@hostfission.com (geoff@hostfission.com) wrote:
>>> Hi All,
>>> 
>>> Over the past week, I have been working to come up with a solution to 
>>> the
>>> memory transfer performance issues that hinder the Looking Glass 
>>> Project.
>>> 
>>> Currently Looking Glass works by using the IVSHMEM shared memory 
>>> device
>>> which
>>> is fed by an application that captures the guest's video output. 
>>> While this
>>> works it is sub-optimal because we first have to perform a CPU copy 
>>> of the
>>> captured frame into shared RAM, and then back out again for display. 
>>> Because
>>> the destination buffers are allocated by closed proprietary code 
>>> (DirectX,
>>> or
>>> NVidia NvFBC) there is no way to have the frame placed directly into 
>>> the
>>> IVSHMEM shared ram.
>>> 
>>> This new device, currently named `introspection` (which needs a more
>>> suitable
>>> name, porthole perhaps?), provides a means of translating guest 
>>> physical
>>> addresses to host virtual addresses, and finally to the host offsets 
>>> in RAM
>>> for
>>> file-backed memory guests. It does this by means of a simple protocol 
>>> over a
>>> unix socket (chardev) which is supplied the appropriate fd for the 
>>> VM's
>>> system
>>> RAM. The guest (in this case, Windows), when presented with the 
>>> address of a
>>> userspace buffer and size, will mlock the appropriate pages into RAM 
>>> and
>>> pass
>>> guest physical addresses to the virtual device.
>> 
>> Hi Geroggrey,
>>   I wonder if the same thing can be done by using the existing 
>> vhost-user
>> mechanism.
>> 
>>   vhost-user is intended for implementing a virtio device outside of 
>> the
>> qemu process; so it has a character device that qemu passes commands 
>> down
>> to the other process, where qemu mostly passes commands via the virtio
>> queues.   To be able to read the virtio queues, the external process
>> mmap's the same memory as the guest - it gets passed a 'set mem table'
>> command by qemu that includes fd's for the RAM, and includes 
>> base/offset
>> pairs saying that a particular chunk of RAM is mapped at a particular
>> guest physical address.
>> 
>>   Whether or not you make use of virtio queues, I think the mechanism
>> for the device to tell the external process the mappings might be what
>> you're after.
>> 
>> Dave
>> 
> 
> While normally I would be all for re-using such code, the vhost-user 
> while
> being very feature-complete from what I understand is overkill for our
> requirements. It will still allocate a communication ring and an events 
> system
> that we will not be using. The goal of this device is to provide a dumb 
> &
> simple method of sharing system ram, both for this project and for 
> others that
> work on a simple polling mechanism, it is not intended to be an 
> end-to-end
> solution like vhost-user is.
> 
> If you still believe that vhost-user should be used, I will do what I 
> can to
> implement it, but for such a simple device I honestly believe it is 
> overkill.
> 
> -Geoff
> 
>>> This device and the windows driver have been designed in such a way 
>>> that
>>> it's a
>>> utility device for any project and/or application that could make use 
>>> of it.
>>> The PCI subsystem vendor and device ID are used to provide a means of 
>>> device
>>> identification in cases where multiple devices may be in use for 
>>> differing
>>> applications. This also allows one common driver to be used for any 
>>> other
>>> projects wishing to build on this device.
>>> 
>>> My ultimate goal is to get this to a state where it could be accepted
>>> upstream
>>> into Qemu at which point Looking Glass would be modified to use it 
>>> instead
>>> of
>>> the IVSHMEM device.
>>> 
>>> My git repository with the new device can be found at:
>>> https://github.com/gnif/qemu
>>> 
>>> The new device is:
>>> https://github.com/gnif/qemu/blob/master/hw/misc/introspection.c
>>> 
>>> Looking Glass:
>>> https://looking-glass.hostfission.com/
>>> 
>>> The windows driver, while working, needs some cleanup before the 
>>> source is
>>> published. I intend to maintain both this device and the windows 
>>> driver
>>> including producing a signed Windows 10 driver if Redhat are 
>>> unwilling or
>>> unable.
>>> 
>>> Kind Regards,
>>> Geoffrey McRae
>>> 
>>> HostFission
>>> https://hostfission.com
>>> 
>> --
>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK