[RFC 0/8] ioregionfd introduction

[Elena Ufimtseva <elena.ufimtseva@oracle.com>]

This patchset is an RFC version for the ioregionfd implementation
in QEMU. The kernel patches are to be posted with some fixes as a v4.

For this implementation version 3 of the posted kernel patches was user:
https://lore.kernel.org/kvm/cover.1613828726.git.eafanasova@gmail.com/

The future version will include support for vfio/libvfio-user.
Please refer to the design discussion here proposed by Stefan:
https://lore.kernel.org/all/YXpb1f3KicZxj1oj@stefanha-x1.localdomain/T/

The vfio-user version needed some bug-fixing and it was decided to send
this for multiprocess first.

The ioregionfd is configured currently trough the command line and each
ioregionfd represent an object. This allow for easy parsing and does
not require device/remote object command line option modifications.

The following command line can be used to specify ioregionfd:
<snip>
  '-object', 'x-remote-object,id=robj1,devid=lsi0,fd='+str(remote.fileno()),\
  '-object', 'ioregionfd-object,id=ioreg2,devid=lsi0,iofd='+str(iord.fileno())+',bar=1',\
  '-object', 'ioregionfd-object,id=ioreg3,devid=lsi0,iofd='+str(iord.fileno())+',bar=2',\
</snip>

Proxy side of ioregionfd in this version uses only one file descriptor:
<snip>
  '-device', 'x-pci-proxy-dev,id=lsi0,fd='+str(proxy.fileno())+',ioregfd='+str(iowr.fileno()), \
</snip>

This is done for RFC version and my though was that next version will
be for vfio-user, so I have not dedicated much effort to this command
line options.

The multiprocess messaging protocol was extended to support inquiries
by the proxy if device has any ioregionfds.
This RFC implements inquires by proxy about the type of BAR (ioregionfd
or not) and the type of it (memory/io).

Currently there are few limitations in this version of ioregionfd.
 - one ioregionfd per bar, only full bar size is supported;
 - one file descriptor per device for all of its ioregionfds;
 - each remote device runs fd handler for all its BARs in one IOThread;
 - proxy supports only one fd.

Some of these limitations will be dropped in the future version.
This RFC is to acquire the feedback/suggestions from the community
on the general approach.

The quick performance test was done for the remote lsi device with
ioregionfd and without for both mem BARs (1 and 2) with help
of the fio tool:

Random R/W:

	             read IOPS	read BW     write IOPS   write BW
no ioregionfd	 889	    3559KiB/s   890          3561KiB/s
ioregionfd	     938	    3756KiB/s   939          3757KiB/s


Sequential Read and Sequential Write:

                 Sequential read		Sequential write	
                 read IOPS	read BW	    write IOPS	 write BW

no ioregionfd    367k	    1434MiB/s	76k	         297MiB/s
ioregionfd       374k	    1459MiB/s	77.3k	     302MiB/s


Please review and send your feedback.

Thank you!
Elena

Elena Ufimtseva (8):
  ioregionfd: introduce a syscall and memory API
  multiprocess: place RemoteObject definition in a header file
  ioregionfd: introduce memory API functions
  ioregionfd: Introduce IORegionDFObject type
  multiprocess: prepare ioregionfds for remote device
  multiprocess: add MPQEMU_CMD_BAR_INFO
  multiprocess: add ioregionfd memory region in proxy
  multiprocess: handle ioregionfd commands

 meson.build                     |  15 +-
 qapi/qom.json                   |  32 ++-
 include/exec/memory.h           |  50 +++++
 include/hw/remote/ioregionfd.h  |  45 ++++
 include/hw/remote/machine.h     |   1 +
 include/hw/remote/mpqemu-link.h |   2 +
 include/hw/remote/proxy.h       |   1 +
 include/hw/remote/remote.h      |  31 +++
 include/sysemu/kvm.h            |  15 ++
 linux-headers/ioregionfd.h      |  30 +++
 linux-headers/linux/kvm.h       |  25 +++
 accel/kvm/kvm-all.c             | 132 ++++++++++++
 accel/stubs/kvm-stub.c          |   1 +
 hw/remote/ioregionfd.c          | 361 ++++++++++++++++++++++++++++++++
 hw/remote/message.c             |  38 ++++
 hw/remote/proxy.c               |  66 +++++-
 hw/remote/remote-obj.c          | 154 ++++++++++++--
 softmmu/memory.c                | 207 ++++++++++++++++++
 Kconfig.host                    |   3 +
 MAINTAINERS                     |   3 +
 hw/remote/Kconfig               |   4 +
 hw/remote/meson.build           |   1 +
 meson_options.txt               |   2 +
 scripts/meson-buildoptions.sh   |   3 +
 24 files changed, 1199 insertions(+), 23 deletions(-)
 create mode 100644 include/hw/remote/ioregionfd.h
 create mode 100644 include/hw/remote/remote.h
 create mode 100644 linux-headers/ioregionfd.h
 create mode 100644 hw/remote/ioregionfd.c

-- 
2.25.1