Re: [PATCH RFC v2 01/16] vfio-user: introduce vfio-user protocol specification

[Alex Williamson <alex.williamson@redhat.com>]

On Mon, 16 Aug 2021 09:42:34 -0700
Elena Ufimtseva <elena.ufimtseva@oracle.com> wrote:
> +Authentication
> +--------------
> +
> +For ``AF_UNIX``, we rely on OS mandatory access controls on the socket files,
> +therefore it is up to the management layer to set up the socket as required.
> +Socket types than span guests or hosts will require a proper authentication

s/than/that/

...
> +``VFIO_USER_DMA_UNMAP``
> +-----------------------
> +
> +This command message is sent by the client to the server to inform it that a
> +DMA region, previously made available via a ``VFIO_USER_DMA_MAP`` command
> +message, is no longer available for DMA. It typically occurs when memory is
> +subtracted from the client or if the client uses a vIOMMU. The DMA region is
> +described by the following structure:
> +
> +Request
> +^^^^^^^
> +
> +The request payload for this message is a structure of the following format:
> +
> ++--------------+--------+------------------------+
> +| Name         | Offset | Size                   |
> ++==============+========+========================+
> +| argsz        | 0      | 4                      |
> ++--------------+--------+------------------------+
> +| flags        | 4      | 4                      |
> ++--------------+--------+------------------------+
> +|              | +-----+-----------------------+ |
> +|              | | Bit | Definition            | |
> +|              | +=====+=======================+ |
> +|              | | 0   | get dirty page bitmap | |
> +|              | +-----+-----------------------+ |
> ++--------------+--------+------------------------+
> +| address      | 8      | 8                      |
> ++--------------+--------+------------------------+
> +| size         | 16     | 8                      |
> ++--------------+--------+------------------------+
> +
> +* *argsz* is the maximum size of the reply payload.
> +* *flags* contains the following DMA region attributes:
> +
> +  * *get dirty page bitmap* indicates that a dirty page bitmap must be
> +    populated before unmapping the DMA region. The client must provide a
> +    `VFIO Bitmap`_ structure, explained below, immediately following this
> +    entry.
> +
> +* *address* is the base DMA address of the DMA region.
> +* *size* is the size of the DMA region.
> +
> +The address and size of the DMA region being unmapped must match exactly a
> +previous mapping. The size of request message depends on whether or not the
> +*get dirty page bitmap* bit is set in Flags:
> +
> +* If not set, the size of the total request message is: 16 + 24.
> +
> +* If set, the size of the total request message is: 16 + 24 + 16.

The address/size paradigm falls into the same issues as the vfio kernel
interface where we can't map or unmap the entire 64-bit address space,
ie. size is limited to 2^64 - 1.  The kernel interface also requires
PAGE_SIZE granularity for the DMA, which means the practical limit is
2^64 - PAGE_SIZE.  If we had a redo on the kernel interface we'd use
start/end so we can express a size of (end - start + 1).

Is following the vfio kernel interface sufficiently worthwhile for
compatibility to incur this same limitation?  I don't recall if we've
already discussed this, but perhaps worth a note in this design doc if
similarity to the kernel interface is being favored here.  See for
example QEMU commit 1b296c3def4b ("vfio: Don't issue full 2^64 unmap").
Thanks,

Alex