Re: RFC: vfio / device assignment -- layout of device fd files

["Michael S. Tsirkin" <mst@redhat.com>]

On Mon, Aug 29, 2011 at 04:51:20PM +0000, Yoder Stuart-B08248 wrote:
> Alex Graf, Scott Wood, and I met last week to try to flesh out
> some details as to how vfio could work for non-PCI devices,
> like we have in embedded systems.   This most likely will
> require a different kernel driver than vfio-- for now we are
> calling it "dtio" (for device tree I/O) as there is no way
> to discover these devices except from the device tree.   But
> the dtio driver would use the same architecture and interfaces
> as vfio.
> 
> For devices on a system bus and represented in a device
> tree we have some different requirements than PCI for what
> is exposed in the device fd file.  A device may have multiple
> address regions, multiple interrupts, a variable length device
> tree path, whether a region is mmapable, etc.
> 
> With existing vfio, the device fd file layout is something
> like:
>   0xF Config space offset
>   ...
>   0x6 ROM offset
>   0x5 BAR 5 offset
>   0x4 BAR 4 offset
>   0x3 BAR 3 offset
>   0x2 BAR 2 offset
>   0x1 BAR 1 offset
>   0x0 BAR 0 offset
> 
> We have an alternate proposal that we think is more flexible,
> extensible, and will accommodate both PCI and system bus
> type devices (and others).
> 
> Instead of config space fixed at 0xf, we would propose
> a header and multiple 'device info' records at offset 0x0 that would
> encode everything that user space needs to know about
> the device:
> ....

That's a very rich interface, and easy to get wrong.
AFAIK the only reason vfio uses read/write for PCI was to avoid inventing
a custom interface. But if you do, it looks like a set of ioctls would
be much easier? You can even fit the existing uio infrastructure if you like.

> There may be other more complex device or bus types that
> need their own special encodings, and this format would
> allow the definition of new records to define devices.  Two
> other types that come to mind are Serial Rapid I/O busses
> commonly used in our networking SoCs and the FSL DPAA
> portals which are very strange devices that may require
> their own unique interface exposed to user space.
> 
> In short, when user space opens up a device fd it needs
> some information about what this device is, and this
> proposal tries to address that.
> 
> Regards,
> Stuart Yoder

Here's another idea:  all the information is likely already available
in sysfs. A way to query where the device is in sysfs
would give you *a ton* of information, including the type etc,
if the info is not there you will be able to add it
in a way that is useful to more than just vfio,
and you won't need to extend the interface each time
you find you need a new piece of info.


-- 
MST