Re: VFIO based vGPU(was Re: [Announcement] 2015-Q3 release of XenGT - a Mediated ...)

[Alex Williamson <alex.williamson@redhat.com>]

On Tue, 2016-01-26 at 15:41 +0800, Jike Song wrote:
> On 01/26/2016 05:30 AM, Alex Williamson wrote:
> > [cc +Neo @Nvidia]
> > 
> > Hi Jike,
> > 
> > On Mon, 2016-01-25 at 19:34 +0800, Jike Song wrote:
> > > On 01/20/2016 05:05 PM, Tian, Kevin wrote:
> > > > I would expect we can spell out next level tasks toward above
> > > > direction, upon which Alex can easily judge whether there are
> > > > some common VFIO framework changes that he can help :-)
> > > 
> > > Hi Alex,
> > > 
> > > Here is a draft task list after a short discussion w/ Kevin,
> > > would you please have a look?
> > > 
> > > 	Bus Driver
> > > 
> > > 		{ in i915/vgt/xxx.c }
> > > 
> > > 		- define a subset of vfio_pci interfaces
> > > 		- selective pass-through (say aperture)
> > > 		- trap MMIO: interface w/ QEMU
> > 
> > What's included in the subset?  Certainly the bus reset ioctls really
> > don't apply, but you'll need to support the full device interface,
> > right?  That includes the region info ioctl and access through the vfio
> > device file descriptor as well as the interrupt info and setup ioctls.
> > 
> 
> [All interfaces I thought are via ioctl:)  For other stuff like file
> descriptor we'll definitely keep it.]
> 
> The list of ioctl commands provided by vfio_pci:
> 
> 	- VFIO_DEVICE_GET_PCI_HOT_RESET_INFO
> 	- VFIO_DEVICE_PCI_HOT_RESET
> 
> As you said, above 2 don't apply. But for this:
> 
> 	- VFIO_DEVICE_RESET
> 
> In my opinion it should be kept, no matter what will be provided in
> the bus driver.

Yes, the DEVICE_INFO ioctl describes whether it's present, I would
encourage implementing it.

> 	- VFIO_PCI_ROM_REGION_INDEX
> 	- VFIO_PCI_VGA_REGION_INDEX
> 
> I suppose above 2 don't apply neither? For a vgpu we don't provide a
> ROM BAR or VGA region.

Right, these aren't ioctls, just indexes into the REGION_INFO ioctl,
they're optional.

> 	- VFIO_DEVICE_GET_INFO
> 	- VFIO_DEVICE_GET_REGION_INFO
> 	- VFIO_DEVICE_GET_IRQ_INFO
> 	- VFIO_DEVICE_SET_IRQS
> 
> Above 4 are needed of course.
> 
> We will need to extend:
> 
> 	- VFIO_DEVICE_GET_REGION_INFO
> 
> 
> a) adding a flag: DONT_MAP. For example, the MMIO of vgpu
> should be trapped instead of being mmap-ed.

There's already an MMAP flag, mmap is only allowed when this is set, so
there's no need for the anti-flag.  I'm also working on support for
sparse mmap capabilities so that within a region some portions can
support mmap.

> b) adding other information. For example, for the OpRegion, QEMU need
> to do more than mmap a region, it has to:
> 
> 	- allocate a region
> 	- copy contents from somewhere in host to that region
> 	- mmap it to guest
> 
> 
> I remember you already have a prototype for this?

Yes, I'm working on this currently, it will by a device specific region
and QEMU can either copy the contents to a new buffer in guest memory
or provided trapped access to the host opregion.  I thought vgpus
weren't going to need opregions though, I figured it was more for GVT-d 
support.  Thanks,

Alex