Re: [RFC PATCH 0/8] Add host i915 support for vGPU

[Daniel Vetter <daniel@ffwll.ch>]

On Tue, Sep 30, 2014 at 06:05:30PM +0800, Jike Song wrote:
> Intel GVT-g (previously known as XenGT), is a complete GPU
> virtualization solution with mediated pass-through for 4th
> generation Intel Core processors - Haswell platform. This
> technology presents a virtual full-fledged GPU to each Virtual
> Machine (VM). VMs can directly access performance-critical
> resources, without intervention from the hypervisor in most
> cases, while privileged operations from VMs are trap-and-emulated
> at minimal cost. For details, please refer to:
> 
>   https://01.org/xen/blogs/wangbo85/2014/intel-gvt-gxengt-pubic-release
> 
> The patches of adding vGPU guest support for i915, is already posted at:
> 
>   http://lists.freedesktop.org/archives/intel-gfx/2014-September/052571.html
> 
> 
> This patch set is about to add vGPU host support.
> 
> When running as vGPU host, the i915 driver can't simply occupy
> the whole GPU resources, it needs to proactively meidate the
> accesses to most hardware resources: MMIO, GTT, and interrupt.
> Only by handling the hardware resources, the vgt can have
> centralized management about sharing between VMs(including
> host and guest).
> 
> 
> This patch set adds:
> 
> 	- an i915 extension, named vgt. vgt is the major part of Intel
> 	  GVT-g implementation, it manages and shares GPU among VMs
> 	  and host;
> 
> 	- interfaces between i915 and vgt, including:
> 
> 		Mediated MMIO access
> 		Mediated GTT access
> 		Mediated IRQ handling
> 
> NOTE of RFC:
> 
> 	- the vgt in-kernel GPU device-model is not yet integrated, though
> 	  the interfaces between i915 and vgt are provided;
> 
> 	- the host i915 driver has some logic same with the guest i915,
> 	  e.g. the ballooning. When the guest patches(see above) are
> 	  finalized, we need to rebase upon them;
> 
> 	- vgt_suspend/vgt_resume are still under cleanup;
> 
> 	- GPU reset is still under cleanup
> 
> 
>   We send out the framework changes in this patch set for review
> at first, and meanwhile, vgt integration and cleanup are ongoing.

So on a very high level I don't understand this design. For the guest side
it's completely clear that we need a bunch of hooks over the driver to
make paravirtualization work.

But on the host side I expect the driver to be in full control of the
hardware. I haven't really seen the other side but it looks like with vgt
we actually have two drivers fighting over the hardware, which requires
the various hooks to avoid disaster. The drm subsystem is littered with
such attempts, and they all didn't end up in a pretty way.

So way can't we have the vgt support for guests sit on top of i915, using
the i915 functions to set up pagetables, contexts and reserve gtt areas
for the guests? Then we'd have just one driver in control of the hardware,
and vgt on the host side would just look like a really crazy interace
layer between virtual hosts and the low-level driver, similar to who the
execbuf ioctl is a really crazy interface between userspace and the
low-level driver.

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch