From mboxrd@z Thu Jan 1 00:00:00 1970 From: Erik Skultety Subject: Re: [libvirt] Expose vfio device display/migration to libvirt and above, was Re: [PATCH 0/3] sample: vfio mdev display devices. Date: Thu, 10 May 2018 13:00:29 +0200 Message-ID: <20180510110029.GA9645@erzo-ntb> References: <20180424165918.5c2ef037@w520.home> <0a1d6487-0dfb-2ffc-4774-ebaf65c15892@nvidia.com> <20180425120057.0fabb70e@w520.home> <20180425195229.GK2496@work-vm> <20180426185522.GQ2631@work-vm> <20180503125800.76cc7582@w520.home> <20180504074944.GD8859@erzo-ntb> <20180504100344.221e399f@t450s.home> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: Neo Jia , kvm@vger.kernel.org, libvirt , "Dr. David Alan Gilbert" , Tina Zhang , Kirti Wankhede , Gerd Hoffmann , Laine Stump , Jiri Denemark , intel-gvt-dev@lists.freedesktop.org To: Alex Williamson Return-path: Content-Disposition: inline In-Reply-To: <20180504100344.221e399f@t450s.home> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com List-Id: kvm.vger.kernel.org ... > > Now, if we (theoretically) can settle on easing the restrictions Alex > > has mentioned, we in fact could introduce a QMP command to probe > > these devices and provide libvirt with useful information at that > > point in time. Of course, since the 3rd party vendor is "de-coupled" > > from qemu, libvirt would have no way to find out that the driver has > > changed in the meantime, thus still using the old information we > > gathered, ergo potentially causing the QEMU process to fail > > eventually. But then again, there's very often a strong > > recommendation to reboot your host after a driver update, especially > > in NVIDIA's case, which means this fact wouldn't matter. However, > > there's also a significant drawback to my proposal which probably > > renders it completely useless (but we can continue from there...) and > > that is the devices would either have to be present already (not an > > option) or QEMU would need to be enhanced in a way, that it would > > create a dummy device during QMP probing, open it, collect the > > information libvirt needs, close it and remove it. If the driver > > doesn't change in the meantime, this should be sufficient for a VM to > > be successfully instantiated with a display, right? > > I don't think this last requirement is possible, QEMU is as clueless > about the capabilities of an mdev device as anyone else until that > device is opened and probed, so how would we invent this "dummy > device"? I don't really see how there's any ability for > pre-determination of the device capabilities, we can only probe the > actual device we intend to use. Hmm, let's say libvirt is able to create mdevs. Do the vendor drivers impose any kind of limitations on whether a specific device-type or a specific instance of a type does or does not present certain features like display or migration in comparison to the other types/instances? IOW I would assume that once the driver version does support display/migration, any mdev instance of any mdev type the driver supports will "inherit" the support for display/migration. If this assumption works, libvirt, knowing there are some mdev capable parent devices, could technically create a dummy instance of the first type it can for each parent device, passing the UUID to qemu QMP query command, qemu would then open and probe the device, returning the capabilities which libvirt would then cache. Next time a VM is due to start, libvirt can use the device UUID to check the capabilities we cached and try setting appropriate config options. However, as you've mentioned, this approach is fairly policy-driven, which doesn't cope with what libvirt's goal is. Would such a suggestion help at all from QEMU's POV? > > > > The above has pressed the need for investigating some sort of > > > alternative API through which libvirt might introspect a vfio device > > > and with vfio device migration on the horizon, it's natural that > > > some sort of support for migration state compatibility for the > > > device need be considered as a second user of such an API. > > > However, we currently have no concept of migration compatibility on > > > a per-device level as there are no migratable devices that live > > > outside of the QEMU code base. It's therefore assumed that per > > > device migration compatibility is encompassed by the versioned > > > machine type for the overall VM. We need participation all the way > > > to the top of the VM management stack to resolve this issue and > > > it's dragging down the (possibly) more simple question of how do we > > > resolve the display situation. Therefore I'm looking for > > > alternatives for display that work within what we have available to > > > us at the moment. > > > > > > Erik Skultety, who initially raised the display question, has > > > identified one possible solution, which is to simply make the > > > display configuration the user's problem (apologies if I've > > > misinterpreted Erik). I believe this would work something like: > > > > > > - libvirt identifies a version of QEMU that includes 'display' > > > support for vfio-pci devices and defaults to adding display=off for > > > every vfio-pci device [have we chosen the wrong default (auto) in > > > QEMU?]. > > > > From libvirt's POV, having a new XML attribute display to the host > > device type mdev should with a default value 'off', potentially > > extending this to 'auto' once we have enough information to base our > > decision on. We'll need to combine this with a new attribute value > > for the