Re: GTX 760 passed through

[Gordan Bobic <gordan@bobich.net>]

On 12/02/2013 08:16 PM, Konrad Rzeszutek Wilk wrote:
> On Mon, Dec 02, 2013 at 06:36:48PM +0100, Nvidia Reverse wrote:
>> The nvlddmkm.sys needs to be patched removing the whitelist for the device
>> ids allowed to be virtualized.
>> But the interesting part is how NVIDIA detects that the GPU is
>> virtualized...
>
> Interesting. I was thinking that the BIOS/firmware would run itself
> in the virtualized or non-virtualized code depending on the device id.
> But you seem to imply that it is all in the OS driver code.

Certainly not the case with the BIOS part - Quadro BIOS is quite 
different. GeForce BIOS, for example, doesn't have ECC control code and 
a few other things. For example, flash a 4GB GTX680 with a K5000 BIOS 
and the ECC control options start to show up in the control panel. If 
they went to the trouble of stripping all that out of the GeForce BIOS 
I'd rather like to think that they would have tripped out any magic 
required to run virtualized.

Since we know that isn't the case (since removing a single resistor on a 
680 makes it work just fine virtualized as a Tesla K10), the only 
logical conclusion there is nothing that the BIOS does that makes any 
difference to virtualization.

> At which point the idea of just modifying in QEMU the PCI device ID
> to be different would .. well, make it possible to do a lot of
> neat stuff as Gordan pointed out.

Only on pre-Kepler GPUs. As far as I can tell, there is a register in 
the GPU where the hard-strapped device ID is kept, and this gets set at 
boot time BEFORE the BIOS runs. It is the pre-BIOS device ID that gets 
stuck in the GPU, and the soft-strap only affects the PCI device ID - 
yes, the two can differ on Kepler. Driver keys off the hard-strapped ID.

> In terms of legal issues of patching up a windows kernel driver and
> showing other folks how to do it?
>
> No idea. Presumarily there is some license thing that you had agreed
> when you installed the Nvidia driver - check to see what it says.

I think the OP was more referring to specifically applying a patch to 
Xen to prevent the Nvidia driver in domU from being able to figure out 
it's running in a VM. If it doesn't think it's running in a VM, it 
doesn't disable the device.

BTW, Konrad, did your GTX460 work with PCI passthrough after modifying 
it to a Q4000M?

Gordan