Re: Nvidia GPU passthrough and Device IDs

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Thu, Nov 28, 2013 at 11:29:49AM +0000, Gordan Bobic wrote:
> On Thu, 28 Nov 2013 11:27:09 +0000, Andrew Cooper
> <andrew.cooper3@citrix.com> wrote:
> >On 28/11/13 11:21, Gordan Bobic wrote:
> >>This has recently come to my attention:
> >>
> >>http://www.nvidia.com/object/user-selectable-machines.html
> >>
> >>along with this snippet of information:
> >>
> >>- Physical Function PCIid: 10ed:11bf - GK104GL [GRID K2]
> >>- Virtual Function selectable from
> >>PCIid: 10ed:118b - GK104 [GeForce K2 USM]
> >>PCIid: 10ed:118c - GK104 [NVS K2 USM]
> >>PCIid: 10ed: 11b0 - GK104GL [Quadro K2 USM]
> >>PCIid: 10ed:11b1 - GK104GL [Tesla K2 USM]
> >>
> >>The way this reads to me is that it means that a different
> >>device ID is exposed to domU than what runs on the host.
> >>
> >>Therefore, it must be possible to change the device ID
> >>visible to domU to something other than what is visible
> >>on the host.
> >>
> >>So, as per a question that has recently been asked, is
> >>there a way to apply a software shim to expose a different
> >>device ID to the domU when doing PCI passthrough? This
> >>might completely avoid the need to modify the GeForce
> >>cards in ways discussed here recently to make them
> >>work in PCI passthrough mode by simply faking the
> >>device ID exposed.
> >>
> >>Gordan
> >
> >All config space accesses get vetted by Qemu or pcifront.
> >
> >We have had one bug for a particular graphics card with PCI
> >passthrough
> >which was fixed by faking up the wrong class code.  (There was a
> >windows
> >XPDM/WDDM stack issue with the affected card, and the cirrus emulated
> >graphics card was detected as "better" than the passed-through one)
> 
> I'll take that to be a resounding "yes" in answer to my
> question. :)
> 
> Could you please point me in the rough direction of the code
> and process flow that handles this?

You would be wanting to look in /git/xen/tools/qemu-xen-dir/hw/xen_pt_config_init.c

There is this code:

591     /* Device ID reg */                                                         
 592     {                                                                           
 593         .offset     = PCI_DEVICE_ID,                                            
 594         .size       = 2,                                                        
 595         .init_val   = 0x0000,                                                   
 596         .ro_mask    = 0xFFFF,                                                   
 597         .emu_mask   = 0xFFFF,                                                   
 598         .init       = xen_pt_device_reg_init,                                   
 599         .u.w.read   = xen_pt_word_reg_read,                                     
 600         .u.w.write  = xen_pt_word_reg_write,                                    
 601     },                                                                          

And the xen_pt_... are the default ones. You want to over-write 
xen_pt_device_reg_init with your own function that sets
the init_val to something. Or modify xen_pt_device_reg_init a bit.

something that provides a different value.

Looking forward to hearing how it goes!