Re: RE: PCI devices remap from dom0 to domU with pv-ops kernel

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Thu, Oct 14, 2010 at 11:26:03AM -0600, Lin, Ray wrote:
> 
> Konrad,
> 
>   We choose "Virtual" and the dom0/domU show like this. It doesn't remap to what we want. You mentioned about "write your own plugin that would do remapping per your desire". Could you elaborate it a bit ? Thanks.

Sure. Look in drivers/xen/pciback. There are four files of interest:

vpci.c
slot.c
passthrough.c
controller.c

By default, the vpci.c is selected, where in the domU the devices start
from 00:00.0. The passthrough is obvious. 

The slot.c is where we have only two bus and all the devices are stuck under the slots.
So a maximum of 64 devices, and the device function is ignored:

This is what it would show up (I think, you might want to double-check this):
00:01.0:0c04: 11f8:8032 (rev 08)
00:02.0:0c04: 11f8:8032 (rev 08)
00:03.0:0c04: 11f8:8032 (rev 08)
00:04.0:0c04: 11f8:8032 (rev 08)

The controller, is "PCI "Controller" Backend - virtualize PCI bus topology based on PCI
 controllers.  Devices under the same PCI controller are exposed on the
 same virtual domain:bus.  Within a bus, device slots are virtualized
 to compact the bus. " (from controller.c)

I think based on what you said earlier, you just want to use the "slot" backend:
CONFIG_XEN_PCIDEV_BACKEND_SLOT=y
> 
> Dom0:
> kaan-18:~# lspci -n | grep 11f8
> 07:00.0 0c04: 11f8:8032 (rev 08)
> 07:00.1 0c04: 11f8:8032 (rev 08)
> 07:00.2 0c04: 11f8:8032 (rev 08)
> 07:00.3 0c04: 11f8:8032 (rev 08)
> 
> DomU:
> kaan-18-dpm:~# lspci -n
> 00:00.0 0c04: 11f8:8032 (rev 08)
> 00:00.1 0c04: 11f8:8032 (rev 08)
> 00:00.2 0c04: 11f8:8032 (rev 08)
> 00:00.3 0c04: 11f8:8032 (rev 08)
> 
> 
> -Ray
> 
> -----Original Message-----
> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] 
> Sent: Wednesday, October 13, 2010 11:52 AM
> To: Lin, Ray
> Cc: bruce.edge@gmail.com; xen-devel@lists.xensource.com
> Subject: Re: PCI devices remap from dom0 to domU with pv-ops kernel
> 
> On Wed, Oct 13, 2010 at 10:49:12AM -0600, Lin, Ray wrote:
> > 
> > Konrad,
> > 
> >     We try to remap the pci devices from dom0 to domU by assigning the 
> > pci in the domU configuration file. Such as
> > 
> >     pci="['01:00.0@11', '01:00.1@10', '04:00.0@13', '04:00.1@12']"
> > 
> > In hvm kernel, the domU would see the pci devices remapped like this,
> > 
> >     dom0 pci device 01:00.0   => domU pci device 00:11.0
> >     dom0 pci device 01:00.1  =>  domU pci device 00:10.0
> >     dom0 pci device 04:00.0  =>  domU pci device 00:13.0
> >     dom0 pci device 04:00.1  =>  domU pci device 00:12.0
> > 
> >  But pv-ops kernel seems ignore the remapping. Is there any way (kernel configuration setting ??) to make it work ?
> 
> Not completly.
> 
> When you compile your Xen PCI backend, you should see an option whether you want Virtual PCI, Passthrough or some other type. By default it will pick the Virtual one so that it just enumerates from zero the PCI devices. You could write your own plugin that would do remaping per your desire.
> > 
> > Thanks,
> > Ray
> > 
> > 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel