From mboxrd@z Thu Jan 1 00:00:00 1970 From: sanket hase Subject: Re: [Xen-users] Passthrough support ? Date: Tue, 27 Nov 2007 20:27:52 -0500 Message-ID: <474CC418.8050707@andrew.cmu.edu> References: <474B5E98.1060302@andrew.cmu.edu> <200711272055.39363.mark.williamson@cl.cam.ac.uk> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <200711272055.39363.mark.williamson@cl.cam.ac.uk> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Mark Williamson Cc: xen-devel@lists.xensource.com, xen-users@lists.xensource.com List-Id: xen-devel@lists.xenproject.org Good brainer. ... I am geting back to few basics. I will leave HVM and emulation aside :) 1. When PCI Passthrough support 'is not enabled' , how does domU access PCI device ? I suppose they continue to communicate using pcifront/back split drivers and now dom0's drivers are used ( right ? ) 2. PCI-passthrough enables domU to own the PCI device. Now dom0 can no longer use that device . If domU has mapped address space of the required PCI device , why need it talk to dom0 any further ? What kind of hand shake is involved ( setup and tear down as you have mentioned ) between domU and domU " in passthrough case" ? 3. DomU discovers all PCI devices through Xenstore. Mark Williamson wrote: >> I have been trying to understand PCI -passthrough support. Please >> correct me if I am wrong in my following inferences. >> >> 1. Device emulation and pass through are both implemented using split >> drivers. >> > > I'm going to be pedantic now ;-) > > Device "emulation" is really what we do for fully virtualised (HVM) guests: > the device models provided by QEmu emulate real world devices in terms of > their responses to particular port IOs, mmapped IO operations, etc. This > isn't done using a front / back model because the guest is just using it's > normal drivers for the "real" devices. > > The virtual devices used by the PV drivers are implemented using split > drivers, though, as you say. > > The PCI passthrough support for PV guest is also implemented using a split > driver that implements the functions of the PCI bus in order to give the > guest the information it Which is this information we are referring to here ? > requires to talk to its assigned device(s). How are devices "assigned" to domU ? I am specifically talking about late binding. > Having > obtained this information, communication with the device is possible directly > using IO ports, memory IO regions, and DMA. > > >> 2. But, in case of emulation drivers of dom0 are used where as in case >> of passthrough ( as the name suggests ) native drivers in domU are used . >> > > For true emulation (qemu device model), a userspace process in dom0 handles > modelling a "real" device and then issues IO using normal userspace APIs. > These get serviced by the dom0 kernel using the normal device driver. > > For PV drivers, the frontend driver in the domU kernel issues requests which > are picked up by the backend driver in dom0's kernel, which then issues > requests into the IO stack. Again this uses the normal device driver in dom0 > to talk to the actual device, it's just that the request is made using a > kernel-internal API rather than a userspace API (which results in slightly > different actions being taken). > > >> 3. dom0 provides a virtual PCI device { an interface for device-OPs and >> status of this virtual device} to domU and through associated event >> channel domU makes >> "synchronous" use of this device. >> > > domU uses this for control plane operations, but for most work it can talk to > its PCI device directly without going through dom0. > > >> === >> Queries: >> >> 1. What i am really not so sure about is ... passthrough case >> Will there be requirement to map the address space of this PCI >> device in domU ? Will the page which was being shared so-far >> {xen_pci_sharedinfo} >> for emulation , be "flipped" ( transferred ) into domU ? >> > > xen_pci_sharedinfo - is that the page used to talk to the PCI backend from > pcifront? If so, then no, that's just used for dom0-domU communications. > Well As I read in the xen0linux code , xen_pci_sharedinfo contains xen_pci_op where front end specifies which operation is to be done on which pci device. And then it keeps checking the status of this device through another flag in xen_pci_sharedinfo. I infer here that dom0's drivers are being used. Can you point me towards a code path where domU actually uses its own drivers having owned the PCI device ? > In order to map the address space of the PCI device directly, the domU is > given permissions to map the IO memory regions of that device into it's page > tables. I think this is now possible to do using a grant table operation... > > It is also given permission to access certain IO port ranges so that it can > use the device's port IO interfaces. > > >> 2. Well , >> Having read the code for linux (dom0,domU) I see that there are split >> device drivers for PCI. (pci front and PCIback). Which are normally >> communicating over xenbus. which looks almost like other split >> drivers. How exactly then passthrough enables use of domU's drivers ? >> > > The key thing to understand is that the pcifront / pciback is basically just > used for setup and teardown, not for the actual IO. The real IO is done > directly by the domU without going through dom0. For the block and net > drivers, *all* IO goes through dom0. > > >> 3. And if passthrough support isnt provided how will communication >> between pcifront-pciback be different ? ( I guess netbsd , freebsd do >> not have passthrough support yet ) >> > > I'm not entirely clear what you're asking here, but I'll take a stab at it: > > if pcifront (in domU) and pciback (in dom0) aren't available then passthrough > won't work. The dom0 has to support the backend functions of PCI passthrough > and the domU has to know how to talk to it. It's also implicit that they're > using the same interface version to talk to each other - I'm not sure whether > that's frozen stable or not. > > So *if* NetBSD lacks pciback support, it can't pass PCI devices to guests that > do. Similarly, *if* it lacks pcifront support, it can't have devices passed > to it. > > >> 4. What restricts other domUs from accessing PCI device given to other >> domU via passrthrough support. >> > > There are some restrictions on what can be done in PCI config space to prevent > a guest fouling things up. These need to be relaxed for some awkward > devices, though. > > For the device IO itself, domUs are only allowed to map mmio regions and > access io ports that are relevant to their device. It's possible for these > to overlap with those for other devices, in which case you're trusting the > domU to be well behaved. More crucially, though, giving a domain a device > with DMA capabilities is equivalent to giving it the ability to subvert the > entire machine. DMA can't be sandboxed on most current hardware, so if you > give DMA rights to a VM it's automatically just as trusted as dom0 with > respect to not fooling about with other domains, hardware, etc. > > Cheers, > Mark > > Thanks, Sanket