From mboxrd@z Thu Jan 1 00:00:00 1970 From: sanket hase Subject: Re: [Xen-users] Passthrough support ? Date: Tue, 27 Nov 2007 20:30:46 -0500 Message-ID: <474CC4C6.4020906@andrew.cmu.edu> References: <474B5E98.1060302@andrew.cmu.edu> <200711272055.39363.mark.williamson@cl.cam.ac.uk> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <200711272055.39363.mark.williamson@cl.cam.ac.uk> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Mark Williamson Cc: xen-devel@lists.xensource.com, xen-users@lists.xensource.com List-Id: xen-devel@lists.xenproject.org Good brainer. ... I am geting back to few basics. I will leave HVM and emulation aside :) 1. When PCI Passthrough support 'is not enabled' , how does domU access PCI device ? I suppose they continue to communicate using pcifront/back split drivers and now dom0's drivers are used ( right ? ) 2. PCI-passthrough enables domU to own the PCI device. Now dom0 can no longer use that device . If domU has mapped address space of the required PCI device , why need it talk to dom0 any further ? What kind of hand shake is involved ( setup and tear down as you have mentioned ) between domU and domU " in passthrough case" ? 3. DomU discovers all PCI devices through Xenstore. Mark Williamson wrote: >> I have been trying to understand PCI -passthrough support. Please >> correct me if I am wrong in my following inferences. >> >> 1. Device emulation and pass through are both implemented using split >> drivers. >> > > I'm going to be pedantic now ;-) > > Device "emulation" is really what we do for fully virtualised (HVM) > guests: the device models provided by QEmu emulate real world devices > in terms of their responses to particular port IOs, mmapped IO > operations, etc. This isn't done using a front / back model because > the guest is just using it's normal drivers for the "real" devices. > > The virtual devices used by the PV drivers are implemented using split > drivers, though, as you say. > > The PCI passthrough support for PV guest is also implemented using a > split driver that implements the functions of the PCI bus in order to > give the guest the information it Which is this information we are referring to here ? > requires to talk to its assigned device(s). How are devices "assigned" to domU ? I am specifically talking about late binding. > Having obtained this information, communication with the device is > possible directly using IO ports, memory IO regions, and DMA. > > >> 2. But, in case of emulation drivers of dom0 are used where as in case >> of passthrough ( as the name suggests ) native drivers in domU are >> used . >> > > For true emulation (qemu device model), a userspace process in dom0 > handles modelling a "real" device and then issues IO using normal > userspace APIs. These get serviced by the dom0 kernel using the > normal device driver. > > For PV drivers, the frontend driver in the domU kernel issues requests > which are picked up by the backend driver in dom0's kernel, which then > issues requests into the IO stack. Again this uses the normal device > driver in dom0 to talk to the actual device, it's just that the > request is made using a kernel-internal API rather than a userspace > API (which results in slightly different actions being taken). > > >> 3. dom0 provides a virtual PCI device { an interface for device-OPs and >> status of this virtual device} to domU and through associated event >> channel domU makes >> "synchronous" use of this device. >> > > domU uses this for control plane operations, but for most work it can > talk to its PCI device directly without going through dom0. > > >> === >> Queries: >> >> 1. What i am really not so sure about is ... passthrough case >> Will there be requirement to map the address space of this PCI >> device in domU ? Will the page which was being shared so-far >> {xen_pci_sharedinfo} >> for emulation , be "flipped" ( transferred ) into domU ? >> > > xen_pci_sharedinfo - is that the page used to talk to the PCI backend > from pcifront? If so, then no, that's just used for dom0-domU > communications. > Well As I read in the xen0linux code , xen_pci_sharedinfo contains xen_pci_op where front end specifies which operation is to be done on which pci device. And then it keeps checking the status of this device through another flag in xen_pci_sharedinfo. I infer here that dom0's drivers are being used. Can you point me towards a code path where domU actually uses its own drivers having owned the PCI device ? > In order to map the address space of the PCI device directly, the domU > is given permissions to map the IO memory regions of that device into > it's page tables. I think this is now possible to do using a grant > table operation... > > It is also given permission to access certain IO port ranges so that > it can use the device's port IO interfaces. > > >> 2. Well , Having read the code for linux (dom0,domU) I see that >> there are split >> device drivers for PCI. (pci front and PCIback). Which are normally >> communicating over xenbus. which looks almost like other split >> drivers. How exactly then passthrough enables use of domU's drivers ? >> > > The key thing to understand is that the pcifront / pciback is > basically just used for setup and teardown, not for the actual IO. > The real IO is done directly by the domU without going through dom0. > For the block and net drivers, *all* IO goes through dom0. > > >> 3. And if passthrough support isnt provided how will communication >> between pcifront-pciback be different ? ( I guess netbsd , freebsd do >> not have passthrough support yet ) >> > > I'm not entirely clear what you're asking here, but I'll take a stab > at it: > > if pcifront (in domU) and pciback (in dom0) aren't available then > passthrough won't work. The dom0 has to support the backend functions > of PCI passthrough and the domU has to know how to talk to it. It's > also implicit that they're using the same interface version to talk to > each other - I'm not sure whether that's frozen stable or not. > > So *if* NetBSD lacks pciback support, it can't pass PCI devices to > guests that do. Similarly, *if* it lacks pcifront support, it can't > have devices passed to it. > > >> 4. What restricts other domUs from accessing PCI device given to other >> domU via passrthrough support. >> > > There are some restrictions on what can be done in PCI config space to > prevent a guest fouling things up. These need to be relaxed for some > awkward devices, though. > > For the device IO itself, domUs are only allowed to map mmio regions > and access io ports that are relevant to their device. It's possible > for these to overlap with those for other devices, in which case > you're trusting the domU to be well behaved. More crucially, though, > giving a domain a device with DMA capabilities is equivalent to giving > it the ability to subvert the entire machine. DMA can't be sandboxed > on most current hardware, so if you give DMA rights to a VM it's > automatically just as trusted as dom0 with respect to not fooling > about with other domains, hardware, etc. > > Cheers, > Mark > > Thanks, Sanket