From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: GPU passthrough with Xen 4.2 on Ubuntu 12.04 Date: Thu, 9 Aug 2012 12:31:43 -0400 Message-ID: <20120809163143.GB4540@phenom.dumpdata.com> References: <50226909.9090804@cam.ac.uk> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <50226909.9090804@cam.ac.uk> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Matthew Dean Cc: xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org On Wed, Aug 08, 2012 at 02:26:33PM +0100, Matthew Dean wrote: > I have been trying to setup GPU passthrough for a couple of days now > with little luck. I'm hoping someone can shed some light as to > where I may be going wrong or at least identify some genuine bugs. > Essentially pci passthrough works for me but gpu passthrough > doesn't. > > My system is currently configured as follows (please ask if you need > further details) > > Asrock Z77 e-Itx (stock bios, not sure the version but I can find > out. vt-d is enabled) > i7 3770S > AMD Radeon HD 7750 > > Dom0 - Ubuntu server 12.04 with desktop environment and the > following extra packages installed before the build > > build-essential zlib1g-dev python-dev libncurses5-dev libssl-dev > libx11-dev uuid-dev libyajl-dev libaio-dev libglib2.0-dev pkg-config > bridge-utils iproute udev bison flex gettext bin86 bcc iasl markdown > ocaml-nox ocaml-findlib git gcc-multilib texinfo pciutils-dev gawk > libcurl3 libcurl4-openssl-dev bzip2 module-init-tools transfig tgif > texlive-latex-base texlive-latex-recommended texlive-fonts-extra > texlive-fonts-recommended mercurial make gcc libc6-dev python > python-twisted patch libsdl-dev libjpeg62-dev libbz2-dev > e2fslibs-dev git-core xz-utils liblzma-dev liblzo2-dev > libvncserver-dev > > I've used the stock Ubuntu provided kernel which is version > 3.2.0-27-generic. Dom0 is connected to a display via the intel > integrated graphics. > > xen 4.2 has been build from xen unstable tag 4.2.0-rc1 changeset > 25689 using the following commands > > ./configure > make world > make install > > I had to manually add a line to /etc/fstab to get /proc/xen to mount > on startup > Modules xen-evtchn, xen-gntdev and xen-pciback were setup to load on > boot in /etc/modules > I've then setup xencommons to start on boot: "update-rc.d xencommons > defaults 19 18" > > Grub2 has then been setup to automatically boot the xen kernel. I've > also had to add the option xsave=0 to the xen boot command line to > get things to boot > > On restart everything looks good. "xl list" returns the Dom0 domain > only. "cat /proc/xen/capabilities" returns control_d. I've created > a windows 7 domU without any pci passthrough and successfully > installed windows 7 ultimate. The config file looks like: > > builder='hvm' > vcpus = 4 > memory = 8192 > shadow_memory = 48 > name = "xenwin7" > vif = [ 'bridge=br0' ] > acpi = 1 > apic = 1 > disk = [ 'file:/usr/local/xenImages/xenwin7.img,hda,w'] > boot="c" > sdl=0 > vnc=1 > vncconsole=1 > vncpasswd='' > viridian=1 > usb=1 > serial='pty' > usbdevice='tablet' > on_poweroff="destroy" > on_reboot="restart" > on_crash="destroy" > > I would like to be able to pass through the HD 7750 and a USB > controller to the VM. I see from lspci there are three devices I > need to pass through, two for the gpu (the card itseft and the hd > audio for hdmi device) and one for the USB controller. The > bus/device ids are: > > 0000:01:00.0 > 0000:01:00.1 > 0000:00:14.0 > > There are other usb controllers but I would like to leave them with > dom0. I can successfully configure the devices for passthrough > using "xl pci-assignable-add". In the config I then add the line: > > pci=["01:00.0","01:00.1","00:14.0"] > > When I boot the VM everything seems to work fine. Windows picks up > all three devices and I can install drivers for them. The USB > controller works flawlessly and I can use an attached mouse and > keyboard. The radeon card requires a restart after which an > attached display comes to life and I have 3D acceleration. > Restarting the VM seems to work OK. If however I shut down the VM I > have no end of problems. On any subsequent startup the vm struggles > to get past the windows splash screen, waiting much longer than > usual. During this period dom0 is sluggish regarding mouse and > keyboard input even though cpu and memory usage are very low. When > windows finally loads I have no active display and I have to view > the VM via VNC. In device manage I find that windows has disabled > the GPU saying there are not enough resources to run the card. From > this point onwards I can do nothing to get the gpu working again > aside from removing the device, manually deleting the drivers, and > starting again. > > Note that at this point I am only trying to do secondary passthrough > though I would ideally like to get to the point of doing primary > passthrough. Adding the line gfx_passthru=1 at any point in all > this to the machine config however just prevents the VM from booting > entirely; when I VNC in all I get is a qemu prompt and I never get I am not sure if this the same problem I've been seeing is but every time I restart a Windows VM with GPU passthrough it crashed. I found that if "Eject" the GPU before shutting down it would work just great the next time I started it. Try that and see if that works. This is btw with an unmodified Xen 4.1 and Fedora's 3.4.something kernel. > any ouput to the real display. > > Any suggestions as to how to get this to work would be greatly > appreciated as I've hit a bit of a brick wall. I should also say > that I have managed to get secondary passthrough working using > Debian Wheezy and the repository version of xen 4.1. In that case > though Dom0 didn't boot reliably. > > Sincerely, > > Matthew Dean > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel