From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: PCI passthrough issue Date: Wed, 2 Feb 2011 12:42:50 -0500 Message-ID: <20110202174250.GA8148@dumpdata.com> References: <4D47F9CF.2040107@jbfavre.org> <1296566401.13091.171.camel@zakaz.uk.xensource.com> <4D4814CE.5050303@jbfavre.org> <1296569931.13091.194.camel@zakaz.uk.xensource.com> <4D48234F.2020907@jbfavre.org> <4D4828D9.6090601@jbfavre.org> <1296577389.13091.288.camel@zakaz.uk.xensource.com> <4D488355.8010706@jbfavre.org> <1296638873.13091.315.camel@zakaz.uk.xensource.com> <4D4930F3.608@jbfavre.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Return-path: Content-Disposition: inline In-Reply-To: <4D4930F3.608@jbfavre.org> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: xen-devel@lists.xensource.com, Jean Baptiste Favre , Ian Campbell List-Id: xen-devel@lists.xenproject.org On Wed, Feb 02, 2011 at 11:24:51AM +0100, Jean Baptiste Favre wrote: > Hello Ian, >=20 > My domU config file: >=20 > # cat /cluster/xen/xps-106.cfg > kernel =3D '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem' > ramdisk =3D '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem' > builder =3D 'linux' > memory =3D '398' > vcpus =3D '1' > cpus =3D '2' > localtime =3D 0 > serial =3D 'pty' > boot =3D 'cdn' > disk =3D [ 'drbd:xps-106,xvda,w' ] > on_poweroff =3D 'destroy' > on_reboot =3D 'restart' > on_crash =3D 'restart' > extra =3D "root=3D/dev/mapper/xps--106-root ro iommu=3Dsoft swiotlb=3Df= orce > console=3Dhvc0 xencons=3Dtty" > pci =3D [ '04:00.0' ] > name =3D 'xps-106' > hostname =3D 'xps-106.clichy.jbfavre.org' >=20 >=20 > Le 02/02/2011 10:27, Ian Campbell a =E9crit : > > On Tue, 2011-02-01 at 22:04 +0000, Jean Baptiste Favre wrote: > > > Le 01/02/2011 17:23, Ian Campbell a =E9crit : > > > > >> I assume you are not seeing "rx mapping error" in your domU dmesg?= Did > > >> you post a full guest console log at some point? Comparing the log= s for > > >> the 256MB, 398MB and 512MB guest RAM case might be useful. > > > No sure I've ever posted that logs. But I can redo my tests :) > > > > yes, please do that. > Please find attached both console startup logs with 256M and 512M: > 256M_domU_console_logs.txt > 512M_domU_console_logs.txt >=20 > For 512M, I saw some kernel CallTrace I can not explain. There are not > present with 256M. >=20 > For 398M memory, I can't even start domU : > # xm create /cluster/xen/xps-106.cfg -c > Using config file "/cluster/xen/xps-106.cfg". > [215739.007871] pciback 0000:04:00.0: device has been assigned to > another domain! Over-writting the ownership, but beware. > Started domain xps-106 (id=3D23) > (XEN) mm.c:798:d23 Non-privileged (23) attempt to map I/O space 0000000= 0 > (XEN) mm.c:4644:d23 ptwr_emulate: could not get_page_from_l1e() >=20 > As I told you, I'm still using Debian 2.6.37 kernel because I've some > problem to compile 2.6.32.27 from Jeremy's git repository. > I hope I can get it compiled today so I'll be able to test with that > kernel as well. So I've tried this on my Abit IP-35 box which has a=20 04:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8056 PCI-E = Gigabit Ethernet Controller (rev 13) Subsystem: ABIT Computer Corp. Device 1085 Flags: bus master, fast devsel, latency 0, IRQ 29 Memory at fdefc000 (64-bit, non-prefetchable) [size=3D16K] I/O ports at be00 [size=3D256] Expansion ROM at [disabled] Capabilities: [48] Power Management version 3 Capabilities: [50] Vital Product Data Capabilities: [5c] MSI: Enable+ Count=3D1/1 Maskable- 64bit+ Capabilities: [e0] Express Legacy Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Kernel driver in use: sky2 Kernel modules: sky2 And when I launch this guest with a 32-bit DomU: kernel=3D"/mnt/lab/latest/vmlinuz" ramdisk=3D"/mnt/lab/latest/initramfs.cpio.gz" extra=3D"console=3Dhvc0 debug iommu=3Dsoft" memory=3D320 vcpus=3D1 cpu=3D'2' on_crash=3D"preserve" #vif =3D [ 'bridge=3Dswitch' ] pci =3D ["04:00.0"] vfb =3D [ 'vnc=3D1, vnclisten=3D0.0.0.0,vncunused=3D1'] And played around with the 'extra' to add 'swiotlb=3Dforce'. The moment I had 'swiotlb=3Dforce' I could not get any DHCP address from the NIC. If I did not have 'swiotlb=3Dforce' it would work just fine (can ping any size, etc, this is with 320MB) For fun, I upped the memory (320->720) and kept 'swiotlb=3Dforce' in effe= ct. Same effect: can't do DHCP. I look to have a different issue than you, which is that whenever I use swiotlb=3Dforce, things go haywire. Fyi, this is what DomU tells me: 12:22:41 # 9 :~/=20 > dmesg |grep Memor [ 0.000000] Memory: 145640k/335872k available (3731k kernel code, 1897= 84k reserved, 1565k data, 436k init, 0k highmem) 12:22:56 # 10 :~/=20 > uname -a Linux (none) 2.6.38-rc2-00028-gf2a2d8b #2 SMP Wed Feb 2 12:10:25 EST 2011= i686 i686 i386 GNU/Linux rnet driver. [ 0.921406] udevd (1126): /proc/1126/oom_adj is deprecated, please use= /proc/1126/oom_score_adj instead. [ 0.984886] sky2: driver version 1.28 [ 0.995595] sky2 0000:04:00.0: BAR 0: set to [mem 0xfdefc000-0xfdeffff= f 64bit] (PCI address [0xfdefc000-0xfdefffff]) [ 0.995662] sky2 0000:04:00.0: BAR 2: set to [io 0xbe00-0xbeff] (PCI = address [0xbe00-0xbeff]) [ 0.995697] sky2 0000:04:00.0: enabling device (0000 -> 0003) [ 0.996440] sky2 0000:04:00.0: Xen PCI mapped GSI18 to IRQ27 (This is the #master branch from git://git.kernel.org/pub/scm/linux/kerne= l/git/konrad/xen.git)