From mboxrd@z Thu Jan 1 00:00:00 1970 From: Joanna Rutkowska Subject: Re: Assigning contiguous memory to a driver domain Date: Wed, 15 Sep 2010 12:42:50 +0200 Message-ID: <4C90A32A.7010803@invisiblethingslab.com> References: <20100914092439.GA1000@email> <4C8F5E580200007800015F71@vpn.id2.novell.com> <20100915090842.GB1583@email> <20100915093910.GC1583@email> <4C90B31902000078000163E8@vpn.id2.novell.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1887859289==" Return-path: In-Reply-To: <4C90B31902000078000163E8@vpn.id2.novell.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Jan Beulich Cc: Dan Magenheimer , xen-devel@lists.xensource.com, Rafal Wojtczuk , Konrad Rzeszutek Wilk List-Id: xen-devel@lists.xenproject.org This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --===============1887859289== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig65A1357F8D19CEAE78E66A68" This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig65A1357F8D19CEAE78E66A68 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 09/15/10 11:50, Jan Beulich wrote: >>>> On 15.09.10 at 11:39, Rafal Wojtczuk = wrote: >> On Wed, Sep 15, 2010 at 11:08:42AM +0200, Rafal Wojtczuk wrote: >>> On Tue, Sep 14, 2010 at 10:36:56AM +0100, Jan Beulich wrote: >>>>>>> On 14.09.10 at 11:24, Rafal Wojtczuk wrote: >>>>> Hello, >>>>> >>>>> Could someone guide me in the right direction with the topic of ass= igning >>>>> contiguous memory to a domain. >>>>> >>>>> I have an issue with a PV domain that is assigned a PCI device. Som= etimes,=20 >> >>>>> the=20 >>>>> driver fails to load >>>>> Sep 13 10:36:43 localhost kernel: [ 103.651858] iwlagn 0000:00:01.= 0: >>>>> firmware: requesting iwlwifi-4965-2.ucode >>>>> Sep 13 10:36:43 localhost kernel: [ 103.669105] iwlagn 0000:00:01.= 0:=20 >> loaded >>>>> firmware version 228.61.2.24 >>>>> Sep 13 10:36:43 localhost kernel: [ 103.669263] iwlagn 0000:00:01.= 0:=20 >> failed >>>>> to allocate pci memory >>>>> >>>>> The reason seems to be that the domain does not have enough contigu= ous >>>>> memory, in mfn terms. >>>> >>>> No, how (dis)contiguous the memory of a domain is doesn't matter >>>> here. What matters is whether the domain can *make* the requested >>>> memory contiguous, and that depends on how much contiguous >>>> memory Xen has at the point of the allocation. >>> >>> Ah, so you are saying that regardless of whether a domain has some >>> contiguous memory, the driver will call xen_create_contiguous_region = when=20 >> allocating >>> memory (via dma_alloc_coherent ?). >>> >>> Slightly out-of-the-list-scope: is there a convention when a driver s= hould >>> allocate DMA-able memory ? Is it safe to assume that as soon as the d= river >>> has loaded, it will no longer need to call xen_create_contiguous_regi= on >>> anymore and we can use up all free Xen memory ?=20 >> >> Hmm, at least in case of tg3 driver, if xen free memory =3D 0, after I= have >> done "ifconfig eth0 down" in the driver domain, the subsequent "ifconf= ig=20 >> eth0 up"=20 >> failed with "SIOCSIFFLAGS: Cannot allocate memory". >=20 > Sure - why would the driver waste resources when the device may > not be used. >=20 >> So, it looks like in order to make a PV driver domain work, there must= be=20 >> some Xen free memory=20 >> all the time ? >=20 > Potentially yes, but this really depends on how the respective > driver is written. >=20 >> Moreover, this free Xen memory must be contiguous to some >> extent; is there any way to assure this ? >=20 > No. >=20 Wait! Are you saying there is no *way* to guarantee proper operation of a driver domain in Xen? Sure, we can tune our memory balancer to always keep some 100MB (or 200MB, or maybe 500MB?) of xen free memory, and *hope* that it will contain enough continues pages, in case some driver in some driver domain calls dma_alloc_coherent(), so the call will succeed. But this is not a good solution: not only because it's a waste of memory (I'd rather use this memory for Dom0/storage domain page cache instead) but also, and most importantly, because I don't want to build a system based on *hope*! Can we do something about it? joanna. --------------enig65A1357F8D19CEAE78E66A68 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJMkKMqAAoJEDaIqHeRBUM0TyEIAII6pW1phZ5/6K5IhfNV8MaW pFqBWaeOCdXEVoxMvYYSf26PV2U76ucS9V0oTI8Q7OqJsnrudnkqXTox1l+eAl0q g+k/Ra2GZ99cOAeJ5BxO1eOgDNVm/WBSF3EQMHSBDzps4L8f5uy07CZKjGBMH7UZ a+rNGhbf6Pz8kahzKDZiR6GxSMy73xcPFUrEOA73m4nA9noN9lCN5j1CqMlrK4Ii uANbzB/U+G9a7BS8UAZkDMHQQBTZn2fEiHMAutMAayggT8D0pN3GlGPiAtdHCB/u G/79t0eVFnxQ8lFGP+Y/LUtu4Yjk1ZXlsw/Hv6nAb8ap6kZ1Pbh4fo/sOlOXL5U= =5dDK -----END PGP SIGNATURE----- --------------enig65A1357F8D19CEAE78E66A68-- --===============1887859289== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============1887859289==--