From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marek Marczykowski Subject: Memory fragmentation and PCI passthrough Date: Tue, 06 Sep 2011 19:54:27 +0200 Message-ID: <4E665E53.3@invisiblethingslab.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1655502185==" Return-path: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: "xen-devel@lists.xensource.com" Cc: Rafal Wojtczuk , Joanna Rutkowska List-Id: xen-devel@lists.xenproject.org This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --===============1655502185== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig5A5420FFFDF5DD86CC10B256" This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig5A5420FFFDF5DD86CC10B256 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hello, I've hit known problem with dynamic memory management - memory fragmentation... This dynamic memory management basically does xl mem-set to balance memory. After some time of running system, xen memory is so fragmented that it is impossible to start new VM with PCI device. Sometimes it crashes during boot (no 64MB contiguous memory for SWIOTLB), or later - eg. iwlagn cannot allocate memory for loading firmware (few allocs, each bellow 100k). DomU kernel cmdline: console=3Dhvc0 iommu=3Dsoft earlyprintk=3Dxen There is two cases (I think): 1. With IOMMU 2. Without IOMMU I've tried only the second one. Is there any known solution for this problem? Some ideas: 1. With IOMMU pass iommu=3Dpv to Xen. AFAIU domU will not need iommu=3Dso= ft parameter then, right? Will it work then with fragmented memory? 2. Force somehow on xen/libxl to allocate memory (for domU) in chunks of, say 4MB, to not fragment it so badly. Is it doable? In tmem documentation is also described some workaround for this: reserve some memory region for allocations with 0