From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: can't create a vNUMA enabled PV guest Date: Wed, 1 Jul 2015 16:42:56 +0200 Message-ID: <1435761776.25170.380.camel@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============3351657252356487176==" Return-path: Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1ZAJE7-0002yk-Fd for xen-devel@lists.xenproject.org; Wed, 01 Jul 2015 14:43:03 +0000 List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Wei Liu Cc: Ian Campbell , "xen-devel@lists.xenproject.org" , "Ian.Jackson" List-Id: xen-devel@lists.xenproject.org --===============3351657252356487176== Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-9D+MXo3FtBcJ8y2oZuix" --=-9D+MXo3FtBcJ8y2oZuix Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hey, I know Wei is away, so I'll try to find the time to look at this myself, but I figured I'll let know about it, in case someone has obvious (or not :-D) ideas. I think I'm facing a bug that prevents creating PV guests with a vNUMA topology. I'm pretty sure I tested this before, while reviewing Wei's patches, so it must be something introduced between then and now (yes, we need a vNUMA OSSTest test case... I'll see about putting one together). So, here we are. With this as base config: name =3D 'test' # Kernel, params and imags kernel =3D '/root/3.19.0+/vmlinuz-3.19.0+' ramdisk =3D '/root/3.19.0+/initrd.img-3.19.0+' # CPUs and Memory and related vcpus =3D '4' memory =3D '1024' vnuma =3D [ [ "pnode=3D0","size=3D512","vcpus=3D0-1","vdistances=3D10,20"= ], [ "pnode=3D1","size=3D512","vcpus=3D2-3","vdistances=3D20,10" = ] ] # Disks root =3D '/dev/xvda1 ro' disk =3D [ 'phy:/dev/vms/test-pv-disk,xvda1,w', ] # Networking dhcp =3D 'dhcp' vif =3D [ 'mac=3D00:16:3E:FA:A7:9B,bridge=3Dxenbr0' ] If I build a HVM guest, everything works: (XEN) Memory location of each domain: (XEN) Domain 0 (total: 129874): (XEN) Node 0: 113466 (XEN) Node 1: 16408 (XEN) Domain 14 (total: 262251): (XEN) Node 0: 131029 (XEN) Node 1: 131222 (XEN) 2 vnodes, 4 vcpus, guest physical layout: (XEN) 0: pnode 0, vcpus 0-1=20 (XEN) 0000000000000000 - 000000001f800000 (XEN) 1: pnode 1, vcpus 2-3 (XEN) 000000001f800000 - 000000003f800000 root@test:~# numactl --hardware available: 2 nodes (0-1) node 0 cpus: 0 1 node 0 size: 411 MB node 0 free: 311 MB node 1 cpus: 2 3 node 1 size: 442 MB node 1 free: 406 MB node distances: node 0 1=20 0: 10 20=20 1: 20 10=20 If I build a PV guest, it breaks: root@Zhaman:~# xl create -c /etc/xen/test.cfg Parsing config from /etc/xen/test.cfg xc: error: panic: xc_dom_x86.c:940: arch_setup_meminit: failed to allocate = 0x20000 pages (v=3D1, p=3D1) : Internal error xc: error: panic: xc_dom_boot.c:155: xc_dom_boot_mem_init: can't allocate l= ow memory for domain: Out of memory libxl: error: libxl_dom.c:731:libxl__build_pv: xc_dom_boot_mem_init failed:= Device or resource busy libxl: error: libxl_create.c:1174:domcreate_rebuild_done: cannot (re-)build= domain: -3 libxl: error: libxl.c:1586:libxl__destroy_domid: non-existant domain 15 libxl: error: libxl.c:1544:domain_destroy_callback: unable to destroy guest= with domid 15 libxl: error: libxl.c:1471:domain_destroy_cb: destruction of domain 15 fail= ed (XEN) d0v1 Over-allocation for domain 15: 262656 > 262400 (XEN) memory.c:155:d0v1 Could not allocate order=3D9 extent: id=3D15 memfla= gs=3D210 (0 of 512) (XEN) d0v1 Over-allocation for domain 15: 262401 > 262400 (XEN) memory.c:155:d0v1 Could not allocate order=3D0 extent: id=3D15 memfla= gs=3D210 (256 of 131072) As said, I'll be looking into it in the next days. If, in the meanwhile, someone has any ideas, that would be much appreciated. :-) Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-9D+MXo3FtBcJ8y2oZuix Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEABECAAYFAlWT/HAACgkQk4XaBE3IOsRqnQCdFDEmuHUNXrCmR/KIPph7LbWK 17oAnR03Z+0JShuW0GLrp5Y82WEzjtcY =WHEq -----END PGP SIGNATURE----- --=-9D+MXo3FtBcJ8y2oZuix-- --===============3351657252356487176== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============3351657252356487176==--