From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dario Faggioli Subject: Re: Error during domain creation on xen 4.8 Date: Wed, 14 Jun 2017 15:48:37 +0200 Message-ID: <1497448117.20371.1.camel@citrix.com> References: <20170613165626.wy6qyqum4a76j3yx@citrix.com> <20170613170815.f5ulsbiaqg3gyvk3@citrix.com> <8e759444-202d-ce06-002a-e4a61790d3cf@prgmr.com> <03b6c5ad-47ba-ccd1-3076-a40b3ec2f80c@prgmr.com> <20170614093833.nqc4vc7ooz2d66w6@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============6378573093785384401==" Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dL8fC-0007Fy-EA for xen-devel@lists.xenproject.org; Wed, 14 Jun 2017 13:48:50 +0000 In-Reply-To: <20170614093833.nqc4vc7ooz2d66w6@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Wei Liu , Sarah Newman Cc: xen-devel , Ian Jackson List-Id: xen-devel@lists.xenproject.org --===============6378573093785384401== Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="=-+lTowHkmj2djS8vHBE6D" --=-+lTowHkmj2djS8vHBE6D Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, 2017-06-14 at 10:38 +0100, Wei Liu wrote: > On Tue, Jun 13, 2017 at 11:27:27AM -0700, Sarah Newman wrote: > > On 06/13/2017 10:28 AM, Sarah Newman wrote: > > libxl: debug: libxl_device.c:1157:device_hotplug: calling hotplug > > script: /etc/xen/scripts/block add > > libxl: debug: libxl_device.c:1158:device_hotplug: extra args: > > libxl: debug: libxl_device.c:1166:device_hotplug: env: > > libxl: debug: libxl_device.c:1173:device_hotplug:=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0script: > > /etc/xen/scripts/block > > libxl: debug: > > libxl_device.c:1173:device_hotplug:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0XENBUS_TYPE: vbd > > libxl: debug: > > libxl_device.c:1173:device_hotplug:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0XENBUS_PATH: > > backend/vbd/2853/268441856 > > libxl: debug: > > libxl_device.c:1173:device_hotplug:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0XENBUS_BASE_PATH: backend > > libxl: debug: libxl_aoutils.c:593:libxl__async_exec_start: forking > > to execute: /etc/xen/scripts/block add > > libxl: debug: libxl_event.c:686:libxl__ev_xswatch_deregister: watch > > w=3D0x1ad98f0: deregister unregistered > > libxl: debug: libxl_linux.c:200:libxl__get_hotplug_script_info: > > num_exec 1, not running hotplug scripts > > libxl: debug: libxl_device.c:1143:device_hotplug: No hotplug script > > to execute > > libxl: debug: libxl_event.c:686:libxl__ev_xswatch_deregister: watch > > w=3D0x1ad98f0: deregister unregistered > > libxl: error: libxl_dom.c:60:libxl__domain_cpupool: got info for > > dom2852, wanted dom2851 > > : No such file or directory > > libxl: debug: libxl_numa.c:502:libxl__get_numa_candidate: New best > > NUMA placement candidate found: nr_nodes=3D1, nr_cpus=3D20, > > nr_vcpus=3D184, free_memkb=3D151897 > > libxl: debug: libxl_numa.c:502:libxl__get_numa_candidate: New best > > NUMA placement candidate found: nr_nodes=3D1, nr_cpus=3D20, > > nr_vcpus=3D183, free_memkb=3D157905 > >=20 >=20 > No likely from the block script. The code in question is trying to > get > cpupool / NUMA information. Block does neither of those. > But then (as, in fact, is also said in the first message of the thread) it goes on, and domains are created, aren't they? I believe this is this message is produced by this call: static int nr_vcpus_on_nodes() { =C2=A0=C2=A0=C2=A0=C2=A0dinfo =3D libxl_list_domain(CTX, &nr_doms); =C2=A0=C2=A0=C2=A0=C2=A0if (dinfo =3D=3D NULL) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0return ERROR_FAIL; ... for (i =3D 0; i < nr_doms; i++) { ... =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0cpupool =3D libxl__domain_c= pupool(gc, dinfo[i].domid); =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0if (cpupool < 0) =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0got= o next; ... } in libxl_numa.c. What appears to have happened is that a domain disappeared (was shutdown, was destroyed, crashed, whatever) between when we got the list of existing domains (with libxl_list_domain()), and when we go through it, with the for(). Looking at other places, in libxl and xl, where libxl_list_domain() is used, that happens in a similar way, and I don't think we have anything in place for preventing this race to happen... and in fact, is not considered fatal. Perhaps we should reorg logging so that it will produce a WARN instead of an ERROR? Regards, Dario --=20 <> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) --=-+lTowHkmj2djS8vHBE6D Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCAAGBQJZQT62AAoJEBZCeImluHPueuwP/2kdma3PsM1Rnr22aa5Bi0bC 8n1F4Sm94GJHt9uOvN0gOvagNpWRWyu+0xLnI9iNpqnGXf0iT9JBBSxMyXrX97ki JsiiOM/vIVNyZETS6BYunNK08w5bFNgZFA4WPXQVaGD/h86znv7kJuolz9w2LQ9k wL40kcS+/li7d5s7/9R62dvvQFVetytiQmDIu0UDz1r/PrqvFhMqgkTnKwaAcgtz k/WZ1MNf7NwuEpp2EFIppcixcR3fW0gyuhohlisBlUFrDkdNaBJokQO4/UIOs7OL 8IeIq+ABtapnou9zIfeAakqj3e6Kxp7lSmz790ankaNke1L489hHImB5GpHNPnOL W1JRXv6EKuk8w3ioPPpTIvCV7RTckBlK+kgzhrffd80z5Iya9gbWvW49PgzHpJYh v//4rc4um668MCF4K748MM88q0xoTayt5wE0IkRGSAMxIiy3LJ8FADTchI5LxSvY ecLtGFrwzOribmdbtkomKaVR4DNNnHIVoA4Y+O7xPp9L9r9uShDsad5B3TpHooa7 EGlYqWIHVotVbmvhEljxiJ+pfM9YCFGl+2RYa2bF2pgEX5TKoWl6sDJTW795uBjW e11VlbDvpmQSowK+QhsT/Rp5OXwplZgQQoYABYTfm6nuFqdOGBivJnOIqA+T3wOv 5GrcMvzyDNptdNnX6mWJ =iQA2 -----END PGP SIGNATURE----- --=-+lTowHkmj2djS8vHBE6D-- --===============6378573093785384401== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwczovL2xpc3RzLnhlbi5v cmcveGVuLWRldmVsCg== --===============6378573093785384401==--