* Error during domain creation on xen 4.8 @ 2017-06-13 16:29 Sarah Newman 2017-06-13 16:56 ` Wei Liu 0 siblings, 1 reply; 7+ messages in thread From: Sarah Newman @ 2017-06-13 16:29 UTC (permalink / raw) To: xen-devel Hi, With xen 4.8.1, I got the error message: libxl: error: libxl_dom.c:60:libxl__domain_cpupool: got info for dom2098, wanted dom2097 : No such file or directory This was while creating an HVM domain with a stub domain, probably concurrent to creating a PV domain. The domains were created as 2098 + 2099 I believe. I was also using oxenstored. Please keep me CC'ed as I am not subscribed. --Sarah _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Error during domain creation on xen 4.8 2017-06-13 16:29 Error during domain creation on xen 4.8 Sarah Newman @ 2017-06-13 16:56 ` Wei Liu 2017-06-13 17:08 ` Wei Liu 0 siblings, 1 reply; 7+ messages in thread From: Wei Liu @ 2017-06-13 16:56 UTC (permalink / raw) To: Sarah Newman; +Cc: xen-devel, Wei Liu, Ian Jackson On Tue, Jun 13, 2017 at 09:29:22AM -0700, Sarah Newman wrote: > Hi, > > With xen 4.8.1, I got the error message: > > libxl: error: libxl_dom.c:60:libxl__domain_cpupool: got info for dom2098, wanted dom2097 > : No such file or directory > > This was while creating an HVM domain with a stub domain, probably concurrent to creating a PV domain. The domains were created as 2098 + 2099 I > believe. I was also using oxenstored. The xc_domain_getinfolist function is a bit awful to work with. Can you reliable reproduce this? If so, can you provide xl -vvv create output? Wei. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Error during domain creation on xen 4.8 2017-06-13 16:56 ` Wei Liu @ 2017-06-13 17:08 ` Wei Liu 2017-06-13 17:28 ` Sarah Newman 0 siblings, 1 reply; 7+ messages in thread From: Wei Liu @ 2017-06-13 17:08 UTC (permalink / raw) To: Sarah Newman; +Cc: xen-devel, Wei Liu, Ian Jackson On Tue, Jun 13, 2017 at 05:56:26PM +0100, Wei Liu wrote: > On Tue, Jun 13, 2017 at 09:29:22AM -0700, Sarah Newman wrote: > > Hi, > > > > With xen 4.8.1, I got the error message: > > > > libxl: error: libxl_dom.c:60:libxl__domain_cpupool: got info for dom2098, wanted dom2097 > > : No such file or directory > > > > This was while creating an HVM domain with a stub domain, probably concurrent to creating a PV domain. The domains were created as 2098 + 2099 I > > believe. I was also using oxenstored. > > The xc_domain_getinfolist function is a bit awful to work with. To give you a bit more information: that function searches for N domains starting from the one you specify. That the parameter was 2097 but there wasn't dom2097 in Xen, so Xen returned 2098. The toolstack (correctly) bailed out. I'm still a bit baffled at this point. Maybe 2097 crashed? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Error during domain creation on xen 4.8 2017-06-13 17:08 ` Wei Liu @ 2017-06-13 17:28 ` Sarah Newman 2017-06-13 18:27 ` Sarah Newman 0 siblings, 1 reply; 7+ messages in thread From: Sarah Newman @ 2017-06-13 17:28 UTC (permalink / raw) To: Wei Liu; +Cc: xen-devel, Ian Jackson On 06/13/2017 10:08 AM, Wei Liu wrote: > On Tue, Jun 13, 2017 at 05:56:26PM +0100, Wei Liu wrote: >> On Tue, Jun 13, 2017 at 09:29:22AM -0700, Sarah Newman wrote: >>> Hi, >>> >>> With xen 4.8.1, I got the error message: >>> >>> libxl: error: libxl_dom.c:60:libxl__domain_cpupool: got info for dom2098, wanted dom2097 >>> : No such file or directory >>> >>> This was while creating an HVM domain with a stub domain, probably concurrent to creating a PV domain. The domains were created as 2098 + 2099 I >>> believe. I was also using oxenstored. >> >> The xc_domain_getinfolist function is a bit awful to work with. > > To give you a bit more information: that function searches for N domains > starting from the one you specify. > > That the parameter was 2097 but there wasn't dom2097 in Xen, so Xen > returned 2098. The toolstack (correctly) bailed out. > > I'm still a bit baffled at this point. Maybe 2097 crashed? > It was probably a domain shut down on purpose. Rerunning with -vvv. --Sarah _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Error during domain creation on xen 4.8 2017-06-13 17:28 ` Sarah Newman @ 2017-06-13 18:27 ` Sarah Newman 2017-06-14 9:38 ` Wei Liu 0 siblings, 1 reply; 7+ messages in thread From: Sarah Newman @ 2017-06-13 18:27 UTC (permalink / raw) To: Wei Liu; +Cc: xen-devel, Ian Jackson On 06/13/2017 10:28 AM, Sarah Newman wrote: > On 06/13/2017 10:08 AM, Wei Liu wrote: >> On Tue, Jun 13, 2017 at 05:56:26PM +0100, Wei Liu wrote: >>> On Tue, Jun 13, 2017 at 09:29:22AM -0700, Sarah Newman wrote: >>>> Hi, >>>> >>>> With xen 4.8.1, I got the error message: >>>> >>>> libxl: error: libxl_dom.c:60:libxl__domain_cpupool: got info for dom2098, wanted dom2097 >>>> : No such file or directory >>>> >>>> This was while creating an HVM domain with a stub domain, probably concurrent to creating a PV domain. The domains were created as 2098 + 2099 I >>>> believe. I was also using oxenstored. >>> >>> The xc_domain_getinfolist function is a bit awful to work with. >> >> To give you a bit more information: that function searches for N domains >> starting from the one you specify. >> >> That the parameter was 2097 but there wasn't dom2097 in Xen, so Xen >> returned 2098. The toolstack (correctly) bailed out. >> >> I'm still a bit baffled at this point. Maybe 2097 crashed? >> > > It was probably a domain shut down on purpose. Rerunning with -vvv. > > --Sarah > Could this be happening from the block script? There's a function that uses xenstore (I'm using oxenstored) and checks whether a given block device is in use by the dom0 or another guest. libxl: debug: libxl_device.c:1157:device_hotplug: calling hotplug script: /etc/xen/scripts/block add libxl: debug: libxl_device.c:1158:device_hotplug: extra args: libxl: debug: libxl_device.c:1166:device_hotplug: env: libxl: debug: libxl_device.c:1173:device_hotplug: script: /etc/xen/scripts/block libxl: debug: libxl_device.c:1173:device_hotplug: XENBUS_TYPE: vbd libxl: debug: libxl_device.c:1173:device_hotplug: XENBUS_PATH: backend/vbd/2853/268441856 libxl: debug: libxl_device.c:1173:device_hotplug: XENBUS_BASE_PATH: backend libxl: debug: libxl_aoutils.c:593:libxl__async_exec_start: forking to execute: /etc/xen/scripts/block add libxl: debug: libxl_event.c:686:libxl__ev_xswatch_deregister: watch w=0x1ad98f0: deregister unregistered libxl: debug: libxl_linux.c:200:libxl__get_hotplug_script_info: num_exec 1, not running hotplug scripts libxl: debug: libxl_device.c:1143:device_hotplug: No hotplug script to execute libxl: debug: libxl_event.c:686:libxl__ev_xswatch_deregister: watch w=0x1ad98f0: deregister unregistered libxl: error: libxl_dom.c:60:libxl__domain_cpupool: got info for dom2852, wanted dom2851 : No such file or directory libxl: debug: libxl_numa.c:502:libxl__get_numa_candidate: New best NUMA placement candidate found: nr_nodes=1, nr_cpus=20, nr_vcpus=184, free_memkb=151897 libxl: debug: libxl_numa.c:502:libxl__get_numa_candidate: New best NUMA placement candidate found: nr_nodes=1, nr_cpus=20, nr_vcpus=183, free_memkb=157905 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Error during domain creation on xen 4.8 2017-06-13 18:27 ` Sarah Newman @ 2017-06-14 9:38 ` Wei Liu 2017-06-14 13:48 ` Dario Faggioli 0 siblings, 1 reply; 7+ messages in thread From: Wei Liu @ 2017-06-14 9:38 UTC (permalink / raw) To: Sarah Newman; +Cc: xen-devel, Wei Liu, Ian Jackson On Tue, Jun 13, 2017 at 11:27:27AM -0700, Sarah Newman wrote: > On 06/13/2017 10:28 AM, Sarah Newman wrote: > > On 06/13/2017 10:08 AM, Wei Liu wrote: > >> On Tue, Jun 13, 2017 at 05:56:26PM +0100, Wei Liu wrote: > >>> On Tue, Jun 13, 2017 at 09:29:22AM -0700, Sarah Newman wrote: > >>>> Hi, > >>>> > >>>> With xen 4.8.1, I got the error message: > >>>> > >>>> libxl: error: libxl_dom.c:60:libxl__domain_cpupool: got info for dom2098, wanted dom2097 > >>>> : No such file or directory > >>>> > >>>> This was while creating an HVM domain with a stub domain, probably concurrent to creating a PV domain. The domains were created as 2098 + 2099 I > >>>> believe. I was also using oxenstored. > >>> > >>> The xc_domain_getinfolist function is a bit awful to work with. > >> > >> To give you a bit more information: that function searches for N domains > >> starting from the one you specify. > >> > >> That the parameter was 2097 but there wasn't dom2097 in Xen, so Xen > >> returned 2098. The toolstack (correctly) bailed out. > >> > >> I'm still a bit baffled at this point. Maybe 2097 crashed? > >> > > > > It was probably a domain shut down on purpose. Rerunning with -vvv. > > > > --Sarah > > > > Could this be happening from the block script? There's a function that uses xenstore (I'm using oxenstored) and checks whether a given block device is > in use by the dom0 or another guest. > > libxl: debug: libxl_device.c:1157:device_hotplug: calling hotplug script: /etc/xen/scripts/block add > libxl: debug: libxl_device.c:1158:device_hotplug: extra args: > libxl: debug: libxl_device.c:1166:device_hotplug: env: > libxl: debug: libxl_device.c:1173:device_hotplug: script: /etc/xen/scripts/block > libxl: debug: libxl_device.c:1173:device_hotplug: XENBUS_TYPE: vbd > libxl: debug: libxl_device.c:1173:device_hotplug: XENBUS_PATH: backend/vbd/2853/268441856 > libxl: debug: libxl_device.c:1173:device_hotplug: XENBUS_BASE_PATH: backend > libxl: debug: libxl_aoutils.c:593:libxl__async_exec_start: forking to execute: /etc/xen/scripts/block add > libxl: debug: libxl_event.c:686:libxl__ev_xswatch_deregister: watch w=0x1ad98f0: deregister unregistered > libxl: debug: libxl_linux.c:200:libxl__get_hotplug_script_info: num_exec 1, not running hotplug scripts > libxl: debug: libxl_device.c:1143:device_hotplug: No hotplug script to execute > libxl: debug: libxl_event.c:686:libxl__ev_xswatch_deregister: watch w=0x1ad98f0: deregister unregistered > libxl: error: libxl_dom.c:60:libxl__domain_cpupool: got info for dom2852, wanted dom2851 > : No such file or directory > libxl: debug: libxl_numa.c:502:libxl__get_numa_candidate: New best NUMA placement candidate found: nr_nodes=1, nr_cpus=20, nr_vcpus=184, free_memkb=151897 > libxl: debug: libxl_numa.c:502:libxl__get_numa_candidate: New best NUMA placement candidate found: nr_nodes=1, nr_cpus=20, nr_vcpus=183, free_memkb=157905 > No likely from the block script. The code in question is trying to get cpupool / NUMA information. Block does neither of those. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Error during domain creation on xen 4.8 2017-06-14 9:38 ` Wei Liu @ 2017-06-14 13:48 ` Dario Faggioli 0 siblings, 0 replies; 7+ messages in thread From: Dario Faggioli @ 2017-06-14 13:48 UTC (permalink / raw) To: Wei Liu, Sarah Newman; +Cc: xen-devel, Ian Jackson [-- Attachment #1.1: Type: text/plain, Size: 3383 bytes --] On Wed, 2017-06-14 at 10:38 +0100, Wei Liu wrote: > On Tue, Jun 13, 2017 at 11:27:27AM -0700, Sarah Newman wrote: > > On 06/13/2017 10:28 AM, Sarah Newman wrote: > > libxl: debug: libxl_device.c:1157:device_hotplug: calling hotplug > > script: /etc/xen/scripts/block add > > libxl: debug: libxl_device.c:1158:device_hotplug: extra args: > > libxl: debug: libxl_device.c:1166:device_hotplug: env: > > libxl: debug: libxl_device.c:1173:device_hotplug: script: > > /etc/xen/scripts/block > > libxl: debug: > > libxl_device.c:1173:device_hotplug: XENBUS_TYPE: vbd > > libxl: debug: > > libxl_device.c:1173:device_hotplug: XENBUS_PATH: > > backend/vbd/2853/268441856 > > libxl: debug: > > libxl_device.c:1173:device_hotplug: XENBUS_BASE_PATH: backend > > libxl: debug: libxl_aoutils.c:593:libxl__async_exec_start: forking > > to execute: /etc/xen/scripts/block add > > libxl: debug: libxl_event.c:686:libxl__ev_xswatch_deregister: watch > > w=0x1ad98f0: deregister unregistered > > libxl: debug: libxl_linux.c:200:libxl__get_hotplug_script_info: > > num_exec 1, not running hotplug scripts > > libxl: debug: libxl_device.c:1143:device_hotplug: No hotplug script > > to execute > > libxl: debug: libxl_event.c:686:libxl__ev_xswatch_deregister: watch > > w=0x1ad98f0: deregister unregistered > > libxl: error: libxl_dom.c:60:libxl__domain_cpupool: got info for > > dom2852, wanted dom2851 > > : No such file or directory > > libxl: debug: libxl_numa.c:502:libxl__get_numa_candidate: New best > > NUMA placement candidate found: nr_nodes=1, nr_cpus=20, > > nr_vcpus=184, free_memkb=151897 > > libxl: debug: libxl_numa.c:502:libxl__get_numa_candidate: New best > > NUMA placement candidate found: nr_nodes=1, nr_cpus=20, > > nr_vcpus=183, free_memkb=157905 > > > > No likely from the block script. The code in question is trying to > get > cpupool / NUMA information. Block does neither of those. > But then (as, in fact, is also said in the first message of the thread) it goes on, and domains are created, aren't they? I believe this is this message is produced by this call: static int nr_vcpus_on_nodes() { dinfo = libxl_list_domain(CTX, &nr_doms); if (dinfo == NULL) return ERROR_FAIL; ... for (i = 0; i < nr_doms; i++) { ... cpupool = libxl__domain_cpupool(gc, dinfo[i].domid); if (cpupool < 0) goto next; ... } in libxl_numa.c. What appears to have happened is that a domain disappeared (was shutdown, was destroyed, crashed, whatever) between when we got the list of existing domains (with libxl_list_domain()), and when we go through it, with the for(). Looking at other places, in libxl and xl, where libxl_list_domain() is used, that happens in a similar way, and I don't think we have anything in place for preventing this race to happen... and in fact, is not considered fatal. Perhaps we should reorg logging so that it will produce a WARN instead of an ERROR? Regards, Dario -- <<This happens because I choose it to happen!>> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) [-- Attachment #1.2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 819 bytes --] [-- Attachment #2: Type: text/plain, Size: 127 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-06-14 13:48 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2017-06-13 16:29 Error during domain creation on xen 4.8 Sarah Newman 2017-06-13 16:56 ` Wei Liu 2017-06-13 17:08 ` Wei Liu 2017-06-13 17:28 ` Sarah Newman 2017-06-13 18:27 ` Sarah Newman 2017-06-14 9:38 ` Wei Liu 2017-06-14 13:48 ` Dario Faggioli
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.