From mboxrd@z Thu Jan 1 00:00:00 1970 From: Atom2 Subject: Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough Date: Thu, 20 Mar 2014 20:32:47 +0100 Message-ID: <532B425F.4010701@web2web.at> References: <5325B828.1060303@web2web.at> <1395050430.4122.29.camel@kazak.uk.xensource.com> <53273B3C.40707@web2web.at> <1395137709.12847.29.camel@kazak.uk.xensource.com> <5328439B.8050807@web2web.at> <1395155249.12847.66.camel@kazak.uk.xensource.com> <5328E403.8010506@web2web.at> <1395228384.10203.65.camel@kazak.uk.xensource.com> <20140319130002.GC8694@phenom.dumpdata.com> <5329A3C0.3000609@web2web.at> <21289.48007.604643.322509@mariner.uk.xensource.com> <532A530D.1050504@web2web.at> <21290.54937.269235.163107@mariner.uk.xensource.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------030905030504030906080307" Return-path: In-Reply-To: <21290.54937.269235.163107@mariner.uk.xensource.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Ian Jackson Cc: xen-users@lists.xenproject.org, xen-devel , Ian Campbell , Roger Pau Monne List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --------------030905030504030906080307 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Am 20.03.14 12:52, schrieb Ian Jackson: > Atom2 writes ("Re: [Xen-devel] [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough"): >> the patch unfortunately doesn't apply to my sources - some comments to >> the reasons why further below. > > Here's a backport. I have compiled but not executed it. It compiled at my end as well, but I am sorry to report that the problem with the 40s delay persits. Attached please find the new xl-output created with xl -vvv create -F domain. This file again contains a few annotations with regards to where the delays happen. To my untrained eye it looks largely identical to the last xl-output with the obvious change of domain-id, addresses, line-numbers for debug output where changes in the sourve have happende and the use of the new function libxl__wait_for_backend_deprecated instead of libxl__wait_for_backend due to your patch, the latter of which I take as proof that your patches have been applied. For other obvious changes I have commented in the file on those lines that I could identify: There are a few new lines which were not there last time and now there's also a new 10s delay which, however, is only visible through xl -vvvv -F domain as the prompt has already returned by the time this new delay happens and this delay therefore is not visible in dom0. I hope that helps. Thanks for your continued support and best regards, Atom2 > >> Sorry for my delay in answering - this is a resend as the first e-Mail >> with uncompressed attachments did not go through. > >> Just FYI: the version I am using is 4.3.1-r5; I have attached the >> relevant source files referred to by your patches. > > Thanks, but our revision control system enables us to retrieve old > versions very easily :-). So there is not any need to provide us with > these files. > > Having said that, I have no record of 4.3.1-rc5. But I'm pretty sure > the patch below, which is against staging-4.3, will apply to your > tree. It applies cleanly to 4.3.0-rc5 and 4.3.1-rc2, which are my two > guesses as to which version you mean. > > Thanks, > Ian. > > From f9df128cd4d4ad6c7ed6ffd9bd8ba0633af78389 Mon Sep 17 00:00:00 2001 > From: Ian Jackson > Date: Wed, 19 Mar 2014 15:47:02 +0000 > Subject: [PATCH] libxl: Tolerate backend state "6" on pciback remove > > When shutting down a domain with pci passthrough, it can happen that > the backend has actually shut down (xenbus state 6) before we try to > remove it. When this happens, libxl would time out waiting for the > backend to reach state 4. > > Instead, deal with this by having libxl__wait_for_backend take a list > of suitable states. > > The arrangements are still fundamentally incorrect: > - libxl__wait_for_backend is a slow synchronous function, which is > forbidden; > - There is no way to deal properly with the various xenbus states > that might arise (including erroneous ones). > We will hopefully fix this later, although it's not trivial. For > the moment, rename the function to libxl__wait_for_backend_deprecated. > > Reported-by: Atom2 > Signed-off-by: Ian Jackson > CC: Atom2 > CC: Konrad Rzeszutek Wilk > CC: Roger Pau Monne > CC: Ian Campbell > > Backported to 4.3. Conflicts: > tools/libxl/libxl_device.c > tools/libxl/libxl_internal.h > Signed-off-by: Ian Jackson > --- > tools/libxl/libxl.c | 2 +- > tools/libxl/libxl_device.c | 21 ++++++++++++++------- > tools/libxl/libxl_internal.h | 3 ++- > tools/libxl/libxl_pci.c | 8 +++++--- > 4 files changed, 22 insertions(+), 12 deletions(-) > > diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c > index 3d9543b..c0cc0b7 100644 > --- a/tools/libxl/libxl.c > +++ b/tools/libxl/libxl.c > @@ -2689,7 +2689,7 @@ static void local_device_attach_cb(libxl__egc *egc, libxl__ao_device *aodev) > if (rc < 0) > goto out; > be_path = libxl__device_backend_path(gc, &device); > - rc = libxl__wait_for_backend(gc, be_path, "4"); > + rc = libxl__wait_for_backend_deprecated(gc, be_path, "4", (char*)0); > if (rc < 0) > goto out; > > diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c > index ea845b7..779b38b 100644 > --- a/tools/libxl/libxl_device.c > +++ b/tools/libxl/libxl_device.c > @@ -1094,7 +1094,8 @@ int libxl__wait_for_device_model(libxl__gc *gc, > check_callback, check_callback_userdata); > } > > -int libxl__wait_for_backend(libxl__gc *gc, char *be_path, char *state) > +int libxl__wait_for_backend_deprecated(libxl__gc *gc, const char *be_path, > + ...) > { > libxl_ctx *ctx = libxl__gc_owner(gc); > int watchdog = 100; > @@ -1115,13 +1116,19 @@ int libxl__wait_for_backend(libxl__gc *gc, char *be_path, char *state) > } > goto out; > } else { > - if (!strcmp(p, state)) { > - rc = 0; > - goto out; > - } else { > - usleep(100000); > - watchdog--; > + const char *want; > + va_list al; > + va_start(al,be_path); > + while ((want = va_arg(al, char*))) { > + if (!strcmp(p, want)) { > + va_end(al); > + rc = 0; > + goto out; > + } > } > + va_end(al); > + usleep(100000); > + watchdog--; > } > } > LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "Backend %s not ready", be_path); > diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h > index f051d91..4485c56 100644 > --- a/tools/libxl/libxl_internal.h > +++ b/tools/libxl/libxl_internal.h > @@ -944,7 +944,8 @@ _hidden char *libxl__device_frontend_path(libxl__gc *gc, libxl__device *device); > _hidden int libxl__parse_backend_path(libxl__gc *gc, const char *path, > libxl__device *dev); > _hidden int libxl__device_destroy(libxl__gc *gc, libxl__device *dev); > -_hidden int libxl__wait_for_backend(libxl__gc *gc, char *be_path, char *state); > +_hidden int libxl__wait_for_backend_deprecated(libxl__gc *gc, > + const char *be_path, ...) __attribute__((sentinel)); > _hidden int libxl__nic_type(libxl__gc *gc, libxl__device *dev, > libxl_nic_type *nictype); > > diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c > index 0295e0b..e22852c 100644 > --- a/tools/libxl/libxl_pci.c > +++ b/tools/libxl/libxl_pci.c > @@ -126,7 +126,7 @@ static int libxl__device_pci_add_xenstore(libxl__gc *gc, uint32_t domid, libxl_d > return ERROR_FAIL; > > if (!starting && domtype == LIBXL_DOMAIN_TYPE_PV) { > - if (libxl__wait_for_backend(gc, be_path, "4") < 0) > + if (libxl__wait_for_backend_deprecated(gc, be_path, "4", (char*)0) < 0) > return ERROR_FAIL; > } > > @@ -169,7 +169,8 @@ static int libxl__device_pci_remove_xenstore(libxl__gc *gc, uint32_t domid, libx > return ERROR_FAIL; > > if (domtype == LIBXL_DOMAIN_TYPE_PV) { > - if (libxl__wait_for_backend(gc, be_path, "4") < 0) { > + if (libxl__wait_for_backend_deprecated(gc, be_path, "4", "6", (char*)0) > + < 0) { > LIBXL__LOG(ctx, LIBXL__LOG_DEBUG, "pci backend at %s is not ready", be_path); > return ERROR_FAIL; > } > @@ -198,7 +199,8 @@ retry_transaction: > goto retry_transaction; > > if (domtype == LIBXL_DOMAIN_TYPE_PV) { > - if (libxl__wait_for_backend(gc, be_path, "4") < 0) { > + if (libxl__wait_for_backend_deprecated(gc, be_path, "4", "6", (char*)0) > + < 0) { > LIBXL__LOG(ctx, LIBXL__LOG_DEBUG, "pci backend at %s is not ready", be_path); > return ERROR_FAIL; > } > --------------030905030504030906080307 Content-Type: text/plain; charset=windows-1252; name="xl-output" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="xl-output" Parsing config from 5:voip.9 libxl: debug: libxl_create.c:1230:do_domain_create: ao 0x7fe37ba02850: create: how=(nil) callback=(nil) poller=0x7fe37ba03d70 libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda1 spec.backend=unknown libxl: debug: libxl_device.c:296:libxl__device_disk_set_backend: Disk vdev=xvda1, using backend phy libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda2 spec.backend=unknown libxl: debug: libxl_device.c:296:libxl__device_disk_set_backend: Disk vdev=xvda2, using backend phy libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda3 spec.backend=unknown libxl: debug: libxl_device.c:296:libxl__device_disk_set_backend: Disk vdev=xvda3, using backend phy libxl: debug: libxl_create.c:675:initiate_domain_create: running bootloader libxl: debug: libxl_bootloader.c:327:libxl__bootloader_run: no bootloader configured, using user supplied kernel libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba02bd8: deregister unregistered libxl: debug: libxl_x86.c:82:e820_sanitize: Memory: 1048576kB End of RAM: 0x20000 (PFN) Delta: 524288kB, PCI start: 524288kB (0x20000 PFN), Balloon 0kB libxl: debug: libxl_x86.c:201:e820_sanitize: : [0 -> 20000] RAM libxl: debug: libxl_x86.c:201:e820_sanitize: : [20000 -> 20200] Reserved libxl: debug: libxl_x86.c:201:e820_sanitize: : [20200 -> 40000] Unusable libxl: debug: libxl_x86.c:201:e820_sanitize: : [40000 -> 40200] Reserved libxl: debug: libxl_x86.c:201:e820_sanitize: : [40200 -> db9f0] Unusable libxl: debug: libxl_x86.c:201:e820_sanitize: : [db9f0 -> dc0da] Reserved libxl: debug: libxl_x86.c:201:e820_sanitize: : [dc0da -> dc1f9] ACPI NVS libxl: debug: libxl_x86.c:201:e820_sanitize: : [dc1f9 -> dc651] Reserved libxl: debug: libxl_x86.c:201:e820_sanitize: : [dc651 -> dc652] Unusable libxl: debug: libxl_x86.c:201:e820_sanitize: : [dc652 -> dc695] ACPI NVS libxl: debug: libxl_x86.c:201:e820_sanitize: : [dc695 -> dcdba] Unusable libxl: debug: libxl_x86.c:201:e820_sanitize: : [dcdba -> dcff2] Reserved libxl: debug: libxl_x86.c:201:e820_sanitize: : [dcff2 -> dd000] Unusable libxl: debug: libxl_x86.c:201:e820_sanitize: : [dd800 -> dfa00] Reserved libxl: debug: libxl_x86.c:201:e820_sanitize: : [f8000 -> fc000] Reserved libxl: debug: libxl_x86.c:201:e820_sanitize: : [fec00 -> fec01] Reserved libxl: debug: libxl_x86.c:201:e820_sanitize: : [fed00 -> fed04] Reserved libxl: debug: libxl_x86.c:201:e820_sanitize: : [fed1c -> fed20] Reserved libxl: debug: libxl_x86.c:201:e820_sanitize: : [fee00 -> fee01] Reserved libxl: debug: libxl_x86.c:201:e820_sanitize: : [fee01 -> fef00] Reserved libxl: debug: libxl_x86.c:201:e820_sanitize: : [ff000 -> 100000] Reserved libxl: debug: libxl_x86.c:201:e820_sanitize: : [100000 -> 120000] RAM domainbuilder: detail: xc_dom_allocate: cmdline="", features="(null)" libxl: debug: libxl_dom.c:341:libxl__build_pv: pv kernel mapped 0 path /usr/lib/xen/boot/pv-grub-x86_64.gz domainbuilder: detail: xc_dom_kernel_file: filename="/usr/lib/xen/boot/pv-grub-x86_64.gz" domainbuilder: detail: xc_dom_malloc_filemap : 1240 kB domainbuilder: detail: xc_dom_malloc : 15110 kB domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x13631e -> 0xec1ae1 domainbuilder: detail: xc_dom_ramdisk_file: filename="/etc/xen/guests/grub.d/voip.grub" domainbuilder: detail: xc_dom_boot_xen_init: ver 4.3, caps xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 domainbuilder: detail: xc_dom_parse_image: called domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ... domainbuilder: detail: loader probe failed domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ... domainbuilder: detail: xc_dom_probe_bzimage_kernel: kernel is not a bzImage domainbuilder: detail: loader probe failed domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ... domainbuilder: detail: loader probe OK xc: detail: elf_parse_binary: phdr: paddr=0x0 memsz=0x9a96e0 xc: detail: elf_parse_binary: memory: 0x0 -> 0x9a96e0 xc: detail: elf_xen_parse: __xen_guest: "GUEST_OS=Mini-OS,XEN_VER=xen-3.0,VIRT_BASE=0x0,ELF_PADDR_OFFSET=0x0,HYPERCALL_PAGE=0x2,LOADER=generic" xc: detail: elf_xen_parse_guest_info: GUEST_OS="Mini-OS" xc: detail: elf_xen_parse_guest_info: XEN_VER="xen-3.0" xc: detail: elf_xen_parse_guest_info: VIRT_BASE="0x0" xc: detail: elf_xen_parse_guest_info: ELF_PADDR_OFFSET="0x0" xc: detail: elf_xen_parse_guest_info: HYPERCALL_PAGE="0x2" xc: detail: elf_xen_parse_guest_info: LOADER="generic" xc: detail: elf_xen_addr_calc_check: addresses: xc: detail: virt_base = 0x0 xc: detail: elf_paddr_offset = 0x0 xc: detail: virt_offset = 0x0 xc: detail: virt_kstart = 0x0 xc: detail: virt_kend = 0x9a96e0 xc: detail: virt_entry = 0x0 xc: detail: p2m_base = 0xffffffffffffffff domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-x86_64: 0x0 -> 0x9a96e0 domainbuilder: detail: xc_dom_mem_init: mem 1024 MB, pages 0x40000 pages, 4k each domainbuilder: detail: xc_dom_mem_init: 0x40000 pages domainbuilder: detail: xc_dom_boot_mem_init: called domainbuilder: detail: x86_compat: guest xen-3.0-x86_64, address size 64 domainbuilder: detail: xc_dom_malloc : 2048 kB domainbuilder: detail: xc_dom_build_image: called domainbuilder: detail: xc_dom_alloc_segment: kernel : 0x0 -> 0x9aa000 (pfn 0x0 + 0x9aa pages) domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x0+0x9aa at 0x7fe377817000 xc: detail: elf_load_binary: phdr 0 at 0x7fe377817000 -> 0x7fe3781c06e0 domainbuilder: detail: xc_dom_alloc_segment: ramdisk : 0x9aa000 -> 0x9ab000 (pfn 0x9aa + 0x1 pages) domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x9aa+0x1 at 0x7fe37b7ce000 domainbuilder: detail: xc_dom_alloc_segment: phys2mach : 0x9ab000 -> 0xbab000 (pfn 0x9ab + 0x200 pages) domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x9ab+0x200 at 0x7fe377617000 domainbuilder: detail: xc_dom_alloc_page : start info : 0xbab000 (pfn 0xbab) domainbuilder: detail: xc_dom_alloc_page : xenstore : 0xbac000 (pfn 0xbac) domainbuilder: detail: xc_dom_alloc_page : console : 0xbad000 (pfn 0xbad) domainbuilder: detail: nr_page_tables: 0x0000ffffffffffff/48: 0x0000000000000000 -> 0x0000ffffffffffff, 1 table(s) domainbuilder: detail: nr_page_tables: 0x0000007fffffffff/39: 0x0000000000000000 -> 0x0000007fffffffff, 1 table(s) domainbuilder: detail: nr_page_tables: 0x000000003fffffff/30: 0x0000000000000000 -> 0x000000003fffffff, 1 table(s) domainbuilder: detail: nr_page_tables: 0x00000000001fffff/21: 0x0000000000000000 -> 0x0000000000ffffff, 8 table(s) domainbuilder: detail: xc_dom_alloc_segment: page tables : 0xbae000 -> 0xbb9000 (pfn 0xbae + 0xb pages) domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0xbae+0xb at 0x7fe37b680000 domainbuilder: detail: xc_dom_alloc_page : boot stack : 0xbb9000 (pfn 0xbb9) domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0xbba000 domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0x1000000 domainbuilder: detail: xc_dom_boot_image: called domainbuilder: detail: arch_setup_bootearly: doing nothing domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64 <= matches domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32 domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_64 domainbuilder: detail: xc_dom_update_guest_p2m: dst 64bit, pages 0x40000 domainbuilder: detail: clear_page: pfn 0xbad, mfn 0x71aabe domainbuilder: detail: clear_page: pfn 0xbac, mfn 0x71aabf domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0xbab+0x1 at 0x7fe37b7cb000 domainbuilder: detail: start_info_x86_64: called domainbuilder: detail: setup_hypercall_page: vaddr=0x2000 pfn=0x2 domainbuilder: detail: domain builder memory footprint domainbuilder: detail: allocated domainbuilder: detail: malloc : 17231 kB domainbuilder: detail: anon mmap : 0 bytes domainbuilder: detail: mapped domainbuilder: detail: file mmap : 1241 kB domainbuilder: detail: domU mmap : 11996 kB domainbuilder: detail: arch_setup_bootlate: shared_info: pfn 0x0, mfn 0xdb70f domainbuilder: detail: shared_info_x86_64: called domainbuilder: detail: vcpu_x86_64: called domainbuilder: detail: vcpu_x86_64: cr3: pfn 0xbae mfn 0x71aabd domainbuilder: detail: launch_vm: called, ctxt=0x7fff80e7a330 domainbuilder: detail: xc_dom_release: called libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda1 spec.backend=phy libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba04098 wpath=/local/domain/0/backend/vbd/4/51713/state token=3/0: register slotnum=3 libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda2 spec.backend=phy libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda2 spec.backend=phy libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba050b8 wpath=/local/domain/0/backend/vbd/4/51714/state token=2/1: register slotnum=2 libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda3 spec.backend=phy libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda3 spec.backend=phy libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba048a8 wpath=/local/domain/0/backend/vbd/4/51715/state token=1/2: register slotnum=1 libxl: debug: libxl_create.c:1243:do_domain_create: ao 0x7fe37ba02850: inprogress: poller=0x7fe37ba03d70, flags=i libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba04098 wpath=/local/domain/0/backend/vbd/4/51713/state token=3/0: event epath=/local/domain/0/backend/vbd/4/51713/state libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51713/state wanted state 2 ok libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba04098 wpath=/local/domain/0/backend/vbd/4/51713/state token=3/0: deregister slotnum=3 libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba04098: deregister unregistered libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block add libxl: debug: libxl_event.c:472:watchfd_callback: watch epath=/local/domain/0/backend/vbd/4/51713/state token=3/0: empty slot libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba050b8 wpath=/local/domain/0/backend/vbd/4/51714/state token=2/1: event epath=/local/domain/0/backend/vbd/4/51714/state libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51714/state wanted state 2 ok libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba050b8 wpath=/local/domain/0/backend/vbd/4/51714/state token=2/1: deregister slotnum=2 libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba050b8: deregister unregistered libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block add libxl: debug: libxl_event.c:472:watchfd_callback: watch epath=/local/domain/0/backend/vbd/4/51714/state token=2/1: empty slot libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba048a8 wpath=/local/domain/0/backend/vbd/4/51715/state token=1/2: event epath=/local/domain/0/backend/vbd/4/51715/state libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51715/state wanted state 2 ok libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba048a8 wpath=/local/domain/0/backend/vbd/4/51715/state token=1/2: deregister slotnum=1 libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba048a8: deregister unregistered libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block add libxl: debug: libxl_event.c:472:watchfd_callback: watch epath=/local/domain/0/backend/vbd/4/51715/state token=1/2: empty slot libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba091e8 wpath=/local/domain/0/backend/vif/4/0/state token=1/3: register slotnum=1 libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba091e8 wpath=/local/domain/0/backend/vif/4/0/state token=1/3: event epath=/local/domain/0/backend/vif/4/0/state libxl: debug: libxl_event.c:647:devstate_watch_callback: backend /local/domain/0/backend/vif/4/0/state wanted state 2 still waiting state 1 libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba091e8 wpath=/local/domain/0/backend/vif/4/0/state token=1/3: event epath=/local/domain/0/backend/vif/4/0/state libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vif/4/0/state wanted state 2 ok libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba091e8 wpath=/local/domain/0/backend/vif/4/0/state token=1/3: deregister slotnum=1 libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba091e8: deregister unregistered libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/vif-bridge online libxl: error: libxl_pci.c:992:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:09:02.0 libxl: debug: libxl_pci.c:81:libxl__create_pci_backend: Creating pci backend libxl: debug: libxl_event.c:1737:libxl__ao_progress_report: ao 0x7fe37ba02850: progress report: ignored libxl: debug: libxl_event.c:1569:libxl__ao_complete: ao 0x7fe37ba02850: complete, rc=0 libxl: debug: libxl_event.c:1541:libxl__ao__destroy: ao 0x7fe37ba02850: destroy Waiting for domain voip (domid 4) to die [pid 2409] libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba007e0 wpath=@releaseDomain token=1/4: register slotnum=1 libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba007e0 wpath=@releaseDomain token=1/4: event epath=@releaseDomain libxl: debug: libxl.c:1000:domain_death_xswatch_callback: [evg=0x7fe37ba08900:4] from domid=4 nentries=1 rc=1 libxl: debug: libxl.c:1011:domain_death_xswatch_callback: [evg=0x7fe37ba08900:4] got=domaininfos[0] got->domain=4 libxl: debug: libxl.c:1038:domain_death_xswatch_callback: exists shutdown_reported=0 dominf.flags=ffff0020 libxl: debug: libxl.c:1004:domain_death_xswatch_callback: [evg=0] all reported libxl: debug: libxl.c:1068:domain_death_xswatch_callback: domain death search done Domain 4 has shut down, reason code 0 0x0 Action for shutdown reason code 0 is destroy Domain 4 needs to be cleaned up: destroying the domain libxl: debug: libxl.c:1252:libxl_domain_destroy: ao 0x7fe37ba02850: create: how=(nil) callback=(nil) poller=0x7fe37ba03d70 libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission irq=17 libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated: Backend /local/domain/0/backend/pci/4/0 not ready libxl: debug: libxl_pci.c:204:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/4/0 is not ready libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission irq=16 libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated: Backend /local/domain/0/backend/pci/4/0 not ready libxl: debug: libxl_pci.c:174:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/4/0 is not ready libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission irq=18 libxl: error: libxl_pci.c:992:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:09:02.0 libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated: Backend /local/domain/0/backend/pci/4/0 not ready libxl: debug: libxl_pci.c:174:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/4/0 is not ready libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission irq=23 libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated: Backend /local/domain/0/backend/pci/4/0 not ready libxl: debug: libxl_pci.c:174:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/4/0 is not ready libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba048a8 wpath=/local/domain/0/backend/vbd/4/51713/state token=2/5: register slotnum=2 libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba091e8 wpath=/local/domain/0/backend/vbd/4/51714/state token=3/6: register slotnum=3 libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba0c298 wpath=/local/domain/0/backend/vbd/4/51715/state token=0/7: register slotnum=0 libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba043c8 wpath=/local/domain/0/backend/vif/4/0/state token=19/8: register slotnum=19 libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba05408 wpath=/local/domain/0/backend/pci/4/0/state token=18/9: register slotnum=18 libxl: debug: libxl.c:1261:libxl_domain_destroy: ao 0x7fe37ba02850: inprogress: poller=0x7fe37ba03d70, flags=i libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba048a8 wpath=/local/domain/0/backend/vbd/4/51713/state token=2/5: event epath=/local/domain/0/backend/vbd/4/51713/state libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51713/state wanted state 6 ok libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba048a8 wpath=/local/domain/0/backend/vbd/4/51713/state token=2/5: deregister slotnum=2 libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba048a8: deregister unregistered libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block remove libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba091e8 wpath=/local/domain/0/backend/vbd/4/51714/state token=3/6: event epath=/local/domain/0/backend/vbd/4/51714/state libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51714/state wanted state 6 ok libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba091e8 wpath=/local/domain/0/backend/vbd/4/51714/state token=3/6: deregister slotnum=3 libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba091e8: deregister unregistered libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block remove libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba0c298 wpath=/local/domain/0/backend/vbd/4/51715/state token=0/7: event epath=/local/domain/0/backend/vbd/4/51715/state libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51715/state wanted state 6 ok libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba0c298 wpath=/local/domain/0/backend/vbd/4/51715/state token=0/7: deregister slotnum=0 libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba0c298: deregister unregistered libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block remove libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba043c8 wpath=/local/domain/0/backend/vif/4/0/state token=19/8: event epath=/local/domain/0/backend/vif/4/0/state libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vif/4/0/state wanted state 6 ok libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba043c8 wpath=/local/domain/0/backend/vif/4/0/state token=19/8: deregister slotnum=19 libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba043c8: deregister unregistered libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/vif-bridge offline libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba05408 wpath=/local/domain/0/backend/pci/4/0/state token=18/9: event epath=/local/domain/0/backend/pci/4/0/state libxl: debug: libxl_event.c:647:devstate_watch_callback: backend /local/domain/0/backend/pci/4/0/state wanted state 6 still waiting state 5 libxl: debug: libxl_event.c:661:devstate_timeout: backend /local/domain/0/backend/pci/4/0/state wanted state 6 timed out libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba05408 wpath=/local/domain/0/backend/pci/4/0/state token=18/9: deregister slotnum=18 libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba05408: deregister unregistered libxl: error: libxl_device.c:894:device_backend_callback: unable to remove device with path /local/domain/0/backend/pci/4/0 libxl: error: libxl.c:1452:devices_destroy_cb: libxl__devices_destroy failed for 4 libxl: debug: libxl_event.c:1569:libxl__ao_complete: ao 0x7fe37ba02850: complete, rc=0 libxl: debug: libxl_event.c:1541:libxl__ao__destroy: ao 0x7fe37ba02850: destroy Done. Exiting now libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba007e0 wpath=@releaseDomain token=1/4: deregister slotnum=1 xc: debug: hypercall buffer: total allocations:554 total releases:554 xc: debug: hypercall buffer: current allocations:0 maximum allocations:2 xc: debug: hypercall buffer: cache current size:2 xc: debug: hypercall buffer: cache hits:548 misses:2 toobig:4 --------------030905030504030906080307 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --------------030905030504030906080307--