* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
[not found] ` <53273B3C.40707@web2web.at>
@ 2014-03-18 10:15 ` Ian Campbell
2014-03-18 13:01 ` Atom2
0 siblings, 1 reply; 23+ messages in thread
From: Ian Campbell @ 2014-03-18 10:15 UTC (permalink / raw)
To: Atom2; +Cc: xen-devel, Ian Jackson, Roger Pau Monne, xen-users
Adding xen-devel. Full thread starts at
http://lists.xen.org/archives/html/xen-users/2014-03/msg00102.html
On Mon, 2014-03-17 at 19:13 +0100, Atom2 wrote:
> > Any chance you could try 4.3.2, or even 4.4.0?
> Unfortunately neither of these versions are currently available as
> stable ebuilds for my distribution, but I assume it shouldn't be long
> before there's some movement.
Looking at the diff to tools/libxl/libxl_pci.c I don't see any pertinent
looking fixes so it seems probably this issue still exists.
> >
> >> The system is capable of vt-d and uses a Xeon E3-1260L processor.
> >>
> >> Do these observations ring a bell with anybody or is this even expected
> >> behaviour. If this is not normal - which I would expect as I have not
> >> been able to find any information relating to substantial delays during
> >> shutdown - how would I go about getting to the grounds of this?
> >
> > My guess would be that xl process which is managing the domain destroy
> > is waiting for something (perhaps pciback) to confirm shutdown for each
> > device and this is timing out in series, leading to the delays. You
> > might find something in the logs /var/log/xen pointing to something like
> > this.
> >
> > If not then if you start the guest with "xl -vvv create -F <cfg>" then
> > the xl process which is monitoring the domain will stay in the
> > foreground and be logging to stdout (I think). If you then issue the
> > shutdown from another shell perhaps there will be some obvious gaps in
> > the logs as things shutdown which might help.
> That worked and there also was some output - please find the log from
> start to finnish attached to this mail. I have marked various points in
> the log: First the point where the startup was done and the domU was
> live and secondly those 4 points in time (or rather output) where the
> 10s delay occured.
Quoting the relevant bit for -devel, full log is at
http://lists.xen.org/archives/html/xen-users/2014-03/txtl6VscE4NMf.txt:
Domain 3 has shut down, reason code 0 0x0
Action for shutdown reason code 0 is destroy
Domain 3 needs to be cleaned up: destroying the domain
libxl: debug: libxl.c:1252:libxl_domain_destroy: ao 0x7f1dddf2b850: create: how=(nil) callback=(nil) poller=0x7f1dddf2cd70
libxl: error: libxl_pci.c:1248:do_pci_remove: xc_domain_irq_permission irq=17
<NOTE: at this point a 10s pause happens>
libxl: error: libxl_device.c:1127:libxl__wait_for_backend: Backend /local/domain/0/backend/pci/3/0 not ready
libxl: debug: libxl_pci.c:173:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/3/0 is not ready
libxl: error: libxl_pci.c:1248:do_pci_remove: xc_domain_irq_permission irq=16
<NOTE: at this point a 10s pause happens>
libxl: error: libxl_device.c:1127:libxl__wait_for_backend: Backend /local/domain/0/backend/pci/3/0 not ready
libxl: debug: libxl_pci.c:173:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/3/0 is not ready
[repeat for more devices]
Do you get anything in "xl dmesg" or dom0's "dmesg" corresponding to
these events?
Looking at do_pci_remove after the call to xc_domain_irq_permission
(which fails, but I don't think that relates to the delay) we then call
(conditionally) libxl__device_pci_reset, xc_deassign_device,
libxl__device_pci_remove_common and libxl__device_pci_remove_xenstore,
with no logging to indicate which we are calling (not helpful!).
The "is not ready" message comes from libxl__device_pci_remove_xenstore
which calls libxl__wait_for_backend. The latter has been rewritten a bit
since 4.3.1 but not in a way which I think would affect this case.
libxl__wait_for_backend does have a usleep(10000) in it -- which is
certainly the source of the delay, but I'd like to explain how we got to
waiting like that anyway (IanJ: do you have PCI on your hitlist for
asyncing up?)
This thing about pciback not being ready rings a bell. I've cc'd a few
folks who I think might remember more.
While the domain is happily running can you provide the output of
"xenstore-ls -fp" -- I'm curious what state pciback is in. It should be
4, if not then that would be the problem.
> BTW: I don't know whether it makes any difference, but I am only using
> xen-pciback.hide=(bb:dd.f)(...) on the grub command line for a number of
> devices including those that I pass through to this domU - there's
> nothing else happening in the dom0 with those devices priot to starting
> the domU and there are also no driver modules available for any of the
> hidden hardware (except for one of the hidden USB Controllers of the
> motherboard which is also passed through) in dom0.
I don't think that should matter here.
Ian.
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-18 10:15 ` [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough Ian Campbell
@ 2014-03-18 13:01 ` Atom2
2014-03-18 15:07 ` Ian Campbell
0 siblings, 1 reply; 23+ messages in thread
From: Atom2 @ 2014-03-18 13:01 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel, Ian Jackson, Roger Pau Monne, xen-users
[-- Attachment #1: Type: text/plain, Size: 9075 bytes --]
Am 18.03.14 11:15, schrieb Ian Campbell:
> Adding xen-devel. Full thread starts at
> http://lists.xen.org/archives/html/xen-users/2014-03/msg00102.html
>
> On Mon, 2014-03-17 at 19:13 +0100, Atom2 wrote:
>
>>> Any chance you could try 4.3.2, or even 4.4.0?
>> Unfortunately neither of these versions are currently available as
>> stable ebuilds for my distribution, but I assume it shouldn't be long
>> before there's some movement.
>
> Looking at the diff to tools/libxl/libxl_pci.c I don't see any pertinent
> looking fixes so it seems probably this issue still exists.
>
>>>
>>>> The system is capable of vt-d and uses a Xeon E3-1260L processor.
>>>>
>>>> Do these observations ring a bell with anybody or is this even expected
>>>> behaviour. If this is not normal - which I would expect as I have not
>>>> been able to find any information relating to substantial delays during
>>>> shutdown - how would I go about getting to the grounds of this?
>>>
>>> My guess would be that xl process which is managing the domain destroy
>>> is waiting for something (perhaps pciback) to confirm shutdown for each
>>> device and this is timing out in series, leading to the delays. You
>>> might find something in the logs /var/log/xen pointing to something like
>>> this.
>>>
>>> If not then if you start the guest with "xl -vvv create -F <cfg>" then
>>> the xl process which is monitoring the domain will stay in the
>>> foreground and be logging to stdout (I think). If you then issue the
>>> shutdown from another shell perhaps there will be some obvious gaps in
>>> the logs as things shutdown which might help.
>> That worked and there also was some output - please find the log from
>> start to finnish attached to this mail. I have marked various points in
>> the log: First the point where the startup was done and the domU was
>> live and secondly those 4 points in time (or rather output) where the
>> 10s delay occured.
>
> Quoting the relevant bit for -devel, full log is at
> http://lists.xen.org/archives/html/xen-users/2014-03/txtl6VscE4NMf.txt:
>
> Domain 3 has shut down, reason code 0 0x0
> Action for shutdown reason code 0 is destroy
> Domain 3 needs to be cleaned up: destroying the domain
> libxl: debug: libxl.c:1252:libxl_domain_destroy: ao 0x7f1dddf2b850: create: how=(nil) callback=(nil) poller=0x7f1dddf2cd70
> libxl: error: libxl_pci.c:1248:do_pci_remove: xc_domain_irq_permission irq=17
>
> <NOTE: at this point a 10s pause happens>
>
> libxl: error: libxl_device.c:1127:libxl__wait_for_backend: Backend /local/domain/0/backend/pci/3/0 not ready
> libxl: debug: libxl_pci.c:173:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/3/0 is not ready
> libxl: error: libxl_pci.c:1248:do_pci_remove: xc_domain_irq_permission irq=16
>
> <NOTE: at this point a 10s pause happens>
>
> libxl: error: libxl_device.c:1127:libxl__wait_for_backend: Backend /local/domain/0/backend/pci/3/0 not ready
> libxl: debug: libxl_pci.c:173:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/3/0 is not ready
> [repeat for more devices]
>
> Do you get anything in "xl dmesg" or dom0's "dmesg" corresponding to
> these events?
xl dmesg:
There is no additional output in dom0's 'xl dmesg' after creation of the
domain with xl create -c
There are however two additional lines in dom0's 'xl dmesg' output after
I have executed 'shutdown -h now' from the domU's command line following
a login as follows:
==============
(XEN) irq.c:2028: dom3: forcing unbind of pirq 23
(XEN) tmem: flushing tmem pools for domid=3
==============
dmesg:
Additional output of dmesg in dom0 after creating the the domain with
xl create -c is as follows:
==============
[ 322.225345] device vif3.0 entered promiscuous mode
[ 322.828304] xen_pciback: vpci: 0000:03:00.0: assign to virtual slot 0
[ 322.829486] xen_pciback: vpci: 0000:06:00.0: assign to virtual slot 1
[ 322.840174] xen_pciback: vpci: 0000:09:02.0: assign to virtual slot 2
[ 322.841134] xen_pciback: vpci: 0000:00:1d.0: assign to virtual slot 3
[ 322.937946] xen-blkback:ring-ref 2047, event-channel 4, protocol 1
(x86_64-abi)
[ 322.947106] xen-blkback:ring-ref 2046, event-channel 5, protocol 1
(x86_64-abi)
[ 322.955840] xen-blkback:ring-ref 2045, event-channel 6, protocol 1
(x86_64-abi)
[ 327.171287] xen-blkback:backend/vbd/3/51713: prepare for reconnect
[ 327.194441] xen-blkback:backend/vbd/3/51714: prepare for reconnect
[ 327.198687] xen-blkback:backend/vbd/3/51715: prepare for reconnect
[ 327.925926] pciback 0000:00:1d.0: enabling device (0000 -> 0002)
[ 327.926053] xen: registering gsi 23 triggering 0 polarity 1
[ 327.926059] Already setup the GSI :23
[ 327.926219] pciback 0000:00:1d.0: Driver tried to write to a
read-only configuration space field at offset 0x6c, size 4. This may
be harmless, but if you have problems with your device:
1) see permissive attribute in sysfs
2) report problems to the xen-devel mailing list along with details of
your device obtained from lspci.
[ 327.932217] xen-blkback:ring-ref 9, event-channel 18, protocol 1
(x86_64-abi) persistent grants
[ 327.938142] xen-blkback:ring-ref 10, event-channel 19, protocol 1
(x86_64-abi) persistent grants
[ 327.943674] xen-blkback:ring-ref 11, event-channel 20, protocol 1
(x86_64-abi) persistent grants
[ 328.965061] xenbr0: port 4(vif3.0) entered forwarding state
[ 328.965067] xenbr0: port 4(vif3.0) entered forwarding state
[ 330.716771] pciback 0000:00:1d.0: setting latency timer to 64
[ 335.728218] pciback 0000:06:00.0: enabling device (0000 -> 0002)
[ 335.728286] xen: registering gsi 16 triggering 0 polarity 1
[ 335.728290] Already setup the GSI :16
[ 343.992293] xenbr0: port 4(vif3.0) entered forwarding state
==============
FYI: The 0000:00:1d.0 device is the USB host controller being passed
through (Intel Sandybridge, C206 chipset); output of lspci:
00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset
Family USB Enhanced Host Controller #1 (rev 05)
Howvere currently there seems to be no issue with the USB host
controller in domU: There are two USB bluetooth dongles attached and
they do work (they connect to mobile phones). I'm more than happy to
provide more info should this be required.
Further output of dmesg in dom0 after I have executed 'shutdown -h now'
from the domU's command line following a login is follows:
==============
[ 556.864398] xenbr0: port 4(vif3.0) entered disabled state
[ 597.584588] xenbr0: port 4(vif3.0) entered disabled state
[ 597.584663] device vif3.0 left promiscuous mode
[ 597.584668] xenbr0: port 4(vif3.0) entered disabled state
==============
So I guess probably not too much of value ...
>
> Looking at do_pci_remove after the call to xc_domain_irq_permission
> (which fails, but I don't think that relates to the delay) we then call
> (conditionally) libxl__device_pci_reset, xc_deassign_device,
> libxl__device_pci_remove_common and libxl__device_pci_remove_xenstore,
> with no logging to indicate which we are calling (not helpful!).
>
> The "is not ready" message comes from libxl__device_pci_remove_xenstore
> which calls libxl__wait_for_backend. The latter has been rewritten a bit
> since 4.3.1 but not in a way which I think would affect this case.
> libxl__wait_for_backend does have a usleep(10000) in it -- which is
> certainly the source of the delay, but I'd like to explain how we got to
> waiting like that anyway (IanJ: do you have PCI on your hitlist for
> asyncing up?)
>
> This thing about pciback not being ready rings a bell. I've cc'd a few
> folks who I think might remember more.
>
> While the domain is happily running can you provide the output of
> "xenstore-ls -fp" -- I'm curious what state pciback is in. It should be
> 4, if not then that would be the problem.
Full output of xenstore-ls -fp (from dom0) is attached as it is 627
lines long and I am not quiet sure what you are actually after): There's
nothing in it that reads pciback; there are however a few entries named
/local/domain/0/backend/pci
and for one of the subentries named 3/0/state (3 is the domain-id)
the value seems to be 4:
/local/domain/0/backend/pci/3/0/state = "4" (n0,r3)
But then there is also another entry further down named
/local/domain/3/device/pci/0/state = "4" (n3,r0)
But I am speculating here and I guess it's better to leave the
interpretation to your expertise ...
>
>> BTW: I don't know whether it makes any difference, but I am only using
>> xen-pciback.hide=(bb:dd.f)(...) on the grub command line for a number of
>> devices including those that I pass through to this domU - there's
>> nothing else happening in the dom0 with those devices priot to starting
>> the domU and there are also no driver modules available for any of the
>> hidden hardware (except for one of the hidden USB Controllers of the
>> motherboard which is also passed through) in dom0.
>
> I don't think that should matter here.
>
> Ian.
>
[-- Attachment #2: xenstore --]
[-- Type: text/plain, Size: 38891 bytes --]
/tool = "" (n0)
/tool/xenstored = "" (n0)
/vm = "" (n0)
/vm/f8ed69cc-3f5e-4245-995e-0839553892e9 = "" (n0,r1)
/vm/f8ed69cc-3f5e-4245-995e-0839553892e9/uuid = "f8ed69cc-3f5e-4245-995e-0839553892e9" (n0,r1)
/vm/f8ed69cc-3f5e-4245-995e-0839553892e9/name = "ldap" (n0,r1)
/vm/f8ed69cc-3f5e-4245-995e-0839553892e9/image = "" (n0,r1)
/vm/f8ed69cc-3f5e-4245-995e-0839553892e9/image/ostype = "linux" (n0,r1)
/vm/f8ed69cc-3f5e-4245-995e-0839553892e9/image/kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz" (n0,r1)
/vm/f8ed69cc-3f5e-4245-995e-0839553892e9/image/ramdisk = "/etc/xen/guests/grub.d/ldap.grub" (n0,r1)
/vm/f8ed69cc-3f5e-4245-995e-0839553892e9/image/cmdline = "" (n0,r1)
/vm/f8ed69cc-3f5e-4245-995e-0839553892e9/start_time = "1393262439.59" (n0,r1)
/vm/c72b254e-5887-4636-aa17-8c08dc2a756b = "" (n0,r6)
/vm/c72b254e-5887-4636-aa17-8c08dc2a756b/uuid = "c72b254e-5887-4636-aa17-8c08dc2a756b" (n0,r6)
/vm/c72b254e-5887-4636-aa17-8c08dc2a756b/name = "pki" (n0,r6)
/vm/c72b254e-5887-4636-aa17-8c08dc2a756b/image = "" (n0,r6)
/vm/c72b254e-5887-4636-aa17-8c08dc2a756b/image/ostype = "linux" (n0,r6)
/vm/c72b254e-5887-4636-aa17-8c08dc2a756b/image/kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz" (n0,r6)
/vm/c72b254e-5887-4636-aa17-8c08dc2a756b/image/ramdisk = "/etc/xen/guests/grub.d/pki.grub" (n0,r6)
/vm/c72b254e-5887-4636-aa17-8c08dc2a756b/image/cmdline = "" (n0,r6)
/vm/c72b254e-5887-4636-aa17-8c08dc2a756b/start_time = "1394963622.94" (n0,r6)
/vm/ffe4ca76-e888-4a07-af5d-015d72497fd6 = "" (n0,r1)
/vm/ffe4ca76-e888-4a07-af5d-015d72497fd6/uuid = "ffe4ca76-e888-4a07-af5d-015d72497fd6" (n0,r1)
/vm/ffe4ca76-e888-4a07-af5d-015d72497fd6/name = "ldap" (n0,r1)
/vm/ffe4ca76-e888-4a07-af5d-015d72497fd6/image = "" (n0,r1)
/vm/ffe4ca76-e888-4a07-af5d-015d72497fd6/image/ostype = "linux" (n0,r1)
/vm/ffe4ca76-e888-4a07-af5d-015d72497fd6/image/kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz" (n0,r1)
/vm/ffe4ca76-e888-4a07-af5d-015d72497fd6/image/ramdisk = "/etc/xen/guests/grub.d/ldap.grub" (n0,r1)
/vm/ffe4ca76-e888-4a07-af5d-015d72497fd6/image/cmdline = "" (n0,r1)
/vm/ffe4ca76-e888-4a07-af5d-015d72497fd6/start_time = "1395146456.96" (n0,r1)
/vm/3d564878-91c5-4384-a4ae-343508f41d60 = "" (n0,r2)
/vm/3d564878-91c5-4384-a4ae-343508f41d60/uuid = "3d564878-91c5-4384-a4ae-343508f41d60" (n0,r2)
/vm/3d564878-91c5-4384-a4ae-343508f41d60/name = "www" (n0,r2)
/vm/3d564878-91c5-4384-a4ae-343508f41d60/image = "" (n0,r2)
/vm/3d564878-91c5-4384-a4ae-343508f41d60/image/ostype = "linux" (n0,r2)
/vm/3d564878-91c5-4384-a4ae-343508f41d60/image/kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz" (n0,r2)
/vm/3d564878-91c5-4384-a4ae-343508f41d60/image/ramdisk = "/etc/xen/guests/grub.d/www.grub" (n0,r2)
/vm/3d564878-91c5-4384-a4ae-343508f41d60/image/cmdline = "" (n0,r2)
/vm/3d564878-91c5-4384-a4ae-343508f41d60/start_time = "1395146458.12" (n0,r2)
/vm/8d28f3d9-1d51-41d9-853b-f67f1a750567 = "" (n0,r3)
/vm/8d28f3d9-1d51-41d9-853b-f67f1a750567/uuid = "8d28f3d9-1d51-41d9-853b-f67f1a750567" (n0,r3)
/vm/8d28f3d9-1d51-41d9-853b-f67f1a750567/name = "voip" (n0,r3)
/vm/8d28f3d9-1d51-41d9-853b-f67f1a750567/image = "" (n0,r3)
/vm/8d28f3d9-1d51-41d9-853b-f67f1a750567/image/ostype = "linux" (n0,r3)
/vm/8d28f3d9-1d51-41d9-853b-f67f1a750567/image/kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz" (n0,r3)
/vm/8d28f3d9-1d51-41d9-853b-f67f1a750567/image/ramdisk = "/etc/xen/guests/grub.d/voip.grub" (n0,r3)
/vm/8d28f3d9-1d51-41d9-853b-f67f1a750567/image/cmdline = "" (n0,r3)
/vm/8d28f3d9-1d51-41d9-853b-f67f1a750567/start_time = "1395146707.98" (n0,r3)
/libxl = "" (n0)
/libxl/1 = "" (n0)
/libxl/1/dm-version = "qemu_xen" (n0)
/libxl/2 = "" (n0)
/libxl/2/dm-version = "qemu_xen" (n0)
/libxl/3 = "" (n0)
/libxl/3/dm-version = "qemu_xen" (n0)
/local = "" (n0)
/local/domain = "" (n0)
/local/domain/0 = "" (n0)
/local/domain/0/name = "Domain-0" (n0)
/local/domain/0/libxl = "" (n0)
/local/domain/0/libxl/disable_udev = "1" (n0)
/local/domain/0/backend = "" (n0)
/local/domain/0/backend/vbd = "" (n0)
/local/domain/0/backend/vbd/1 = "" (n0)
/local/domain/0/backend/vbd/1/51713 = "" (n0,r1)
/local/domain/0/backend/vbd/1/51713/frontend = "/local/domain/1/device/vbd/51713" (n0,r1)
/local/domain/0/backend/vbd/1/51713/params = "/etc/xen/guests/root.d/live.root" (n0,r1)
/local/domain/0/backend/vbd/1/51713/script = "/etc/xen/scripts/block" (n0,r1)
/local/domain/0/backend/vbd/1/51713/physical-device = "fd:2" (n0,r1)
/local/domain/0/backend/vbd/1/51713/frontend-id = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51713/online = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51713/removable = "0" (n0,r1)
/local/domain/0/backend/vbd/1/51713/bootable = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51713/state = "4" (n0,r1)
/local/domain/0/backend/vbd/1/51713/dev = "xvda1" (n0,r1)
/local/domain/0/backend/vbd/1/51713/type = "phy" (n0,r1)
/local/domain/0/backend/vbd/1/51713/mode = "r" (n0,r1)
/local/domain/0/backend/vbd/1/51713/device-type = "disk" (n0,r1)
/local/domain/0/backend/vbd/1/51713/feature-flush-cache = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51713/feature-discard = "0" (n0,r1)
/local/domain/0/backend/vbd/1/51713/feature-barrier = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51713/feature-persistent = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51713/feature-max-indirect-segments = "256" (n0,r1)
/local/domain/0/backend/vbd/1/51713/sectors = "16777216" (n0,r1)
/local/domain/0/backend/vbd/1/51713/info = "4" (n0,r1)
/local/domain/0/backend/vbd/1/51713/sector-size = "512" (n0,r1)
/local/domain/0/backend/vbd/1/51713/physical-sector-size = "512" (n0,r1)
/local/domain/0/backend/vbd/1/51714 = "" (n0,r1)
/local/domain/0/backend/vbd/1/51714/frontend = "/local/domain/1/device/vbd/51714" (n0,r1)
/local/domain/0/backend/vbd/1/51714/params = "/etc/xen/guests/swap.d/ldap.swap" (n0,r1)
/local/domain/0/backend/vbd/1/51714/script = "/etc/xen/scripts/block" (n0,r1)
/local/domain/0/backend/vbd/1/51714/physical-device = "fd:4" (n0,r1)
/local/domain/0/backend/vbd/1/51714/frontend-id = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51714/online = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51714/removable = "0" (n0,r1)
/local/domain/0/backend/vbd/1/51714/bootable = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51714/state = "4" (n0,r1)
/local/domain/0/backend/vbd/1/51714/dev = "xvda2" (n0,r1)
/local/domain/0/backend/vbd/1/51714/type = "phy" (n0,r1)
/local/domain/0/backend/vbd/1/51714/mode = "w" (n0,r1)
/local/domain/0/backend/vbd/1/51714/device-type = "disk" (n0,r1)
/local/domain/0/backend/vbd/1/51714/feature-flush-cache = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51714/feature-discard = "0" (n0,r1)
/local/domain/0/backend/vbd/1/51714/feature-barrier = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51714/feature-persistent = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51714/feature-max-indirect-segments = "256" (n0,r1)
/local/domain/0/backend/vbd/1/51714/sectors = "2097152" (n0,r1)
/local/domain/0/backend/vbd/1/51714/info = "0" (n0,r1)
/local/domain/0/backend/vbd/1/51714/sector-size = "512" (n0,r1)
/local/domain/0/backend/vbd/1/51714/physical-sector-size = "512" (n0,r1)
/local/domain/0/backend/vbd/1/51715 = "" (n0,r1)
/local/domain/0/backend/vbd/1/51715/frontend = "/local/domain/1/device/vbd/51715" (n0,r1)
/local/domain/0/backend/vbd/1/51715/params = "/etc/xen/guests/overlay.d/ldap.ovly" (n0,r1)
/local/domain/0/backend/vbd/1/51715/script = "/etc/xen/scripts/block" (n0,r1)
/local/domain/0/backend/vbd/1/51715/physical-device = "fd:a" (n0,r1)
/local/domain/0/backend/vbd/1/51715/frontend-id = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51715/online = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51715/removable = "0" (n0,r1)
/local/domain/0/backend/vbd/1/51715/bootable = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51715/state = "4" (n0,r1)
/local/domain/0/backend/vbd/1/51715/dev = "xvda3" (n0,r1)
/local/domain/0/backend/vbd/1/51715/type = "phy" (n0,r1)
/local/domain/0/backend/vbd/1/51715/mode = "w" (n0,r1)
/local/domain/0/backend/vbd/1/51715/device-type = "disk" (n0,r1)
/local/domain/0/backend/vbd/1/51715/feature-flush-cache = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51715/feature-discard = "0" (n0,r1)
/local/domain/0/backend/vbd/1/51715/feature-barrier = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51715/feature-persistent = "1" (n0,r1)
/local/domain/0/backend/vbd/1/51715/feature-max-indirect-segments = "256" (n0,r1)
/local/domain/0/backend/vbd/1/51715/sectors = "2097152" (n0,r1)
/local/domain/0/backend/vbd/1/51715/info = "0" (n0,r1)
/local/domain/0/backend/vbd/1/51715/sector-size = "512" (n0,r1)
/local/domain/0/backend/vbd/1/51715/physical-sector-size = "512" (n0,r1)
/local/domain/0/backend/vbd/2 = "" (n0)
/local/domain/0/backend/vbd/2/51713 = "" (n0,r2)
/local/domain/0/backend/vbd/2/51713/frontend = "/local/domain/2/device/vbd/51713" (n0,r2)
/local/domain/0/backend/vbd/2/51713/params = "/etc/xen/guests/root.d/live.root" (n0,r2)
/local/domain/0/backend/vbd/2/51713/script = "/etc/xen/scripts/block" (n0,r2)
/local/domain/0/backend/vbd/2/51713/physical-device = "fd:2" (n0,r2)
/local/domain/0/backend/vbd/2/51713/frontend-id = "2" (n0,r2)
/local/domain/0/backend/vbd/2/51713/online = "1" (n0,r2)
/local/domain/0/backend/vbd/2/51713/removable = "0" (n0,r2)
/local/domain/0/backend/vbd/2/51713/bootable = "1" (n0,r2)
/local/domain/0/backend/vbd/2/51713/state = "4" (n0,r2)
/local/domain/0/backend/vbd/2/51713/dev = "xvda1" (n0,r2)
/local/domain/0/backend/vbd/2/51713/type = "phy" (n0,r2)
/local/domain/0/backend/vbd/2/51713/mode = "r" (n0,r2)
/local/domain/0/backend/vbd/2/51713/device-type = "disk" (n0,r2)
/local/domain/0/backend/vbd/2/51713/feature-flush-cache = "1" (n0,r2)
/local/domain/0/backend/vbd/2/51713/feature-discard = "0" (n0,r2)
/local/domain/0/backend/vbd/2/51713/feature-barrier = "1" (n0,r2)
/local/domain/0/backend/vbd/2/51713/feature-persistent = "1" (n0,r2)
/local/domain/0/backend/vbd/2/51713/feature-max-indirect-segments = "256" (n0,r2)
/local/domain/0/backend/vbd/2/51713/sectors = "16777216" (n0,r2)
/local/domain/0/backend/vbd/2/51713/info = "4" (n0,r2)
/local/domain/0/backend/vbd/2/51713/sector-size = "512" (n0,r2)
/local/domain/0/backend/vbd/2/51713/physical-sector-size = "512" (n0,r2)
/local/domain/0/backend/vbd/2/51714 = "" (n0,r2)
/local/domain/0/backend/vbd/2/51714/frontend = "/local/domain/2/device/vbd/51714" (n0,r2)
/local/domain/0/backend/vbd/2/51714/params = "/etc/xen/guests/swap.d/www.swap" (n0,r2)
/local/domain/0/backend/vbd/2/51714/script = "/etc/xen/scripts/block" (n0,r2)
/local/domain/0/backend/vbd/2/51714/physical-device = "fd:e" (n0,r2)
/local/domain/0/backend/vbd/2/51714/frontend-id = "2" (n0,r2)
/local/domain/0/backend/vbd/2/51714/online = "1" (n0,r2)
/local/domain/0/backend/vbd/2/51714/removable = "0" (n0,r2)
/local/domain/0/backend/vbd/2/51714/bootable = "1" (n0,r2)
/local/domain/0/backend/vbd/2/51714/state = "4" (n0,r2)
/local/domain/0/backend/vbd/2/51714/dev = "xvda2" (n0,r2)
/local/domain/0/backend/vbd/2/51714/type = "phy" (n0,r2)
/local/domain/0/backend/vbd/2/51714/mode = "w" (n0,r2)
/local/domain/0/backend/vbd/2/51714/device-type = "disk" (n0,r2)
/local/domain/0/backend/vbd/2/51714/feature-flush-cache = "1" (n0,r2)
/local/domain/0/backend/vbd/2/51714/feature-discard = "0" (n0,r2)
/local/domain/0/backend/vbd/2/51714/feature-barrier = "1" (n0,r2)
/local/domain/0/backend/vbd/2/51714/feature-persistent = "1" (n0,r2)
/local/domain/0/backend/vbd/2/51714/feature-max-indirect-segments = "256" (n0,r2)
/local/domain/0/backend/vbd/2/51714/sectors = "2097152" (n0,r2)
/local/domain/0/backend/vbd/2/51714/info = "0" (n0,r2)
/local/domain/0/backend/vbd/2/51714/sector-size = "512" (n0,r2)
/local/domain/0/backend/vbd/2/51714/physical-sector-size = "512" (n0,r2)
/local/domain/0/backend/vbd/2/51715 = "" (n0,r2)
/local/domain/0/backend/vbd/2/51715/frontend = "/local/domain/2/device/vbd/51715" (n0,r2)
/local/domain/0/backend/vbd/2/51715/params = "/etc/xen/guests/overlay.d/www.ovly" (n0,r2)
/local/domain/0/backend/vbd/2/51715/script = "/etc/xen/scripts/block" (n0,r2)
/local/domain/0/backend/vbd/2/51715/physical-device = "fd:d" (n0,r2)
/local/domain/0/backend/vbd/2/51715/frontend-id = "2" (n0,r2)
/local/domain/0/backend/vbd/2/51715/online = "1" (n0,r2)
/local/domain/0/backend/vbd/2/51715/removable = "0" (n0,r2)
/local/domain/0/backend/vbd/2/51715/bootable = "1" (n0,r2)
/local/domain/0/backend/vbd/2/51715/state = "4" (n0,r2)
/local/domain/0/backend/vbd/2/51715/dev = "xvda3" (n0,r2)
/local/domain/0/backend/vbd/2/51715/type = "phy" (n0,r2)
/local/domain/0/backend/vbd/2/51715/mode = "w" (n0,r2)
/local/domain/0/backend/vbd/2/51715/device-type = "disk" (n0,r2)
/local/domain/0/backend/vbd/2/51715/feature-flush-cache = "1" (n0,r2)
/local/domain/0/backend/vbd/2/51715/feature-discard = "0" (n0,r2)
/local/domain/0/backend/vbd/2/51715/feature-barrier = "1" (n0,r2)
/local/domain/0/backend/vbd/2/51715/feature-persistent = "1" (n0,r2)
/local/domain/0/backend/vbd/2/51715/feature-max-indirect-segments = "256" (n0,r2)
/local/domain/0/backend/vbd/2/51715/sectors = "2097152" (n0,r2)
/local/domain/0/backend/vbd/2/51715/info = "0" (n0,r2)
/local/domain/0/backend/vbd/2/51715/sector-size = "512" (n0,r2)
/local/domain/0/backend/vbd/2/51715/physical-sector-size = "512" (n0,r2)
/local/domain/0/backend/vbd/3 = "" (n0)
/local/domain/0/backend/vbd/3/51713 = "" (n0,r3)
/local/domain/0/backend/vbd/3/51713/frontend = "/local/domain/3/device/vbd/51713" (n0,r3)
/local/domain/0/backend/vbd/3/51713/params = "/etc/xen/guests/root.d/live.root" (n0,r3)
/local/domain/0/backend/vbd/3/51713/script = "/etc/xen/scripts/block" (n0,r3)
/local/domain/0/backend/vbd/3/51713/physical-device = "fd:2" (n0,r3)
/local/domain/0/backend/vbd/3/51713/frontend-id = "3" (n0,r3)
/local/domain/0/backend/vbd/3/51713/online = "1" (n0,r3)
/local/domain/0/backend/vbd/3/51713/removable = "0" (n0,r3)
/local/domain/0/backend/vbd/3/51713/bootable = "1" (n0,r3)
/local/domain/0/backend/vbd/3/51713/state = "4" (n0,r3)
/local/domain/0/backend/vbd/3/51713/dev = "xvda1" (n0,r3)
/local/domain/0/backend/vbd/3/51713/type = "phy" (n0,r3)
/local/domain/0/backend/vbd/3/51713/mode = "r" (n0,r3)
/local/domain/0/backend/vbd/3/51713/device-type = "disk" (n0,r3)
/local/domain/0/backend/vbd/3/51713/feature-flush-cache = "1" (n0,r3)
/local/domain/0/backend/vbd/3/51713/feature-discard = "0" (n0,r3)
/local/domain/0/backend/vbd/3/51713/feature-barrier = "1" (n0,r3)
/local/domain/0/backend/vbd/3/51713/feature-persistent = "1" (n0,r3)
/local/domain/0/backend/vbd/3/51713/feature-max-indirect-segments = "256" (n0,r3)
/local/domain/0/backend/vbd/3/51713/sectors = "16777216" (n0,r3)
/local/domain/0/backend/vbd/3/51713/info = "4" (n0,r3)
/local/domain/0/backend/vbd/3/51713/sector-size = "512" (n0,r3)
/local/domain/0/backend/vbd/3/51713/physical-sector-size = "512" (n0,r3)
/local/domain/0/backend/vbd/3/51714 = "" (n0,r3)
/local/domain/0/backend/vbd/3/51714/frontend = "/local/domain/3/device/vbd/51714" (n0,r3)
/local/domain/0/backend/vbd/3/51714/params = "/etc/xen/guests/swap.d/voip.swap" (n0,r3)
/local/domain/0/backend/vbd/3/51714/script = "/etc/xen/scripts/block" (n0,r3)
/local/domain/0/backend/vbd/3/51714/physical-device = "fd:11" (n0,r3)
/local/domain/0/backend/vbd/3/51714/frontend-id = "3" (n0,r3)
/local/domain/0/backend/vbd/3/51714/online = "1" (n0,r3)
/local/domain/0/backend/vbd/3/51714/removable = "0" (n0,r3)
/local/domain/0/backend/vbd/3/51714/bootable = "1" (n0,r3)
/local/domain/0/backend/vbd/3/51714/state = "4" (n0,r3)
/local/domain/0/backend/vbd/3/51714/dev = "xvda2" (n0,r3)
/local/domain/0/backend/vbd/3/51714/type = "phy" (n0,r3)
/local/domain/0/backend/vbd/3/51714/mode = "w" (n0,r3)
/local/domain/0/backend/vbd/3/51714/device-type = "disk" (n0,r3)
/local/domain/0/backend/vbd/3/51714/feature-flush-cache = "1" (n0,r3)
/local/domain/0/backend/vbd/3/51714/feature-discard = "0" (n0,r3)
/local/domain/0/backend/vbd/3/51714/feature-barrier = "1" (n0,r3)
/local/domain/0/backend/vbd/3/51714/feature-persistent = "1" (n0,r3)
/local/domain/0/backend/vbd/3/51714/feature-max-indirect-segments = "256" (n0,r3)
/local/domain/0/backend/vbd/3/51714/sectors = "2097152" (n0,r3)
/local/domain/0/backend/vbd/3/51714/info = "0" (n0,r3)
/local/domain/0/backend/vbd/3/51714/sector-size = "512" (n0,r3)
/local/domain/0/backend/vbd/3/51714/physical-sector-size = "512" (n0,r3)
/local/domain/0/backend/vbd/3/51715 = "" (n0,r3)
/local/domain/0/backend/vbd/3/51715/frontend = "/local/domain/3/device/vbd/51715" (n0,r3)
/local/domain/0/backend/vbd/3/51715/params = "/etc/xen/guests/overlay.d/voip.ovly" (n0,r3)
/local/domain/0/backend/vbd/3/51715/script = "/etc/xen/scripts/block" (n0,r3)
/local/domain/0/backend/vbd/3/51715/physical-device = "fd:12" (n0,r3)
/local/domain/0/backend/vbd/3/51715/frontend-id = "3" (n0,r3)
/local/domain/0/backend/vbd/3/51715/online = "1" (n0,r3)
/local/domain/0/backend/vbd/3/51715/removable = "0" (n0,r3)
/local/domain/0/backend/vbd/3/51715/bootable = "1" (n0,r3)
/local/domain/0/backend/vbd/3/51715/state = "4" (n0,r3)
/local/domain/0/backend/vbd/3/51715/dev = "xvda3" (n0,r3)
/local/domain/0/backend/vbd/3/51715/type = "phy" (n0,r3)
/local/domain/0/backend/vbd/3/51715/mode = "w" (n0,r3)
/local/domain/0/backend/vbd/3/51715/device-type = "disk" (n0,r3)
/local/domain/0/backend/vbd/3/51715/feature-flush-cache = "1" (n0,r3)
/local/domain/0/backend/vbd/3/51715/feature-discard = "0" (n0,r3)
/local/domain/0/backend/vbd/3/51715/feature-barrier = "1" (n0,r3)
/local/domain/0/backend/vbd/3/51715/feature-persistent = "1" (n0,r3)
/local/domain/0/backend/vbd/3/51715/feature-max-indirect-segments = "256" (n0,r3)
/local/domain/0/backend/vbd/3/51715/sectors = "2097152" (n0,r3)
/local/domain/0/backend/vbd/3/51715/info = "0" (n0,r3)
/local/domain/0/backend/vbd/3/51715/sector-size = "512" (n0,r3)
/local/domain/0/backend/vbd/3/51715/physical-sector-size = "512" (n0,r3)
/local/domain/0/backend/console = "" (n0)
/local/domain/0/backend/console/1 = "" (n0)
/local/domain/0/backend/console/1/0 = "" (n0,r1)
/local/domain/0/backend/console/1/0/frontend = "/local/domain/1/console" (n0,r1)
/local/domain/0/backend/console/1/0/frontend-id = "1" (n0,r1)
/local/domain/0/backend/console/1/0/online = "1" (n0,r1)
/local/domain/0/backend/console/1/0/state = "1" (n0,r1)
/local/domain/0/backend/console/1/0/domain = "ldap" (n0,r1)
/local/domain/0/backend/console/1/0/protocol = "vt100" (n0,r1)
/local/domain/0/backend/console/2 = "" (n0)
/local/domain/0/backend/console/2/0 = "" (n0,r2)
/local/domain/0/backend/console/2/0/frontend = "/local/domain/2/console" (n0,r2)
/local/domain/0/backend/console/2/0/frontend-id = "2" (n0,r2)
/local/domain/0/backend/console/2/0/online = "1" (n0,r2)
/local/domain/0/backend/console/2/0/state = "1" (n0,r2)
/local/domain/0/backend/console/2/0/domain = "www" (n0,r2)
/local/domain/0/backend/console/2/0/protocol = "vt100" (n0,r2)
/local/domain/0/backend/console/3 = "" (n0)
/local/domain/0/backend/console/3/0 = "" (n0,r3)
/local/domain/0/backend/console/3/0/frontend = "/local/domain/3/console" (n0,r3)
/local/domain/0/backend/console/3/0/frontend-id = "3" (n0,r3)
/local/domain/0/backend/console/3/0/online = "1" (n0,r3)
/local/domain/0/backend/console/3/0/state = "1" (n0,r3)
/local/domain/0/backend/console/3/0/domain = "voip" (n0,r3)
/local/domain/0/backend/console/3/0/protocol = "vt100" (n0,r3)
/local/domain/0/backend/vif = "" (n0)
/local/domain/0/backend/vif/1 = "" (n0)
/local/domain/0/backend/vif/1/0 = "" (n0,r1)
/local/domain/0/backend/vif/1/0/frontend = "/local/domain/1/device/vif/0" (n0,r1)
/local/domain/0/backend/vif/1/0/frontend-id = "1" (n0,r1)
/local/domain/0/backend/vif/1/0/online = "1" (n0,r1)
/local/domain/0/backend/vif/1/0/state = "4" (n0,r1)
/local/domain/0/backend/vif/1/0/script = "/etc/xen/scripts/vif-bridge" (n0,r1)
/local/domain/0/backend/vif/1/0/mac = "00:16:3e:a0:64:04" (n0,r1)
/local/domain/0/backend/vif/1/0/bridge = "xenbr0" (n0,r1)
/local/domain/0/backend/vif/1/0/handle = "0" (n0,r1)
/local/domain/0/backend/vif/1/0/type = "vif" (n0,r1)
/local/domain/0/backend/vif/1/0/feature-sg = "1" (n0,r1)
/local/domain/0/backend/vif/1/0/feature-gso-tcpv4 = "1" (n0,r1)
/local/domain/0/backend/vif/1/0/feature-rx-copy = "1" (n0,r1)
/local/domain/0/backend/vif/1/0/feature-rx-flip = "0" (n0,r1)
/local/domain/0/backend/vif/1/0/feature-split-event-channels = "1" (n0,r1)
/local/domain/0/backend/vif/1/0/hotplug-status = "connected" (n0,r1)
/local/domain/0/backend/vif/2 = "" (n0)
/local/domain/0/backend/vif/2/0 = "" (n0,r2)
/local/domain/0/backend/vif/2/0/frontend = "/local/domain/2/device/vif/0" (n0,r2)
/local/domain/0/backend/vif/2/0/frontend-id = "2" (n0,r2)
/local/domain/0/backend/vif/2/0/online = "1" (n0,r2)
/local/domain/0/backend/vif/2/0/state = "4" (n0,r2)
/local/domain/0/backend/vif/2/0/script = "/etc/xen/scripts/vif-bridge" (n0,r2)
/local/domain/0/backend/vif/2/0/mac = "00:16:3e:a0:64:07" (n0,r2)
/local/domain/0/backend/vif/2/0/bridge = "xenbr0" (n0,r2)
/local/domain/0/backend/vif/2/0/handle = "0" (n0,r2)
/local/domain/0/backend/vif/2/0/type = "vif" (n0,r2)
/local/domain/0/backend/vif/2/0/feature-sg = "1" (n0,r2)
/local/domain/0/backend/vif/2/0/feature-gso-tcpv4 = "1" (n0,r2)
/local/domain/0/backend/vif/2/0/feature-rx-copy = "1" (n0,r2)
/local/domain/0/backend/vif/2/0/feature-rx-flip = "0" (n0,r2)
/local/domain/0/backend/vif/2/0/feature-split-event-channels = "1" (n0,r2)
/local/domain/0/backend/vif/2/0/hotplug-status = "connected" (n0,r2)
/local/domain/0/backend/vif/3 = "" (n0)
/local/domain/0/backend/vif/3/0 = "" (n0,r3)
/local/domain/0/backend/vif/3/0/frontend = "/local/domain/3/device/vif/0" (n0,r3)
/local/domain/0/backend/vif/3/0/frontend-id = "3" (n0,r3)
/local/domain/0/backend/vif/3/0/online = "1" (n0,r3)
/local/domain/0/backend/vif/3/0/state = "4" (n0,r3)
/local/domain/0/backend/vif/3/0/script = "/etc/xen/scripts/vif-bridge" (n0,r3)
/local/domain/0/backend/vif/3/0/mac = "00:16:3e:a0:64:09" (n0,r3)
/local/domain/0/backend/vif/3/0/bridge = "xenbr0" (n0,r3)
/local/domain/0/backend/vif/3/0/handle = "0" (n0,r3)
/local/domain/0/backend/vif/3/0/type = "vif" (n0,r3)
/local/domain/0/backend/vif/3/0/feature-sg = "1" (n0,r3)
/local/domain/0/backend/vif/3/0/feature-gso-tcpv4 = "1" (n0,r3)
/local/domain/0/backend/vif/3/0/feature-rx-copy = "1" (n0,r3)
/local/domain/0/backend/vif/3/0/feature-rx-flip = "0" (n0,r3)
/local/domain/0/backend/vif/3/0/feature-split-event-channels = "1" (n0,r3)
/local/domain/0/backend/vif/3/0/hotplug-status = "connected" (n0,r3)
/local/domain/0/backend/pci = "" (n0)
/local/domain/0/backend/pci/3 = "" (n0)
/local/domain/0/backend/pci/3/0 = "" (n0,r3)
/local/domain/0/backend/pci/3/0/frontend = "/local/domain/3/device/pci/0" (n0,r3)
/local/domain/0/backend/pci/3/0/frontend-id = "3" (n0,r3)
/local/domain/0/backend/pci/3/0/online = "1" (n0,r3)
/local/domain/0/backend/pci/3/0/state = "4" (n0,r3)
/local/domain/0/backend/pci/3/0/domain = "voip" (n0,r3)
/local/domain/0/backend/pci/3/0/key-0 = "0000:03:00.0" (n0,r3)
/local/domain/0/backend/pci/3/0/dev-0 = "0000:03:00.0" (n0,r3)
/local/domain/0/backend/pci/3/0/opts-0 = "msitranslate=0,power_mgmt=0,permissive=0" (n0,r3)
/local/domain/0/backend/pci/3/0/state-0 = "3" (n0,r3)
/local/domain/0/backend/pci/3/0/key-1 = "0000:06:00.0" (n0,r3)
/local/domain/0/backend/pci/3/0/dev-1 = "0000:06:00.0" (n0,r3)
/local/domain/0/backend/pci/3/0/opts-1 = "msitranslate=0,power_mgmt=0,permissive=0" (n0,r3)
/local/domain/0/backend/pci/3/0/state-1 = "3" (n0,r3)
/local/domain/0/backend/pci/3/0/key-2 = "0000:09:02.0" (n0,r3)
/local/domain/0/backend/pci/3/0/dev-2 = "0000:09:02.0" (n0,r3)
/local/domain/0/backend/pci/3/0/opts-2 = "msitranslate=0,power_mgmt=0,permissive=0" (n0,r3)
/local/domain/0/backend/pci/3/0/state-2 = "3" (n0,r3)
/local/domain/0/backend/pci/3/0/key-3 = "0000:00:1d.0" (n0,r3)
/local/domain/0/backend/pci/3/0/dev-3 = "0000:00:1d.0" (n0,r3)
/local/domain/0/backend/pci/3/0/opts-3 = "msitranslate=0,power_mgmt=0,permissive=0" (n0,r3)
/local/domain/0/backend/pci/3/0/state-3 = "3" (n0,r3)
/local/domain/0/backend/pci/3/0/num_devs = "4" (n0,r3)
/local/domain/0/backend/pci/3/0/vdev-0 = "0000:00:00.00" (n0,r3)
/local/domain/0/backend/pci/3/0/vdev-1 = "0000:00:01.00" (n0,r3)
/local/domain/0/backend/pci/3/0/vdev-2 = "0000:00:02.00" (n0,r3)
/local/domain/0/backend/pci/3/0/vdev-3 = "0000:00:03.00" (n0,r3)
/local/domain/0/backend/pci/3/0/root-0 = "0000:00" (n0,r3)
/local/domain/0/backend/pci/3/0/root_num = "1" (n0,r3)
/local/domain/0/device-model = "" (n0)
/local/domain/0/device-model/0 = "" (n0)
/local/domain/0/device-model/0/state = "running" (n0)
/local/domain/1 = "" (n0,r1)
/local/domain/1/vm = "/vm/ffe4ca76-e888-4a07-af5d-015d72497fd6" (n0,r1)
/local/domain/1/name = "ldap" (n0,r1)
/local/domain/1/cpu = "" (n0,r1)
/local/domain/1/cpu/0 = "" (n0,r1)
/local/domain/1/cpu/0/availability = "online" (n0,r1)
/local/domain/1/cpu/1 = "" (n0,r1)
/local/domain/1/cpu/1/availability = "online" (n0,r1)
/local/domain/1/memory = "" (n0,r1)
/local/domain/1/memory/static-max = "2097152" (n0,r1)
/local/domain/1/memory/target = "1048577" (n0,r1)
/local/domain/1/memory/videoram = "-1" (n0,r1)
/local/domain/1/device = "" (n0,r1)
/local/domain/1/device/suspend = "" (n0,r1)
/local/domain/1/device/suspend/event-channel = "" (n1)
/local/domain/1/device/vbd = "" (n0,r1)
/local/domain/1/device/vbd/51713 = "" (n1,r0)
/local/domain/1/device/vbd/51713/backend = "/local/domain/0/backend/vbd/1/51713" (n1,r0)
/local/domain/1/device/vbd/51713/backend-id = "0" (n1,r0)
/local/domain/1/device/vbd/51713/state = "4" (n1,r0)
/local/domain/1/device/vbd/51713/virtual-device = "51713" (n1,r0)
/local/domain/1/device/vbd/51713/device-type = "disk" (n1,r0)
/local/domain/1/device/vbd/51713/protocol = "x86_64-abi" (n1,r0)
/local/domain/1/device/vbd/51713/ring-ref = "8" (n1,r0)
/local/domain/1/device/vbd/51713/event-channel = "17" (n1,r0)
/local/domain/1/device/vbd/51713/feature-persistent = "1" (n1,r0)
/local/domain/1/device/vbd/51714 = "" (n1,r0)
/local/domain/1/device/vbd/51714/backend = "/local/domain/0/backend/vbd/1/51714" (n1,r0)
/local/domain/1/device/vbd/51714/backend-id = "0" (n1,r0)
/local/domain/1/device/vbd/51714/state = "4" (n1,r0)
/local/domain/1/device/vbd/51714/virtual-device = "51714" (n1,r0)
/local/domain/1/device/vbd/51714/device-type = "disk" (n1,r0)
/local/domain/1/device/vbd/51714/protocol = "x86_64-abi" (n1,r0)
/local/domain/1/device/vbd/51714/ring-ref = "9" (n1,r0)
/local/domain/1/device/vbd/51714/event-channel = "18" (n1,r0)
/local/domain/1/device/vbd/51714/feature-persistent = "1" (n1,r0)
/local/domain/1/device/vbd/51715 = "" (n1,r0)
/local/domain/1/device/vbd/51715/backend = "/local/domain/0/backend/vbd/1/51715" (n1,r0)
/local/domain/1/device/vbd/51715/backend-id = "0" (n1,r0)
/local/domain/1/device/vbd/51715/state = "4" (n1,r0)
/local/domain/1/device/vbd/51715/virtual-device = "51715" (n1,r0)
/local/domain/1/device/vbd/51715/device-type = "disk" (n1,r0)
/local/domain/1/device/vbd/51715/protocol = "x86_64-abi" (n1,r0)
/local/domain/1/device/vbd/51715/ring-ref = "10" (n1,r0)
/local/domain/1/device/vbd/51715/event-channel = "19" (n1,r0)
/local/domain/1/device/vbd/51715/feature-persistent = "1" (n1,r0)
/local/domain/1/device/vif = "" (n0,r1)
/local/domain/1/device/vif/0 = "" (n1,r0)
/local/domain/1/device/vif/0/backend = "/local/domain/0/backend/vif/1/0" (n1,r0)
/local/domain/1/device/vif/0/backend-id = "0" (n1,r0)
/local/domain/1/device/vif/0/state = "4" (n1,r0)
/local/domain/1/device/vif/0/handle = "0" (n1,r0)
/local/domain/1/device/vif/0/mac = "00:16:3e:a0:64:04" (n1,r0)
/local/domain/1/device/vif/0/tx-ring-ref = "11" (n1,r0)
/local/domain/1/device/vif/0/rx-ring-ref = "12" (n1,r0)
/local/domain/1/device/vif/0/event-channel-tx = "20" (n1,r0)
/local/domain/1/device/vif/0/event-channel-rx = "21" (n1,r0)
/local/domain/1/device/vif/0/request-rx-copy = "1" (n1,r0)
/local/domain/1/device/vif/0/feature-rx-notify = "1" (n1,r0)
/local/domain/1/device/vif/0/feature-sg = "1" (n1,r0)
/local/domain/1/device/vif/0/feature-gso-tcpv4 = "1" (n1,r0)
/local/domain/1/control = "" (n0,r1)
/local/domain/1/control/shutdown = "" (n1)
/local/domain/1/control/platform-feature-multiprocessor-suspend = "1" (n0,r1)
/local/domain/1/control/platform-feature-xs_reset_watches = "1" (n0,r1)
/local/domain/1/data = "" (n1)
/local/domain/1/domid = "1" (n0,r1)
/local/domain/1/store = "" (n0,r1)
/local/domain/1/store/port = "1" (n0,r1)
/local/domain/1/store/ring-ref = "7443678" (n0,r1)
/local/domain/1/console = "" (n0,r1)
/local/domain/1/console/backend = "/local/domain/0/backend/console/1/0" (n0,r1)
/local/domain/1/console/backend-id = "0" (n1,r0)
/local/domain/1/console/limit = "1048576" (n0,r1)
/local/domain/1/console/type = "xenconsoled" (n0,r1)
/local/domain/1/console/output = "pty" (n0,r1)
/local/domain/1/console/tty = "/dev/pts/4" (n0,r1)
/local/domain/1/console/port = "2" (n0,r1)
/local/domain/1/console/ring-ref = "7443677" (n0,r1)
/local/domain/2 = "" (n0,r2)
/local/domain/2/vm = "/vm/3d564878-91c5-4384-a4ae-343508f41d60" (n0,r2)
/local/domain/2/name = "www" (n0,r2)
/local/domain/2/cpu = "" (n0,r2)
/local/domain/2/cpu/0 = "" (n0,r2)
/local/domain/2/cpu/0/availability = "online" (n0,r2)
/local/domain/2/memory = "" (n0,r2)
/local/domain/2/memory/static-max = "524288" (n0,r2)
/local/domain/2/memory/target = "524289" (n0,r2)
/local/domain/2/memory/videoram = "-1" (n0,r2)
/local/domain/2/device = "" (n0,r2)
/local/domain/2/device/suspend = "" (n0,r2)
/local/domain/2/device/suspend/event-channel = "" (n2)
/local/domain/2/device/vbd = "" (n0,r2)
/local/domain/2/device/vbd/51713 = "" (n2,r0)
/local/domain/2/device/vbd/51713/backend = "/local/domain/0/backend/vbd/2/51713" (n2,r0)
/local/domain/2/device/vbd/51713/backend-id = "0" (n2,r0)
/local/domain/2/device/vbd/51713/state = "4" (n2,r0)
/local/domain/2/device/vbd/51713/virtual-device = "51713" (n2,r0)
/local/domain/2/device/vbd/51713/device-type = "disk" (n2,r0)
/local/domain/2/device/vbd/51713/protocol = "x86_64-abi" (n2,r0)
/local/domain/2/device/vbd/51713/ring-ref = "8" (n2,r0)
/local/domain/2/device/vbd/51713/event-channel = "10" (n2,r0)
/local/domain/2/device/vbd/51713/feature-persistent = "1" (n2,r0)
/local/domain/2/device/vbd/51714 = "" (n2,r0)
/local/domain/2/device/vbd/51714/backend = "/local/domain/0/backend/vbd/2/51714" (n2,r0)
/local/domain/2/device/vbd/51714/backend-id = "0" (n2,r0)
/local/domain/2/device/vbd/51714/state = "4" (n2,r0)
/local/domain/2/device/vbd/51714/virtual-device = "51714" (n2,r0)
/local/domain/2/device/vbd/51714/device-type = "disk" (n2,r0)
/local/domain/2/device/vbd/51714/protocol = "x86_64-abi" (n2,r0)
/local/domain/2/device/vbd/51714/ring-ref = "9" (n2,r0)
/local/domain/2/device/vbd/51714/event-channel = "11" (n2,r0)
/local/domain/2/device/vbd/51714/feature-persistent = "1" (n2,r0)
/local/domain/2/device/vbd/51715 = "" (n2,r0)
/local/domain/2/device/vbd/51715/backend = "/local/domain/0/backend/vbd/2/51715" (n2,r0)
/local/domain/2/device/vbd/51715/backend-id = "0" (n2,r0)
/local/domain/2/device/vbd/51715/state = "4" (n2,r0)
/local/domain/2/device/vbd/51715/virtual-device = "51715" (n2,r0)
/local/domain/2/device/vbd/51715/device-type = "disk" (n2,r0)
/local/domain/2/device/vbd/51715/protocol = "x86_64-abi" (n2,r0)
/local/domain/2/device/vbd/51715/ring-ref = "10" (n2,r0)
/local/domain/2/device/vbd/51715/event-channel = "12" (n2,r0)
/local/domain/2/device/vbd/51715/feature-persistent = "1" (n2,r0)
/local/domain/2/device/vif = "" (n0,r2)
/local/domain/2/device/vif/0 = "" (n2,r0)
/local/domain/2/device/vif/0/backend = "/local/domain/0/backend/vif/2/0" (n2,r0)
/local/domain/2/device/vif/0/backend-id = "0" (n2,r0)
/local/domain/2/device/vif/0/state = "4" (n2,r0)
/local/domain/2/device/vif/0/handle = "0" (n2,r0)
/local/domain/2/device/vif/0/mac = "00:16:3e:a0:64:07" (n2,r0)
/local/domain/2/device/vif/0/tx-ring-ref = "11" (n2,r0)
/local/domain/2/device/vif/0/rx-ring-ref = "12" (n2,r0)
/local/domain/2/device/vif/0/event-channel-tx = "13" (n2,r0)
/local/domain/2/device/vif/0/event-channel-rx = "14" (n2,r0)
/local/domain/2/device/vif/0/request-rx-copy = "1" (n2,r0)
/local/domain/2/device/vif/0/feature-rx-notify = "1" (n2,r0)
/local/domain/2/device/vif/0/feature-sg = "1" (n2,r0)
/local/domain/2/device/vif/0/feature-gso-tcpv4 = "1" (n2,r0)
/local/domain/2/control = "" (n0,r2)
/local/domain/2/control/shutdown = "" (n2)
/local/domain/2/control/platform-feature-multiprocessor-suspend = "1" (n0,r2)
/local/domain/2/control/platform-feature-xs_reset_watches = "1" (n0,r2)
/local/domain/2/data = "" (n2)
/local/domain/2/domid = "2" (n0,r2)
/local/domain/2/store = "" (n0,r2)
/local/domain/2/store/port = "1" (n0,r2)
/local/domain/2/store/ring-ref = "7631284" (n0,r2)
/local/domain/2/console = "" (n0,r2)
/local/domain/2/console/backend = "/local/domain/0/backend/console/2/0" (n0,r2)
/local/domain/2/console/backend-id = "0" (n2,r0)
/local/domain/2/console/limit = "1048576" (n0,r2)
/local/domain/2/console/type = "xenconsoled" (n0,r2)
/local/domain/2/console/output = "pty" (n0,r2)
/local/domain/2/console/tty = "/dev/pts/5" (n0,r2)
/local/domain/2/console/port = "2" (n0,r2)
/local/domain/2/console/ring-ref = "7631283" (n0,r2)
/local/domain/3 = "" (n0,r3)
/local/domain/3/vm = "/vm/8d28f3d9-1d51-41d9-853b-f67f1a750567" (n0,r3)
/local/domain/3/name = "voip" (n0,r3)
/local/domain/3/cpu = "" (n0,r3)
/local/domain/3/cpu/0 = "" (n0,r3)
/local/domain/3/cpu/0/availability = "online" (n0,r3)
/local/domain/3/cpu/1 = "" (n0,r3)
/local/domain/3/cpu/1/availability = "online" (n0,r3)
/local/domain/3/memory = "" (n0,r3)
/local/domain/3/memory/static-max = "1048576" (n0,r3)
/local/domain/3/memory/target = "1048577" (n0,r3)
/local/domain/3/memory/videoram = "-1" (n0,r3)
/local/domain/3/device = "" (n0,r3)
/local/domain/3/device/suspend = "" (n0,r3)
/local/domain/3/device/suspend/event-channel = "" (n3)
/local/domain/3/device/vbd = "" (n0,r3)
/local/domain/3/device/vbd/51713 = "" (n3,r0)
/local/domain/3/device/vbd/51713/backend = "/local/domain/0/backend/vbd/3/51713" (n3,r0)
/local/domain/3/device/vbd/51713/backend-id = "0" (n3,r0)
/local/domain/3/device/vbd/51713/state = "4" (n3,r0)
/local/domain/3/device/vbd/51713/virtual-device = "51713" (n3,r0)
/local/domain/3/device/vbd/51713/device-type = "disk" (n3,r0)
/local/domain/3/device/vbd/51713/protocol = "x86_64-abi" (n3,r0)
/local/domain/3/device/vbd/51713/ring-ref = "9" (n3,r0)
/local/domain/3/device/vbd/51713/event-channel = "18" (n3,r0)
/local/domain/3/device/vbd/51713/feature-persistent = "1" (n3,r0)
/local/domain/3/device/vbd/51714 = "" (n3,r0)
/local/domain/3/device/vbd/51714/backend = "/local/domain/0/backend/vbd/3/51714" (n3,r0)
/local/domain/3/device/vbd/51714/backend-id = "0" (n3,r0)
/local/domain/3/device/vbd/51714/state = "4" (n3,r0)
/local/domain/3/device/vbd/51714/virtual-device = "51714" (n3,r0)
/local/domain/3/device/vbd/51714/device-type = "disk" (n3,r0)
/local/domain/3/device/vbd/51714/protocol = "x86_64-abi" (n3,r0)
/local/domain/3/device/vbd/51714/ring-ref = "10" (n3,r0)
/local/domain/3/device/vbd/51714/event-channel = "19" (n3,r0)
/local/domain/3/device/vbd/51714/feature-persistent = "1" (n3,r0)
/local/domain/3/device/vbd/51715 = "" (n3,r0)
/local/domain/3/device/vbd/51715/backend = "/local/domain/0/backend/vbd/3/51715" (n3,r0)
/local/domain/3/device/vbd/51715/backend-id = "0" (n3,r0)
/local/domain/3/device/vbd/51715/state = "4" (n3,r0)
/local/domain/3/device/vbd/51715/virtual-device = "51715" (n3,r0)
/local/domain/3/device/vbd/51715/device-type = "disk" (n3,r0)
/local/domain/3/device/vbd/51715/protocol = "x86_64-abi" (n3,r0)
/local/domain/3/device/vbd/51715/ring-ref = "11" (n3,r0)
/local/domain/3/device/vbd/51715/event-channel = "20" (n3,r0)
/local/domain/3/device/vbd/51715/feature-persistent = "1" (n3,r0)
/local/domain/3/device/vif = "" (n0,r3)
/local/domain/3/device/vif/0 = "" (n3,r0)
/local/domain/3/device/vif/0/backend = "/local/domain/0/backend/vif/3/0" (n3,r0)
/local/domain/3/device/vif/0/backend-id = "0" (n3,r0)
/local/domain/3/device/vif/0/state = "4" (n3,r0)
/local/domain/3/device/vif/0/handle = "0" (n3,r0)
/local/domain/3/device/vif/0/mac = "00:16:3e:a0:64:09" (n3,r0)
/local/domain/3/device/vif/0/tx-ring-ref = "12" (n3,r0)
/local/domain/3/device/vif/0/rx-ring-ref = "13" (n3,r0)
/local/domain/3/device/vif/0/event-channel-tx = "21" (n3,r0)
/local/domain/3/device/vif/0/event-channel-rx = "22" (n3,r0)
/local/domain/3/device/vif/0/request-rx-copy = "1" (n3,r0)
/local/domain/3/device/vif/0/feature-rx-notify = "1" (n3,r0)
/local/domain/3/device/vif/0/feature-sg = "1" (n3,r0)
/local/domain/3/device/vif/0/feature-gso-tcpv4 = "1" (n3,r0)
/local/domain/3/device/pci = "" (n0,r3)
/local/domain/3/device/pci/0 = "" (n3,r0)
/local/domain/3/device/pci/0/backend = "/local/domain/0/backend/pci/3/0" (n3,r0)
/local/domain/3/device/pci/0/backend-id = "0" (n3,r0)
/local/domain/3/device/pci/0/state = "4" (n3,r0)
/local/domain/3/device/pci/0/pci-op-ref = "8" (n3,r0)
/local/domain/3/device/pci/0/event-channel = "17" (n3,r0)
/local/domain/3/device/pci/0/magic = "7" (n3,r0)
/local/domain/3/control = "" (n0,r3)
/local/domain/3/control/shutdown = "" (n3)
/local/domain/3/control/platform-feature-multiprocessor-suspend = "1" (n0,r3)
/local/domain/3/control/platform-feature-xs_reset_watches = "1" (n0,r3)
/local/domain/3/data = "" (n3)
/local/domain/3/domid = "3" (n0,r3)
/local/domain/3/store = "" (n0,r3)
/local/domain/3/store/port = "1" (n0,r3)
/local/domain/3/store/ring-ref = "7627436" (n0,r3)
/local/domain/3/console = "" (n0,r3)
/local/domain/3/console/backend = "/local/domain/0/backend/console/3/0" (n0,r3)
/local/domain/3/console/backend-id = "0" (n3,r0)
/local/domain/3/console/limit = "1048576" (n0,r3)
/local/domain/3/console/type = "xenconsoled" (n0,r3)
/local/domain/3/console/output = "pty" (n0,r3)
/local/domain/3/console/tty = "/dev/pts/6" (n0,r3)
/local/domain/3/console/port = "2" (n0,r3)
/local/domain/3/console/ring-ref = "7627435" (n0,r3)
[-- Attachment #3: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-18 13:01 ` Atom2
@ 2014-03-18 15:07 ` Ian Campbell
2014-03-19 0:25 ` Atom2
0 siblings, 1 reply; 23+ messages in thread
From: Ian Campbell @ 2014-03-18 15:07 UTC (permalink / raw)
To: Atom2; +Cc: xen-devel, Ian Jackson, Roger Pau Monne, xen-users
On Tue, 2014-03-18 at 14:01 +0100, Atom2 wrote:
[...]
> So I guess probably not too much of value ...
Unfortunately not, but thanks anyway, it's good to check.
> > While the domain is happily running can you provide the output of
> > "xenstore-ls -fp" -- I'm curious what state pciback is in. It should be
> > 4, if not then that would be the problem.
> Full output of xenstore-ls -fp (from dom0) is attached as it is 627
> lines long and I am not quiet sure what you are actually after): There's
> nothing in it that reads pciback; there are however a few entries named
> /local/domain/0/backend/pci
> and for one of the subentries named 3/0/state (3 is the domain-id)
> the value seems to be 4:
> /local/domain/0/backend/pci/3/0/state = "4" (n0,r3)
Yes, this is the one which libxl appears to be looking for and it is set
to "4" which is what libxl is looking for.
Please can you try this again but take the dump while the destruction is
going on, i.e. in among the 10s delays somewhere (I don't think where
exactly will matter. I don't see it in the logs but I'm wondering if the
pciback directory is getting torn down too soon.
Ian.
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-18 15:07 ` Ian Campbell
@ 2014-03-19 0:25 ` Atom2
2014-03-19 11:26 ` Ian Campbell
0 siblings, 1 reply; 23+ messages in thread
From: Atom2 @ 2014-03-19 0:25 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel, Ian Jackson, Roger Pau Monne, xen-users
Am 18.03.14 16:07, schrieb Ian Campbell:
> On Tue, 2014-03-18 at 14:01 +0100, Atom2 wrote:
> [...]
>> So I guess probably not too much of value ...
>
> Unfortunately not, but thanks anyway, it's good to check.
>
>>> While the domain is happily running can you provide the output of
>>> "xenstore-ls -fp" -- I'm curious what state pciback is in. It should be
>>> 4, if not then that would be the problem.
>> Full output of xenstore-ls -fp (from dom0) is attached as it is 627
>> lines long and I am not quiet sure what you are actually after): There's
>> nothing in it that reads pciback; there are however a few entries named
>> /local/domain/0/backend/pci
>> and for one of the subentries named 3/0/state (3 is the domain-id)
>> the value seems to be 4:
>> /local/domain/0/backend/pci/3/0/state = "4" (n0,r3)
>
> Yes, this is the one which libxl appears to be looking for and it is set
> to "4" which is what libxl is looking for.
>
> Please can you try this again but take the dump while the destruction is
> going on, i.e. in among the 10s delays somewhere (I don't think where
> exactly will matter. I don't see it in the logs but I'm wondering if the
> pciback directory is getting torn down too soon.
>
O.K. - that's what I have done: I created a loop at the shell prompt in
dom0 that executes 'xenstore-ls -fp' for a total of 16 times with a
delay of one second between invocations and stores each output in a
separate file with an extension that counts up from 0. I have started
that "script" just before the "reboot: System halted" message appeared
in the domU, so at least the first file must still contain the info
prior to the 10s delay when the domU is still sort of up and running.
The output below is the result of
grep -E '/local/domain/(0|8)/(backend|device)/pci/.*/state' xenstore.n
where the domain id this time was 8 and n is the counter of my loop.
Please find the output below:
.0 and .1 file:
/local/domain/0/backend/pci/8/0/state = "4" (n0,r8)
/local/domain/0/backend/pci/8/0/state-0 = "3" (n0,r8)
/local/domain/0/backend/pci/8/0/state-1 = "3" (n0,r8)
/local/domain/0/backend/pci/8/0/state-2 = "3" (n0,r8)
/local/domain/0/backend/pci/8/0/state-3 = "3" (n0,r8)
/local/domain/8/device/pci/0/state = "4" (n8,r0)
.2 upto .15 file:
/local/domain/0/backend/pci/8/0/state = "6" (n0,r8)
/local/domain/0/backend/pci/8/0/state-0 = "3" (n0,r8)
/local/domain/0/backend/pci/8/0/state-1 = "3" (n0,r8)
/local/domain/0/backend/pci/8/0/state-2 = "3" (n0,r8)
/local/domain/0/backend/pci/8/0/state-3 = "3" (n0,r8)
/local/domain/8/device/pci/0/state = "6" (n8,r0)
So it seems that pretty much at the start of the 10s delay the state
changed from 4 to 6 and stays at that value even after the first 10s
delay is over - whatever that means.
And for the sake of being complete:
Below is the output from starting the domain prior to the grub menu
showing on screen. There is one device with which xl seems to have an
issue (for which I still have to find and compile a driver anyway), but
I think that's unrelated because if I drop that one device from being
passed through, the delay of 10s per remaining device nevertheless remains.
===========================
Parsing config from 5:voip.9
libxl: error: libxl_pci.c:990:libxl__device_pci_reset: The kernel
doesn't support reset from sysfs for PCI device 0000:09:02.0
Daemon running with PID 4759
Xen Minimal OS!
start_info: 0xbab000(VA)
nr_pages: 0x40000
shared_inf: 0xdb4f9000(MA)
pt_base: 0xbae000(VA)
nr_pt_frames: 0xb
mfn_list: 0x9ab000(VA)
mod_start: 0x9aa000(VA)
mod_len: 4096
flags: 0x0
cmd_line:
stack: 0x9690e0-0x9890e0
MM: Init
_text: 0x0(VA)
_etext: 0x7b4c5(VA)
_erodata: 0x96000(VA)
_edata: 0x9bd20(VA)
stack start: 0x9690e0(VA)
_end: 0x9a96e0(VA)
start_pfn: bbc
max_pfn: 40000
Mapping memory range 0x1000000 - 0x40000000
setting 0x0-0x96000 readonly
skipped 0x1000
MM: Initialise page allocator for db4000(db4000)-40000000(40000000) MM: done
Demand map pfns at 40001000-2040001000.
Heap resides at 2040002000-4040002000.
Initialising timer interface
Initialising console ... done.
gnttab_table mapped at 0x40001000.
Initialising scheduler
Thread "Idle": pointer: 0x2040002050, stack: 0xfd0000
Thread "xenstore": pointer: 0x2040002800, stack: 0xfe0000
xenbus initialised on irq 1 mfn 0x7a512a
Thread "shutdown": pointer: 0x2040002fb0, stack: 0xff0000
Dummy main: start_info=0x9891e0
Thread "main": pointer: 0x2040003760, stack: 0x1000000
"main"
vbd 51713 is hd0
******************* BLKFRONT for device/vbd/51713 **********
Shutting down ()
Shutdown requested: 3
Thread "shutdown" exited.
backend at /local/domain/0/backend/vbd/8/51713
16777216 sectors of 512 bytes
**************************
vbd 51714 is hd1
******************* BLKFRONT for device/vbd/51714 **********
backend at /local/domain/0/backend/vbd/8/51714
2097152 sectors of 512 bytes
**************************
vbd 51715 is hd2
******************* BLKFRONT for device/vbd/51715 **********
backend at /local/domain/0/backend/vbd/8/51715
2097152 sectors of 512 bytes
**************************
===========================
Thanks Atom2
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-19 0:25 ` Atom2
@ 2014-03-19 11:26 ` Ian Campbell
2014-03-19 13:00 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 23+ messages in thread
From: Ian Campbell @ 2014-03-19 11:26 UTC (permalink / raw)
To: Atom2; +Cc: xen-devel, Ian Jackson, Roger Pau Monne, xen-users
On Wed, 2014-03-19 at 01:25 +0100, Atom2 wrote:
> So it seems that pretty much at the start of the 10s delay the state
> changed from 4 to 6 and stays at that value even after the first 10s
> delay is over - whatever that means.
4 == Connected
6 == Closed
I think what is happening is that the domain is shutting down, which
causes pciback to transition to the closed state (because the f.e. went
away, so this is a reasonable thing for it to do).
The bug appears to be that libxl is trying to "hot unplug" the devices
on shutdown when they have already been effectively "cold unplugged" by
the domain going down.
Perhaps libxl__device_pci_remove_xenstore should observe that the state
is > 4 (hence closing/closed) and not bother doing anything, i.e. only
waiting iff the state is <4 (init, connecting etc)? Or unconditionally
removing the nodes if state > 4. (perhaps state 7, reconfiguring needs
handling here too)
Or perhaps the force parameter passed to remove_common (which indicates
destroy rather than unplug) ought to be propagated down to this code and
$something done with it.
Roger, Ian, any thoughts on that?
Ian.
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-19 11:26 ` Ian Campbell
@ 2014-03-19 13:00 ` Konrad Rzeszutek Wilk
2014-03-19 14:03 ` Atom2
0 siblings, 1 reply; 23+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-03-19 13:00 UTC (permalink / raw)
To: Ian Campbell; +Cc: Roger Pau Monne, xen-devel, Ian Jackson, Atom2, xen-users
On Wed, Mar 19, 2014 at 11:26:24AM +0000, Ian Campbell wrote:
> On Wed, 2014-03-19 at 01:25 +0100, Atom2 wrote:
> > So it seems that pretty much at the start of the 10s delay the state
> > changed from 4 to 6 and stays at that value even after the first 10s
> > delay is over - whatever that means.
>
> 4 == Connected
> 6 == Closed
>
> I think what is happening is that the domain is shutting down, which
> causes pciback to transition to the closed state (because the f.e. went
> away, so this is a reasonable thing for it to do).
>
> The bug appears to be that libxl is trying to "hot unplug" the devices
> on shutdown when they have already been effectively "cold unplugged" by
> the domain going down.
>
> Perhaps libxl__device_pci_remove_xenstore should observe that the state
> is > 4 (hence closing/closed) and not bother doing anything, i.e. only
> waiting iff the state is <4 (init, connecting etc)? Or unconditionally
> removing the nodes if state > 4. (perhaps state 7, reconfiguring needs
> handling here too)
>
> Or perhaps the force parameter passed to remove_common (which indicates
> destroy rather than unplug) ought to be propagated down to this code and
> $something done with it.
>
> Roger, Ian, any thoughts on that?
This reminds me of this bug:
commit 098b1aeaf4d6149953b8f1f8d55c21d85536fbff
Author: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Date: Mon Jun 10 16:48:09 2013 -0400
xen/pcifront: Deal with toolstack missing 'XenbusStateClosing' state.
... snip..
In other words, this 4(Connected)->5(Closing)->4(Connected) state
was expected, while 4(Connected)->.... anything but 5(Closing)->4(Connected)
was not. This patch removes that aggressive check and allows
Xen pcifront to work with the 'xl' toolstack (for one or more
PCI devices) and with 'xm' toolstack (for more than two PCI
devices).
But this seems to be a different state issue?
Ariel/Atom2, do you see this behavior with 'xend'? And what is the version of Linux
kernel you are running as guest?
>
> Ian.
>
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-19 13:00 ` Konrad Rzeszutek Wilk
@ 2014-03-19 14:03 ` Atom2
2014-03-19 15:45 ` Ian Jackson
0 siblings, 1 reply; 23+ messages in thread
From: Atom2 @ 2014-03-19 14:03 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk, Ian Campbell
Cc: xen-devel, Ian Jackson, Roger Pau Monne, xen-users
Am 19.03.14 14:00, schrieb Konrad Rzeszutek Wilk:
> On Wed, Mar 19, 2014 at 11:26:24AM +0000, Ian Campbell wrote:
>> On Wed, 2014-03-19 at 01:25 +0100, Atom2 wrote:
>>> So it seems that pretty much at the start of the 10s delay the state
>>> changed from 4 to 6 and stays at that value even after the first 10s
>>> delay is over - whatever that means.
>>
>> 4 == Connected
>> 6 == Closed
>>
>> I think what is happening is that the domain is shutting down, which
>> causes pciback to transition to the closed state (because the f.e. went
>> away, so this is a reasonable thing for it to do).
>>
>> The bug appears to be that libxl is trying to "hot unplug" the devices
>> on shutdown when they have already been effectively "cold unplugged" by
>> the domain going down.
I might be wrong, but this behaviour is somehow reminescent of (although
not identical to) the bug in the vif-bridge script that I reported some
time ago (see http://xen.markmail.org/thread/auroivzr4vje3bzn ; btw
discussions there seem to have stalled): The vif-bridge script also
tried to do something (i.e. deleting an i/f from the bridge and bringing
down the i/f) which obviously has already been done through shutting
down the guest domain.
>>
>> Perhaps libxl__device_pci_remove_xenstore should observe that the state
>> is > 4 (hence closing/closed) and not bother doing anything, i.e. only
>> waiting iff the state is <4 (init, connecting etc)? Or unconditionally
>> removing the nodes if state > 4. (perhaps state 7, reconfiguring needs
>> handling here too)
>>
>> Or perhaps the force parameter passed to remove_common (which indicates
>> destroy rather than unplug) ought to be propagated down to this code and
>> $something done with it.
>>
>> Roger, Ian, any thoughts on that?
>
> This reminds me of this bug:
>
> commit 098b1aeaf4d6149953b8f1f8d55c21d85536fbff
> Author: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Date: Mon Jun 10 16:48:09 2013 -0400
>
> xen/pcifront: Deal with toolstack missing 'XenbusStateClosing' state.
>
> ... snip..
>
> In other words, this 4(Connected)->5(Closing)->4(Connected) state
> was expected, while 4(Connected)->.... anything but 5(Closing)->4(Connected)
> was not. This patch removes that aggressive check and allows
> Xen pcifront to work with the 'xl' toolstack (for one or more
> PCI devices) and with 'xm' toolstack (for more than two PCI
> devices).
>
> But this seems to be a different state issue?
>
> Ariel/Atom2, do you see this behavior with 'xend'? And what is the version of Linux
> kernel you are running as guest?
Hi Konrad -
nope, I am using xl; there is no xend or xm installed on the machine or
involved anyhow (I assumed with xend you referred back to xm instead of xl).
The xen (and xen-tools) version is 4.3.1-r5 and the linux kernel is
3.11.7-r1 from gentoo hardened-sources (that's both for guest and for
dom0 - although clearly with different kernel configs). Both the kernel
and xen/xen-tools are the latest stable versions available as ebuilds
from gentoo.
>>
>> Ian.
>>
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-19 14:03 ` Atom2
@ 2014-03-19 15:45 ` Ian Jackson
2014-03-20 2:31 ` Atom2
0 siblings, 1 reply; 23+ messages in thread
From: Ian Jackson @ 2014-03-19 15:45 UTC (permalink / raw)
To: Atom2; +Cc: Roger Pau Monne, xen-users, xen-devel, Ian Campbell
Atom2 writes ("Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough"):
> nope, I am using xl; there is no xend or xm installed on the machine or
> involved anyhow (I assumed with xend you referred back to xm instead of xl).
Can you try this patch ?
Thanks,
Ian.
diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 30b0b06..1583498 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -2728,7 +2728,7 @@ static void local_device_attach_cb(libxl__egc *egc, libxl__ao_device *aodev)
if (rc < 0)
goto out;
be_path = libxl__device_backend_path(gc, &device);
- rc = libxl__wait_for_backend(gc, be_path, "4");
+ rc = libxl__wait_for_backend_deprecated(gc, be_path, "4", (char*)0);
if (rc < 0)
goto out;
diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c
index fa99f77..11a9885 100644
--- a/tools/libxl/libxl_device.c
+++ b/tools/libxl/libxl_device.c
@@ -1208,12 +1208,14 @@ int libxl__wait_for_device_model_deprecated(libxl__gc *gc,
check_callback, check_callback_userdata);
}
-int libxl__wait_for_backend(libxl__gc *gc, const char *be_path,
- const char *state)
+int libxl__wait_for_backend_deprecated(libxl__gc *gc, const char *be_path,
+ ...)
{
int watchdog = 100;
const char *p, *path = GCSPRINTF("%s/state", be_path);
+ const char *want;
int rc;
+ va_list al;
while (watchdog-- > 0) {
rc = libxl__xs_read_checked(gc, XBT_NULL, path, &p);
@@ -1224,8 +1226,14 @@ int libxl__wait_for_backend(libxl__gc *gc, const char *be_path,
return ERROR_FAIL;
}
- if (!strcmp(p, state))
- return 0;
+ va_start(al,be_path);
+ while ((want = va_arg(al, char*))) {
+ if (!strcmp(p, want)) {
+ va_end(al);
+ return 0;
+ }
+ }
+ va_end(al);
usleep(100000);
}
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index b3a200d..bdcce35 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1025,8 +1025,8 @@ _hidden char *libxl__device_frontend_path(libxl__gc *gc, libxl__device *device);
_hidden int libxl__parse_backend_path(libxl__gc *gc, const char *path,
libxl__device *dev);
_hidden int libxl__device_destroy(libxl__gc *gc, libxl__device *dev);
-_hidden int libxl__wait_for_backend(libxl__gc *gc, const char *be_path,
- const char *state);
+_hidden int libxl__wait_for_backend_deprecated(libxl__gc *gc,
+ const char *be_path, ...) __attribute__((sentinel));
_hidden int libxl__nic_type(libxl__gc *gc, libxl__device *dev,
libxl_nic_type *nictype);
diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c
index 44d0453..43ffd57 100644
--- a/tools/libxl/libxl_pci.c
+++ b/tools/libxl/libxl_pci.c
@@ -126,7 +126,7 @@ static int libxl__device_pci_add_xenstore(libxl__gc *gc, uint32_t domid, libxl_d
return ERROR_FAIL;
if (!starting && domtype == LIBXL_DOMAIN_TYPE_PV) {
- if (libxl__wait_for_backend(gc, be_path, "4") < 0)
+ if (libxl__wait_for_backend_deprecated(gc, be_path, "4", (char*)0) < 0)
return ERROR_FAIL;
}
@@ -169,7 +169,8 @@ static int libxl__device_pci_remove_xenstore(libxl__gc *gc, uint32_t domid, libx
return ERROR_FAIL;
if (domtype == LIBXL_DOMAIN_TYPE_PV) {
- if (libxl__wait_for_backend(gc, be_path, "4") < 0) {
+ if (libxl__wait_for_backend_deprecated(gc, be_path, "4", "6", (char*)0)
+ < 0) {
LIBXL__LOG(ctx, LIBXL__LOG_DEBUG, "pci backend at %s is not ready", be_path);
return ERROR_FAIL;
}
@@ -198,7 +199,8 @@ retry_transaction:
goto retry_transaction;
if (domtype == LIBXL_DOMAIN_TYPE_PV) {
- if (libxl__wait_for_backend(gc, be_path, "4") < 0) {
+ if (libxl__wait_for_backend_deprecated(gc, be_path, "4", "6", (char*)0)
+ < 0) {
LIBXL__LOG(ctx, LIBXL__LOG_DEBUG, "pci backend at %s is not ready", be_path);
return ERROR_FAIL;
}
^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-19 15:45 ` Ian Jackson
@ 2014-03-20 2:31 ` Atom2
2014-03-20 11:52 ` Ian Jackson
0 siblings, 1 reply; 23+ messages in thread
From: Atom2 @ 2014-03-20 2:31 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-users, xen-devel, Ian Campbell, Roger Pau Monne
[-- Attachment #1: Type: text/plain, Size: 5510 bytes --]
Sorry for my delay in answering - this is a resend as the first e-Mail
with uncompressed attachments did not go through.
Am 19.03.14 16:45, schrieb Ian Jackson:
> Atom2 writes ("Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough"):
>> nope, I am using xl; there is no xend or xm installed on the machine or
>> involved anyhow (I assumed with xend you referred back to xm instead of xl).
>
> Can you try this patch ?
>
> Thanks,
> Ian.
Hi Ian,
the patch unfortunately doesn't apply to my sources - some comments to
the reasons why further below.
Just FYI: the version I am using is 4.3.1-r5; I have attached the
relevant source files referred to by your patches.
Thanks and regards,
Atom2
>
> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
> index 30b0b06..1583498 100644
> --- a/tools/libxl/libxl.c
> +++ b/tools/libxl/libxl.c
> @@ -2728,7 +2728,7 @@ static void local_device_attach_cb(libxl__egc *egc, libxl__ao_device *aodev)
> if (rc < 0)
> goto out;
> be_path = libxl__device_backend_path(gc, &device);
> - rc = libxl__wait_for_backend(gc, be_path, "4");
> + rc = libxl__wait_for_backend_deprecated(gc, be_path, "4", (char*)0);
> if (rc < 0)
> goto out;
>
This one would apply with an offset of 43 lines - it should therefore be
o.k I guess.
> diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c
> index fa99f77..11a9885 100644
> --- a/tools/libxl/libxl_device.c
> +++ b/tools/libxl/libxl_device.c
> @@ -1208,12 +1208,14 @@ int libxl__wait_for_device_model_deprecated(libxl__gc *gc,
> check_callback, check_callback_userdata);
> }
>
> -int libxl__wait_for_backend(libxl__gc *gc, const char *be_path,
> - const char *state)
> +int libxl__wait_for_backend_deprecated(libxl__gc *gc, const char *be_path,
> + ...)
> {
> int watchdog = 100;
> const char *p, *path = GCSPRINTF("%s/state", be_path);
> + const char *want;
> int rc;
> + va_list al;
>
> while (watchdog-- > 0) {
> rc = libxl__xs_read_checked(gc, XBT_NULL, path, &p);
> @@ -1224,8 +1226,14 @@ int libxl__wait_for_backend(libxl__gc *gc, const char *be_path,
> return ERROR_FAIL;
> }
>
> - if (!strcmp(p, state))
> - return 0;
> + va_start(al,be_path);
> + while ((want = va_arg(al, char*))) {
> + if (!strcmp(p, want)) {
> + va_end(al);
> + return 0;
> + }
> + }
> + va_end(al);
>
> usleep(100000);
> }
This one does not apply, not the least because there is no function
libxl__wait_for_device_model_deprecated in the source. Furthermore the
definition and the body of libxl__wait_for_backend looks differently to
what the patch seems to expect - please see attached source file.
> diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
> index b3a200d..bdcce35 100644
> --- a/tools/libxl/libxl_internal.h
> +++ b/tools/libxl/libxl_internal.h
> @@ -1025,8 +1025,8 @@ _hidden char *libxl__device_frontend_path(libxl__gc *gc, libxl__device *device);
> _hidden int libxl__parse_backend_path(libxl__gc *gc, const char *path,
> libxl__device *dev);
> _hidden int libxl__device_destroy(libxl__gc *gc, libxl__device *dev);
> -_hidden int libxl__wait_for_backend(libxl__gc *gc, const char *be_path,
> - const char *state);
> +_hidden int libxl__wait_for_backend_deprecated(libxl__gc *gc,
> + const char *be_path, ...) __attribute__((sentinel));
> _hidden int libxl__nic_type(libxl__gc *gc, libxl__device *dev,
> libxl_nic_type *nictype);
>
This one fails because the definition of libxl__wait_for_backend(...) is
different (see above) - please see attached source file.
> diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c
> index 44d0453..43ffd57 100644
> --- a/tools/libxl/libxl_pci.c
> +++ b/tools/libxl/libxl_pci.c
> @@ -126,7 +126,7 @@ static int libxl__device_pci_add_xenstore(libxl__gc *gc, uint32_t domid, libxl_d
> return ERROR_FAIL;
>
> if (!starting && domtype == LIBXL_DOMAIN_TYPE_PV) {
> - if (libxl__wait_for_backend(gc, be_path, "4") < 0)
> + if (libxl__wait_for_backend_deprecated(gc, be_path, "4", (char*)0) < 0)
> return ERROR_FAIL;
> }
>
> @@ -169,7 +169,8 @@ static int libxl__device_pci_remove_xenstore(libxl__gc *gc, uint32_t domid, libx
> return ERROR_FAIL;
>
> if (domtype == LIBXL_DOMAIN_TYPE_PV) {
> - if (libxl__wait_for_backend(gc, be_path, "4") < 0) {
> + if (libxl__wait_for_backend_deprecated(gc, be_path, "4", "6", (char*)0)
> + < 0) {
> LIBXL__LOG(ctx, LIBXL__LOG_DEBUG, "pci backend at %s is not ready", be_path);
> return ERROR_FAIL;
> }
> @@ -198,7 +199,8 @@ retry_transaction:
> goto retry_transaction;
>
> if (domtype == LIBXL_DOMAIN_TYPE_PV) {
> - if (libxl__wait_for_backend(gc, be_path, "4") < 0) {
> + if (libxl__wait_for_backend_deprecated(gc, be_path, "4", "6", (char*)0)
> + < 0) {
> LIBXL__LOG(ctx, LIBXL__LOG_DEBUG, "pci backend at %s is not ready", be_path);
> return ERROR_FAIL;
> }
>
The last one does apply cleanly - even without any offset.
[-- Attachment #2: libxl_device.c.7z --]
[-- Type: application/octet-stream, Size: 8161 bytes --]
[-- Attachment #3: libxl_internal.h.7z --]
[-- Type: application/octet-stream, Size: 28765 bytes --]
[-- Attachment #4: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-20 2:31 ` Atom2
@ 2014-03-20 11:52 ` Ian Jackson
2014-03-20 13:53 ` Pasi Kärkkäinen
2014-03-20 19:32 ` Atom2
0 siblings, 2 replies; 23+ messages in thread
From: Ian Jackson @ 2014-03-20 11:52 UTC (permalink / raw)
To: Atom2; +Cc: xen-users, xen-devel, Ian Campbell, Roger Pau Monne
Atom2 writes ("Re: [Xen-devel] [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough"):
> the patch unfortunately doesn't apply to my sources - some comments to
> the reasons why further below.
Here's a backport. I have compiled but not executed it.
> Sorry for my delay in answering - this is a resend as the first e-Mail
> with uncompressed attachments did not go through.
> Just FYI: the version I am using is 4.3.1-r5; I have attached the
> relevant source files referred to by your patches.
Thanks, but our revision control system enables us to retrieve old
versions very easily :-). So there is not any need to provide us with
these files.
Having said that, I have no record of 4.3.1-rc5. But I'm pretty sure
the patch below, which is against staging-4.3, will apply to your
tree. It applies cleanly to 4.3.0-rc5 and 4.3.1-rc2, which are my two
guesses as to which version you mean.
Thanks,
Ian.
>From f9df128cd4d4ad6c7ed6ffd9bd8ba0633af78389 Mon Sep 17 00:00:00 2001
From: Ian Jackson <ian.jackson@eu.citrix.com>
Date: Wed, 19 Mar 2014 15:47:02 +0000
Subject: [PATCH] libxl: Tolerate backend state "6" on pciback remove
When shutting down a domain with pci passthrough, it can happen that
the backend has actually shut down (xenbus state 6) before we try to
remove it. When this happens, libxl would time out waiting for the
backend to reach state 4.
Instead, deal with this by having libxl__wait_for_backend take a list
of suitable states.
The arrangements are still fundamentally incorrect:
- libxl__wait_for_backend is a slow synchronous function, which is
forbidden;
- There is no way to deal properly with the various xenbus states
that might arise (including erroneous ones).
We will hopefully fix this later, although it's not trivial. For
the moment, rename the function to libxl__wait_for_backend_deprecated.
Reported-by: Atom2 <ariel.atom2@web2web.at>
Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
CC: Atom2 <ariel.atom2@web2web.at>
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Roger Pau Monne <roger.pau@citrix.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
Backported to 4.3. Conflicts:
tools/libxl/libxl_device.c
tools/libxl/libxl_internal.h
Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
tools/libxl/libxl.c | 2 +-
tools/libxl/libxl_device.c | 21 ++++++++++++++-------
tools/libxl/libxl_internal.h | 3 ++-
tools/libxl/libxl_pci.c | 8 +++++---
4 files changed, 22 insertions(+), 12 deletions(-)
diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 3d9543b..c0cc0b7 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -2689,7 +2689,7 @@ static void local_device_attach_cb(libxl__egc *egc, libxl__ao_device *aodev)
if (rc < 0)
goto out;
be_path = libxl__device_backend_path(gc, &device);
- rc = libxl__wait_for_backend(gc, be_path, "4");
+ rc = libxl__wait_for_backend_deprecated(gc, be_path, "4", (char*)0);
if (rc < 0)
goto out;
diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c
index ea845b7..779b38b 100644
--- a/tools/libxl/libxl_device.c
+++ b/tools/libxl/libxl_device.c
@@ -1094,7 +1094,8 @@ int libxl__wait_for_device_model(libxl__gc *gc,
check_callback, check_callback_userdata);
}
-int libxl__wait_for_backend(libxl__gc *gc, char *be_path, char *state)
+int libxl__wait_for_backend_deprecated(libxl__gc *gc, const char *be_path,
+ ...)
{
libxl_ctx *ctx = libxl__gc_owner(gc);
int watchdog = 100;
@@ -1115,13 +1116,19 @@ int libxl__wait_for_backend(libxl__gc *gc, char *be_path, char *state)
}
goto out;
} else {
- if (!strcmp(p, state)) {
- rc = 0;
- goto out;
- } else {
- usleep(100000);
- watchdog--;
+ const char *want;
+ va_list al;
+ va_start(al,be_path);
+ while ((want = va_arg(al, char*))) {
+ if (!strcmp(p, want)) {
+ va_end(al);
+ rc = 0;
+ goto out;
+ }
}
+ va_end(al);
+ usleep(100000);
+ watchdog--;
}
}
LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "Backend %s not ready", be_path);
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index f051d91..4485c56 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -944,7 +944,8 @@ _hidden char *libxl__device_frontend_path(libxl__gc *gc, libxl__device *device);
_hidden int libxl__parse_backend_path(libxl__gc *gc, const char *path,
libxl__device *dev);
_hidden int libxl__device_destroy(libxl__gc *gc, libxl__device *dev);
-_hidden int libxl__wait_for_backend(libxl__gc *gc, char *be_path, char *state);
+_hidden int libxl__wait_for_backend_deprecated(libxl__gc *gc,
+ const char *be_path, ...) __attribute__((sentinel));
_hidden int libxl__nic_type(libxl__gc *gc, libxl__device *dev,
libxl_nic_type *nictype);
diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c
index 0295e0b..e22852c 100644
--- a/tools/libxl/libxl_pci.c
+++ b/tools/libxl/libxl_pci.c
@@ -126,7 +126,7 @@ static int libxl__device_pci_add_xenstore(libxl__gc *gc, uint32_t domid, libxl_d
return ERROR_FAIL;
if (!starting && domtype == LIBXL_DOMAIN_TYPE_PV) {
- if (libxl__wait_for_backend(gc, be_path, "4") < 0)
+ if (libxl__wait_for_backend_deprecated(gc, be_path, "4", (char*)0) < 0)
return ERROR_FAIL;
}
@@ -169,7 +169,8 @@ static int libxl__device_pci_remove_xenstore(libxl__gc *gc, uint32_t domid, libx
return ERROR_FAIL;
if (domtype == LIBXL_DOMAIN_TYPE_PV) {
- if (libxl__wait_for_backend(gc, be_path, "4") < 0) {
+ if (libxl__wait_for_backend_deprecated(gc, be_path, "4", "6", (char*)0)
+ < 0) {
LIBXL__LOG(ctx, LIBXL__LOG_DEBUG, "pci backend at %s is not ready", be_path);
return ERROR_FAIL;
}
@@ -198,7 +199,8 @@ retry_transaction:
goto retry_transaction;
if (domtype == LIBXL_DOMAIN_TYPE_PV) {
- if (libxl__wait_for_backend(gc, be_path, "4") < 0) {
+ if (libxl__wait_for_backend_deprecated(gc, be_path, "4", "6", (char*)0)
+ < 0) {
LIBXL__LOG(ctx, LIBXL__LOG_DEBUG, "pci backend at %s is not ready", be_path);
return ERROR_FAIL;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-20 11:52 ` Ian Jackson
@ 2014-03-20 13:53 ` Pasi Kärkkäinen
2014-03-20 15:28 ` Ian Jackson
2014-03-20 19:34 ` Atom2
2014-03-20 19:32 ` Atom2
1 sibling, 2 replies; 23+ messages in thread
From: Pasi Kärkkäinen @ 2014-03-20 13:53 UTC (permalink / raw)
To: Ian Jackson; +Cc: Roger Pau Monne, Ian Campbell, xen-devel, Atom2, xen-users
On Thu, Mar 20, 2014 at 11:52:57AM +0000, Ian Jackson wrote:
> Atom2 writes ("Re: [Xen-devel] [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough"):
> > the patch unfortunately doesn't apply to my sources - some comments to
> > the reasons why further below.
>
> Here's a backport. I have compiled but not executed it.
>
> > Sorry for my delay in answering - this is a resend as the first e-Mail
> > with uncompressed attachments did not go through.
>
> > Just FYI: the version I am using is 4.3.1-r5; I have attached the
> > relevant source files referred to by your patches.
>
> Thanks, but our revision control system enables us to retrieve old
> versions very easily :-). So there is not any need to provide us with
> these files.
>
> Having said that, I have no record of 4.3.1-rc5. But I'm pretty sure
> the patch below, which is against staging-4.3, will apply to your
> tree. It applies cleanly to 4.3.0-rc5 and 4.3.1-rc2, which are my two
> guesses as to which version you mean.
>
Often "r5" style versions refer to Gentoo packaging..
So maybe he actually didn't have a typo in 4.3.1-r5 :)
-- Pasi
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-20 13:53 ` Pasi Kärkkäinen
@ 2014-03-20 15:28 ` Ian Jackson
2014-03-20 19:34 ` Atom2
1 sibling, 0 replies; 23+ messages in thread
From: Ian Jackson @ 2014-03-20 15:28 UTC (permalink / raw)
To: Pasi Kärkkäinen
Cc: Roger Pau Monne, Ian Campbell, xen-devel, Atom2, xen-users
Pasi Kärkkäinen writes ("Re: [Xen-devel] [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough"):
> On Thu, Mar 20, 2014 at 11:52:57AM +0000, Ian Jackson wrote:
...
> > Having said that, I have no record of 4.3.1-rc5. But I'm pretty sure
> > the patch below, which is against staging-4.3, will apply to your
> > tree. It applies cleanly to 4.3.0-rc5 and 4.3.1-rc2, which are my two
> > guesses as to which version you mean.
> >
>
> Often "r5" style versions refer to Gentoo packaging..
> So maybe he actually didn't have a typo in 4.3.1-r5 :)
Ah. I didn't even spot the missing "c". But I bet my 4.3-based patch
will apply.
Ian.
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-20 11:52 ` Ian Jackson
2014-03-20 13:53 ` Pasi Kärkkäinen
@ 2014-03-20 19:32 ` Atom2
2014-03-21 18:11 ` Ian Jackson
1 sibling, 1 reply; 23+ messages in thread
From: Atom2 @ 2014-03-20 19:32 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-users, xen-devel, Ian Campbell, Roger Pau Monne
[-- Attachment #1: Type: text/plain, Size: 8127 bytes --]
Am 20.03.14 12:52, schrieb Ian Jackson:
> Atom2 writes ("Re: [Xen-devel] [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough"):
>> the patch unfortunately doesn't apply to my sources - some comments to
>> the reasons why further below.
>
> Here's a backport. I have compiled but not executed it.
It compiled at my end as well, but I am sorry to report that the problem
with the 40s delay persits.
Attached please find the new xl-output created with xl -vvv create -F
domain.
This file again contains a few annotations with regards to where the
delays happen. To my untrained eye it looks largely identical to the
last xl-output with the obvious change of domain-id, addresses,
line-numbers for debug output where changes in the sourve have happende
and the use of the new function libxl__wait_for_backend_deprecated
instead of libxl__wait_for_backend due to your patch, the latter of
which I take as proof that your patches have been applied.
For other obvious changes I have commented in the file on those lines
that I could identify: There are a few new lines which were not there
last time and now there's also a new 10s delay which, however, is only
visible through xl -vvvv -F domain as the prompt has already returned by
the time this new delay happens and this delay therefore is not visible
in dom0.
I hope that helps. Thanks for your continued support and best regards,
Atom2
>
>> Sorry for my delay in answering - this is a resend as the first e-Mail
>> with uncompressed attachments did not go through.
>
>> Just FYI: the version I am using is 4.3.1-r5; I have attached the
>> relevant source files referred to by your patches.
>
> Thanks, but our revision control system enables us to retrieve old
> versions very easily :-). So there is not any need to provide us with
> these files.
>
> Having said that, I have no record of 4.3.1-rc5. But I'm pretty sure
> the patch below, which is against staging-4.3, will apply to your
> tree. It applies cleanly to 4.3.0-rc5 and 4.3.1-rc2, which are my two
> guesses as to which version you mean.
>
> Thanks,
> Ian.
>
> From f9df128cd4d4ad6c7ed6ffd9bd8ba0633af78389 Mon Sep 17 00:00:00 2001
> From: Ian Jackson <ian.jackson@eu.citrix.com>
> Date: Wed, 19 Mar 2014 15:47:02 +0000
> Subject: [PATCH] libxl: Tolerate backend state "6" on pciback remove
>
> When shutting down a domain with pci passthrough, it can happen that
> the backend has actually shut down (xenbus state 6) before we try to
> remove it. When this happens, libxl would time out waiting for the
> backend to reach state 4.
>
> Instead, deal with this by having libxl__wait_for_backend take a list
> of suitable states.
>
> The arrangements are still fundamentally incorrect:
> - libxl__wait_for_backend is a slow synchronous function, which is
> forbidden;
> - There is no way to deal properly with the various xenbus states
> that might arise (including erroneous ones).
> We will hopefully fix this later, although it's not trivial. For
> the moment, rename the function to libxl__wait_for_backend_deprecated.
>
> Reported-by: Atom2 <ariel.atom2@web2web.at>
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
> CC: Atom2 <ariel.atom2@web2web.at>
> CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> CC: Roger Pau Monne <roger.pau@citrix.com>
> CC: Ian Campbell <Ian.Campbell@citrix.com>
>
> Backported to 4.3. Conflicts:
> tools/libxl/libxl_device.c
> tools/libxl/libxl_internal.h
> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
> ---
> tools/libxl/libxl.c | 2 +-
> tools/libxl/libxl_device.c | 21 ++++++++++++++-------
> tools/libxl/libxl_internal.h | 3 ++-
> tools/libxl/libxl_pci.c | 8 +++++---
> 4 files changed, 22 insertions(+), 12 deletions(-)
>
> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
> index 3d9543b..c0cc0b7 100644
> --- a/tools/libxl/libxl.c
> +++ b/tools/libxl/libxl.c
> @@ -2689,7 +2689,7 @@ static void local_device_attach_cb(libxl__egc *egc, libxl__ao_device *aodev)
> if (rc < 0)
> goto out;
> be_path = libxl__device_backend_path(gc, &device);
> - rc = libxl__wait_for_backend(gc, be_path, "4");
> + rc = libxl__wait_for_backend_deprecated(gc, be_path, "4", (char*)0);
> if (rc < 0)
> goto out;
>
> diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c
> index ea845b7..779b38b 100644
> --- a/tools/libxl/libxl_device.c
> +++ b/tools/libxl/libxl_device.c
> @@ -1094,7 +1094,8 @@ int libxl__wait_for_device_model(libxl__gc *gc,
> check_callback, check_callback_userdata);
> }
>
> -int libxl__wait_for_backend(libxl__gc *gc, char *be_path, char *state)
> +int libxl__wait_for_backend_deprecated(libxl__gc *gc, const char *be_path,
> + ...)
> {
> libxl_ctx *ctx = libxl__gc_owner(gc);
> int watchdog = 100;
> @@ -1115,13 +1116,19 @@ int libxl__wait_for_backend(libxl__gc *gc, char *be_path, char *state)
> }
> goto out;
> } else {
> - if (!strcmp(p, state)) {
> - rc = 0;
> - goto out;
> - } else {
> - usleep(100000);
> - watchdog--;
> + const char *want;
> + va_list al;
> + va_start(al,be_path);
> + while ((want = va_arg(al, char*))) {
> + if (!strcmp(p, want)) {
> + va_end(al);
> + rc = 0;
> + goto out;
> + }
> }
> + va_end(al);
> + usleep(100000);
> + watchdog--;
> }
> }
> LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "Backend %s not ready", be_path);
> diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
> index f051d91..4485c56 100644
> --- a/tools/libxl/libxl_internal.h
> +++ b/tools/libxl/libxl_internal.h
> @@ -944,7 +944,8 @@ _hidden char *libxl__device_frontend_path(libxl__gc *gc, libxl__device *device);
> _hidden int libxl__parse_backend_path(libxl__gc *gc, const char *path,
> libxl__device *dev);
> _hidden int libxl__device_destroy(libxl__gc *gc, libxl__device *dev);
> -_hidden int libxl__wait_for_backend(libxl__gc *gc, char *be_path, char *state);
> +_hidden int libxl__wait_for_backend_deprecated(libxl__gc *gc,
> + const char *be_path, ...) __attribute__((sentinel));
> _hidden int libxl__nic_type(libxl__gc *gc, libxl__device *dev,
> libxl_nic_type *nictype);
>
> diff --git a/tools/libxl/libxl_pci.c b/tools/libxl/libxl_pci.c
> index 0295e0b..e22852c 100644
> --- a/tools/libxl/libxl_pci.c
> +++ b/tools/libxl/libxl_pci.c
> @@ -126,7 +126,7 @@ static int libxl__device_pci_add_xenstore(libxl__gc *gc, uint32_t domid, libxl_d
> return ERROR_FAIL;
>
> if (!starting && domtype == LIBXL_DOMAIN_TYPE_PV) {
> - if (libxl__wait_for_backend(gc, be_path, "4") < 0)
> + if (libxl__wait_for_backend_deprecated(gc, be_path, "4", (char*)0) < 0)
> return ERROR_FAIL;
> }
>
> @@ -169,7 +169,8 @@ static int libxl__device_pci_remove_xenstore(libxl__gc *gc, uint32_t domid, libx
> return ERROR_FAIL;
>
> if (domtype == LIBXL_DOMAIN_TYPE_PV) {
> - if (libxl__wait_for_backend(gc, be_path, "4") < 0) {
> + if (libxl__wait_for_backend_deprecated(gc, be_path, "4", "6", (char*)0)
> + < 0) {
> LIBXL__LOG(ctx, LIBXL__LOG_DEBUG, "pci backend at %s is not ready", be_path);
> return ERROR_FAIL;
> }
> @@ -198,7 +199,8 @@ retry_transaction:
> goto retry_transaction;
>
> if (domtype == LIBXL_DOMAIN_TYPE_PV) {
> - if (libxl__wait_for_backend(gc, be_path, "4") < 0) {
> + if (libxl__wait_for_backend_deprecated(gc, be_path, "4", "6", (char*)0)
> + < 0) {
> LIBXL__LOG(ctx, LIBXL__LOG_DEBUG, "pci backend at %s is not ready", be_path);
> return ERROR_FAIL;
> }
>
[-- Attachment #2: xl-output --]
[-- Type: text/plain, Size: 22228 bytes --]
Parsing config from 5:voip.9
libxl: debug: libxl_create.c:1230:do_domain_create: ao 0x7fe37ba02850: create: how=(nil) callback=(nil) poller=0x7fe37ba03d70
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda1 spec.backend=unknown
libxl: debug: libxl_device.c:296:libxl__device_disk_set_backend: Disk vdev=xvda1, using backend phy
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda2 spec.backend=unknown
libxl: debug: libxl_device.c:296:libxl__device_disk_set_backend: Disk vdev=xvda2, using backend phy
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda3 spec.backend=unknown
libxl: debug: libxl_device.c:296:libxl__device_disk_set_backend: Disk vdev=xvda3, using backend phy
libxl: debug: libxl_create.c:675:initiate_domain_create: running bootloader
libxl: debug: libxl_bootloader.c:327:libxl__bootloader_run: no bootloader configured, using user supplied kernel
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba02bd8: deregister unregistered
libxl: debug: libxl_x86.c:82:e820_sanitize: Memory: 1048576kB End of RAM: 0x20000 (PFN) Delta: 524288kB, PCI start: 524288kB (0x20000 PFN), Balloon 0kB
libxl: debug: libxl_x86.c:201:e820_sanitize: : [0 -> 20000] RAM
libxl: debug: libxl_x86.c:201:e820_sanitize: : [20000 -> 20200] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [20200 -> 40000] Unusable
libxl: debug: libxl_x86.c:201:e820_sanitize: : [40000 -> 40200] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [40200 -> db9f0] Unusable
libxl: debug: libxl_x86.c:201:e820_sanitize: : [db9f0 -> dc0da] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dc0da -> dc1f9] ACPI NVS
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dc1f9 -> dc651] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dc651 -> dc652] Unusable
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dc652 -> dc695] ACPI NVS
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dc695 -> dcdba] Unusable
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dcdba -> dcff2] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dcff2 -> dd000] Unusable
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dd800 -> dfa00] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [f8000 -> fc000] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [fec00 -> fec01] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [fed00 -> fed04] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [fed1c -> fed20] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [fee00 -> fee01] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [fee01 -> fef00] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [ff000 -> 100000] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [100000 -> 120000] RAM
domainbuilder: detail: xc_dom_allocate: cmdline="", features="(null)"
libxl: debug: libxl_dom.c:341:libxl__build_pv: pv kernel mapped 0 path /usr/lib/xen/boot/pv-grub-x86_64.gz
domainbuilder: detail: xc_dom_kernel_file: filename="/usr/lib/xen/boot/pv-grub-x86_64.gz"
domainbuilder: detail: xc_dom_malloc_filemap : 1240 kB
domainbuilder: detail: xc_dom_malloc : 15110 kB
domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x13631e -> 0xec1ae1
domainbuilder: detail: xc_dom_ramdisk_file: filename="/etc/xen/guests/grub.d/voip.grub"
domainbuilder: detail: xc_dom_boot_xen_init: ver 4.3, caps xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
domainbuilder: detail: xc_dom_parse_image: called
domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ...
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ...
domainbuilder: detail: xc_dom_probe_bzimage_kernel: kernel is not a bzImage
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ...
domainbuilder: detail: loader probe OK
xc: detail: elf_parse_binary: phdr: paddr=0x0 memsz=0x9a96e0
xc: detail: elf_parse_binary: memory: 0x0 -> 0x9a96e0
xc: detail: elf_xen_parse: __xen_guest: "GUEST_OS=Mini-OS,XEN_VER=xen-3.0,VIRT_BASE=0x0,ELF_PADDR_OFFSET=0x0,HYPERCALL_PAGE=0x2,LOADER=generic"
xc: detail: elf_xen_parse_guest_info: GUEST_OS="Mini-OS"
xc: detail: elf_xen_parse_guest_info: XEN_VER="xen-3.0"
xc: detail: elf_xen_parse_guest_info: VIRT_BASE="0x0"
xc: detail: elf_xen_parse_guest_info: ELF_PADDR_OFFSET="0x0"
xc: detail: elf_xen_parse_guest_info: HYPERCALL_PAGE="0x2"
xc: detail: elf_xen_parse_guest_info: LOADER="generic"
xc: detail: elf_xen_addr_calc_check: addresses:
xc: detail: virt_base = 0x0
xc: detail: elf_paddr_offset = 0x0
xc: detail: virt_offset = 0x0
xc: detail: virt_kstart = 0x0
xc: detail: virt_kend = 0x9a96e0
xc: detail: virt_entry = 0x0
xc: detail: p2m_base = 0xffffffffffffffff
domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-x86_64: 0x0 -> 0x9a96e0
domainbuilder: detail: xc_dom_mem_init: mem 1024 MB, pages 0x40000 pages, 4k each
domainbuilder: detail: xc_dom_mem_init: 0x40000 pages
domainbuilder: detail: xc_dom_boot_mem_init: called
domainbuilder: detail: x86_compat: guest xen-3.0-x86_64, address size 64
domainbuilder: detail: xc_dom_malloc : 2048 kB
domainbuilder: detail: xc_dom_build_image: called
domainbuilder: detail: xc_dom_alloc_segment: kernel : 0x0 -> 0x9aa000 (pfn 0x0 + 0x9aa pages)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x0+0x9aa at 0x7fe377817000
xc: detail: elf_load_binary: phdr 0 at 0x7fe377817000 -> 0x7fe3781c06e0
domainbuilder: detail: xc_dom_alloc_segment: ramdisk : 0x9aa000 -> 0x9ab000 (pfn 0x9aa + 0x1 pages)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x9aa+0x1 at 0x7fe37b7ce000
domainbuilder: detail: xc_dom_alloc_segment: phys2mach : 0x9ab000 -> 0xbab000 (pfn 0x9ab + 0x200 pages)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x9ab+0x200 at 0x7fe377617000
domainbuilder: detail: xc_dom_alloc_page : start info : 0xbab000 (pfn 0xbab)
domainbuilder: detail: xc_dom_alloc_page : xenstore : 0xbac000 (pfn 0xbac)
domainbuilder: detail: xc_dom_alloc_page : console : 0xbad000 (pfn 0xbad)
domainbuilder: detail: nr_page_tables: 0x0000ffffffffffff/48: 0x0000000000000000 -> 0x0000ffffffffffff, 1 table(s)
domainbuilder: detail: nr_page_tables: 0x0000007fffffffff/39: 0x0000000000000000 -> 0x0000007fffffffff, 1 table(s)
domainbuilder: detail: nr_page_tables: 0x000000003fffffff/30: 0x0000000000000000 -> 0x000000003fffffff, 1 table(s)
domainbuilder: detail: nr_page_tables: 0x00000000001fffff/21: 0x0000000000000000 -> 0x0000000000ffffff, 8 table(s)
domainbuilder: detail: xc_dom_alloc_segment: page tables : 0xbae000 -> 0xbb9000 (pfn 0xbae + 0xb pages)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0xbae+0xb at 0x7fe37b680000
domainbuilder: detail: xc_dom_alloc_page : boot stack : 0xbb9000 (pfn 0xbb9)
domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0xbba000
domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0x1000000
domainbuilder: detail: xc_dom_boot_image: called
domainbuilder: detail: arch_setup_bootearly: doing nothing
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64 <= matches
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_64
domainbuilder: detail: xc_dom_update_guest_p2m: dst 64bit, pages 0x40000
domainbuilder: detail: clear_page: pfn 0xbad, mfn 0x71aabe
domainbuilder: detail: clear_page: pfn 0xbac, mfn 0x71aabf
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0xbab+0x1 at 0x7fe37b7cb000
domainbuilder: detail: start_info_x86_64: called
domainbuilder: detail: setup_hypercall_page: vaddr=0x2000 pfn=0x2
domainbuilder: detail: domain builder memory footprint
domainbuilder: detail: allocated
domainbuilder: detail: malloc : 17231 kB
domainbuilder: detail: anon mmap : 0 bytes
domainbuilder: detail: mapped
domainbuilder: detail: file mmap : 1241 kB
domainbuilder: detail: domU mmap : 11996 kB
domainbuilder: detail: arch_setup_bootlate: shared_info: pfn 0x0, mfn 0xdb70f
domainbuilder: detail: shared_info_x86_64: called
domainbuilder: detail: vcpu_x86_64: called
domainbuilder: detail: vcpu_x86_64: cr3: pfn 0xbae mfn 0x71aabd
domainbuilder: detail: launch_vm: called, ctxt=0x7fff80e7a330
domainbuilder: detail: xc_dom_release: called
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda1 spec.backend=phy
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba04098 wpath=/local/domain/0/backend/vbd/4/51713/state token=3/0: register slotnum=3
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda2 spec.backend=phy
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda2 spec.backend=phy
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba050b8 wpath=/local/domain/0/backend/vbd/4/51714/state token=2/1: register slotnum=2
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda3 spec.backend=phy
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda3 spec.backend=phy
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba048a8 wpath=/local/domain/0/backend/vbd/4/51715/state token=1/2: register slotnum=1
libxl: debug: libxl_create.c:1243:do_domain_create: ao 0x7fe37ba02850: inprogress: poller=0x7fe37ba03d70, flags=i
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba04098 wpath=/local/domain/0/backend/vbd/4/51713/state token=3/0: event epath=/local/domain/0/backend/vbd/4/51713/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51713/state wanted state 2 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba04098 wpath=/local/domain/0/backend/vbd/4/51713/state token=3/0: deregister slotnum=3
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba04098: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block add
libxl: debug: libxl_event.c:472:watchfd_callback: watch epath=/local/domain/0/backend/vbd/4/51713/state token=3/0: empty slot
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba050b8 wpath=/local/domain/0/backend/vbd/4/51714/state token=2/1: event epath=/local/domain/0/backend/vbd/4/51714/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51714/state wanted state 2 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba050b8 wpath=/local/domain/0/backend/vbd/4/51714/state token=2/1: deregister slotnum=2
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba050b8: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block add
libxl: debug: libxl_event.c:472:watchfd_callback: watch epath=/local/domain/0/backend/vbd/4/51714/state token=2/1: empty slot
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba048a8 wpath=/local/domain/0/backend/vbd/4/51715/state token=1/2: event epath=/local/domain/0/backend/vbd/4/51715/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51715/state wanted state 2 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba048a8 wpath=/local/domain/0/backend/vbd/4/51715/state token=1/2: deregister slotnum=1
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba048a8: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block add
libxl: debug: libxl_event.c:472:watchfd_callback: watch epath=/local/domain/0/backend/vbd/4/51715/state token=1/2: empty slot
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba091e8 wpath=/local/domain/0/backend/vif/4/0/state token=1/3: register slotnum=1
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba091e8 wpath=/local/domain/0/backend/vif/4/0/state token=1/3: event epath=/local/domain/0/backend/vif/4/0/state
libxl: debug: libxl_event.c:647:devstate_watch_callback: backend /local/domain/0/backend/vif/4/0/state wanted state 2 still waiting state 1
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba091e8 wpath=/local/domain/0/backend/vif/4/0/state token=1/3: event epath=/local/domain/0/backend/vif/4/0/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vif/4/0/state wanted state 2 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba091e8 wpath=/local/domain/0/backend/vif/4/0/state token=1/3: deregister slotnum=1
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba091e8: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/vif-bridge online
libxl: error: libxl_pci.c:992:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:09:02.0
libxl: debug: libxl_pci.c:81:libxl__create_pci_backend: Creating pci backend
libxl: debug: libxl_event.c:1737:libxl__ao_progress_report: ao 0x7fe37ba02850: progress report: ignored
libxl: debug: libxl_event.c:1569:libxl__ao_complete: ao 0x7fe37ba02850: complete, rc=0
libxl: debug: libxl_event.c:1541:libxl__ao__destroy: ao 0x7fe37ba02850: destroy
Waiting for domain voip (domid 4) to die [pid 2409]
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba007e0 wpath=@releaseDomain token=1/4: register slotnum=1
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba007e0 wpath=@releaseDomain token=1/4: event epath=@releaseDomain
libxl: debug: libxl.c:1000:domain_death_xswatch_callback: [evg=0x7fe37ba08900:4] from domid=4 nentries=1 rc=1
libxl: debug: libxl.c:1011:domain_death_xswatch_callback: [evg=0x7fe37ba08900:4] got=domaininfos[0] got->domain=4
libxl: debug: libxl.c:1038:domain_death_xswatch_callback: exists shutdown_reported=0 dominf.flags=ffff0020
libxl: debug: libxl.c:1004:domain_death_xswatch_callback: [evg=0] all reported
libxl: debug: libxl.c:1068:domain_death_xswatch_callback: domain death search done
<NOTE: this is after the start - no further output until shutdown -h now from ssh connection>
Domain 4 has shut down, reason code 0 0x0
Action for shutdown reason code 0 is destroy
Domain 4 needs to be cleaned up: destroying the domain
libxl: debug: libxl.c:1252:libxl_domain_destroy: ao 0x7fe37ba02850: create: how=(nil) callback=(nil) poller=0x7fe37ba03d70
libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission irq=17
<NOTE: at this point a 10s pause happens>
libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated: Backend /local/domain/0/backend/pci/4/0 not ready
libxl: debug: libxl_pci.c:204:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/4/0 is not ready
libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission irq=16
<NOTE: at this point a 10s pause happens>
libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated: Backend /local/domain/0/backend/pci/4/0 not ready
libxl: debug: libxl_pci.c:174:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/4/0 is not ready
libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission irq=18
libxl: error: libxl_pci.c:992:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:09:02.0
<NOTE: at this point a 10s pause happens>
libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated: Backend /local/domain/0/backend/pci/4/0 not ready
libxl: debug: libxl_pci.c:174:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/4/0 is not ready
libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission irq=23
<NOTE: at this point a 10s pause happens>
libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated: Backend /local/domain/0/backend/pci/4/0 not ready
libxl: debug: libxl_pci.c:174:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/4/0 is not ready
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba048a8 wpath=/local/domain/0/backend/vbd/4/51713/state token=2/5: register slotnum=2
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba091e8 wpath=/local/domain/0/backend/vbd/4/51714/state token=3/6: register slotnum=3
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba0c298 wpath=/local/domain/0/backend/vbd/4/51715/state token=0/7: register slotnum=0
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba043c8 wpath=/local/domain/0/backend/vif/4/0/state token=19/8: register slotnum=19
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7fe37ba05408 wpath=/local/domain/0/backend/pci/4/0/state token=18/9: register slotnum=18
libxl: debug: libxl.c:1261:libxl_domain_destroy: ao 0x7fe37ba02850: inprogress: poller=0x7fe37ba03d70, flags=i
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba048a8 wpath=/local/domain/0/backend/vbd/4/51713/state token=2/5: event epath=/local/domain/0/backend/vbd/4/51713/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51713/state wanted state 6 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba048a8 wpath=/local/domain/0/backend/vbd/4/51713/state token=2/5: deregister slotnum=2
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba048a8: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block remove
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba091e8 wpath=/local/domain/0/backend/vbd/4/51714/state token=3/6: event epath=/local/domain/0/backend/vbd/4/51714/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51714/state wanted state 6 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba091e8 wpath=/local/domain/0/backend/vbd/4/51714/state token=3/6: deregister slotnum=3
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba091e8: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block remove
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba0c298 wpath=/local/domain/0/backend/vbd/4/51715/state token=0/7: event epath=/local/domain/0/backend/vbd/4/51715/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51715/state wanted state 6 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba0c298 wpath=/local/domain/0/backend/vbd/4/51715/state token=0/7: deregister slotnum=0
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba0c298: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block remove
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba043c8 wpath=/local/domain/0/backend/vif/4/0/state token=19/8: event epath=/local/domain/0/backend/vif/4/0/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vif/4/0/state wanted state 6 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba043c8 wpath=/local/domain/0/backend/vif/4/0/state token=19/8: deregister slotnum=19
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba043c8: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/vif-bridge offline
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7fe37ba05408 wpath=/local/domain/0/backend/pci/4/0/state token=18/9: event epath=/local/domain/0/backend/pci/4/0/state
libxl: debug: libxl_event.c:647:devstate_watch_callback: backend /local/domain/0/backend/pci/4/0/state wanted state 6 still waiting state 5
<NOTE1: the above line is new compared to the old output file>
<NOTE2: at this point a new 10s delay happens - which is only visible here as the prompt in dom0 has already returned>
<NOTE3: the next line differs from the old output file>
libxl: debug: libxl_event.c:661:devstate_timeout: backend /local/domain/0/backend/pci/4/0/state wanted state 6 timed out
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba05408 wpath=/local/domain/0/backend/pci/4/0/state token=18/9: deregister slotnum=18
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7fe37ba05408: deregister unregistered
<NOTE: the following 2 lines are new compared to the old output file>
libxl: error: libxl_device.c:894:device_backend_callback: unable to remove device with path /local/domain/0/backend/pci/4/0
libxl: error: libxl.c:1452:devices_destroy_cb: libxl__devices_destroy failed for 4
libxl: debug: libxl_event.c:1569:libxl__ao_complete: ao 0x7fe37ba02850: complete, rc=0
libxl: debug: libxl_event.c:1541:libxl__ao__destroy: ao 0x7fe37ba02850: destroy
Done. Exiting now
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7fe37ba007e0 wpath=@releaseDomain token=1/4: deregister slotnum=1
xc: debug: hypercall buffer: total allocations:554 total releases:554
xc: debug: hypercall buffer: current allocations:0 maximum allocations:2
xc: debug: hypercall buffer: cache current size:2
xc: debug: hypercall buffer: cache hits:548 misses:2 toobig:4
[-- Attachment #3: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-20 13:53 ` Pasi Kärkkäinen
2014-03-20 15:28 ` Ian Jackson
@ 2014-03-20 19:34 ` Atom2
1 sibling, 0 replies; 23+ messages in thread
From: Atom2 @ 2014-03-20 19:34 UTC (permalink / raw)
To: Pasi Kärkkäinen, Ian Jackson
Cc: Roger Pau Monne, xen-devel, Ian Campbell, xen-users
Am 20.03.14 14:53, schrieb Pasi Kärkkäinen:
> On Thu, Mar 20, 2014 at 11:52:57AM +0000, Ian Jackson wrote:
>> Atom2 writes ("Re: [Xen-devel] [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough"):
>>> the patch unfortunately doesn't apply to my sources - some comments to
>>> the reasons why further below.
>>
>> Here's a backport. I have compiled but not executed it.
>>
>>> Sorry for my delay in answering - this is a resend as the first e-Mail
>>> with uncompressed attachments did not go through.
>>
>>> Just FYI: the version I am using is 4.3.1-r5; I have attached the
>>> relevant source files referred to by your patches.
>>
>> Thanks, but our revision control system enables us to retrieve old
>> versions very easily :-). So there is not any need to provide us with
>> these files.
>>
>> Having said that, I have no record of 4.3.1-rc5. But I'm pretty sure
>> the patch below, which is against staging-4.3, will apply to your
>> tree. It applies cleanly to 4.3.0-rc5 and 4.3.1-rc2, which are my two
>> guesses as to which version you mean.
>>
>
> Often "r5" style versions refer to Gentoo packaging..
> So maybe he actually didn't have a typo in 4.3.1-r5 :)
Pasi - you were spot on: I use gentoo and the version number was not a typo.
>
> -- Pasi
>
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-20 19:32 ` Atom2
@ 2014-03-21 18:11 ` Ian Jackson
2014-03-21 19:39 ` Atom2
0 siblings, 1 reply; 23+ messages in thread
From: Ian Jackson @ 2014-03-21 18:11 UTC (permalink / raw)
To: Atom2; +Cc: xen-users, xen-devel, Ian Campbell, Roger Pau Monne
Atom2 writes ("Re: [Xen-devel] [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough"):
> This file again contains a few annotations with regards to where the
> delays happen. To my untrained eye it looks largely identical to the
> last xl-output with the obvious change of domain-id, addresses,
> line-numbers for debug output where changes in the sourve have happende
> and the use of the new function libxl__wait_for_backend_deprecated
> instead of libxl__wait_for_backend due to your patch, the latter of
> which I take as proof that your patches have been applied.
Thanks. I'm puzzled now. I have another tiny patch to suggest,
which won't fix the problem but will produce more debugging output.
Can you run it again with this, on top of the previous patch, please ?
Thanks,
Ian.
diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c
index 779b38b..eff452b 100644
--- a/tools/libxl/libxl_device.c
+++ b/tools/libxl/libxl_device.c
@@ -1131,7 +1131,13 @@ int libxl__wait_for_backend_deprecated(libxl__gc *gc, const char *be_path,
watchdog--;
}
}
- LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "Backend %s not ready", be_path);
+ LOG(ERROR, "Backend %s not ready (state %s)", be_path, p);
+{
+const char *fe = libxl__xs_read(gc,0, GCSPRINTF("%s/frontend", be_path));
+const char *fe_state = !fe ? 0 : libxl__xs_read(gc,0, GCSPRINTF("%s/state", fe));
+LOG(ERROR, "FE %s state %s", fe, fe_state);
+}
+
out:
return rc;
}
^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-21 18:11 ` Ian Jackson
@ 2014-03-21 19:39 ` Atom2
2014-04-02 14:44 ` Ian Jackson
0 siblings, 1 reply; 23+ messages in thread
From: Atom2 @ 2014-03-21 19:39 UTC (permalink / raw)
To: Ian Jackson; +Cc: xen-users, xen-devel, Ian Campbell, Roger Pau Monne
[-- Attachment #1: Type: text/plain, Size: 1138 bytes --]
Am 21.03.14 19:11, schrieb Ian Jackson:
> Thanks. I'm puzzled now. I have another tiny patch to suggest,
> which won't fix the problem but will produce more debugging output.
Many thanks for the new patch.
>
> Can you run it again with this, on top of the previous patch, please ?
Sure, the new output of xl -vvv create -F domain is again attached to
this e-Mail.
Thanks again,
Atom2
>
> Thanks,
> Ian.
>
> diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c
> index 779b38b..eff452b 100644
> --- a/tools/libxl/libxl_device.c
> +++ b/tools/libxl/libxl_device.c
> @@ -1131,7 +1131,13 @@ int libxl__wait_for_backend_deprecated(libxl__gc *gc, const char *be_path,
> watchdog--;
> }
> }
> - LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "Backend %s not ready", be_path);
> + LOG(ERROR, "Backend %s not ready (state %s)", be_path, p);
> +{
> +const char *fe = libxl__xs_read(gc,0, GCSPRINTF("%s/frontend", be_path));
> +const char *fe_state = !fe ? 0 : libxl__xs_read(gc,0, GCSPRINTF("%s/state", fe));
> +LOG(ERROR, "FE %s state %s", fe, fe_state);
> +}
> +
> out:
> return rc;
> }
>
>
[-- Attachment #2: xl-output --]
[-- Type: text/plain, Size: 22450 bytes --]
libxl: debug: libxl_create.c:1230:do_domain_create: ao 0x7feea6a15850: create: how=(nil) callback=(nil) poller=0x7feea6a16d70
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda1 spec.backend=unknown
libxl: debug: libxl_device.c:296:libxl__device_disk_set_backend: Disk vdev=xvda1, using backend phy
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda2 spec.backend=unknown
libxl: debug: libxl_device.c:296:libxl__device_disk_set_backend: Disk vdev=xvda2, using backend phy
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda3 spec.backend=unknown
libxl: debug: libxl_device.c:296:libxl__device_disk_set_backend: Disk vdev=xvda3, using backend phy
libxl: debug: libxl_create.c:675:initiate_domain_create: running bootloader
libxl: debug: libxl_bootloader.c:327:libxl__bootloader_run: no bootloader configured, using user supplied kernel
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7feea6a15bd8: deregister unregistered
libxl: debug: libxl_x86.c:82:e820_sanitize: Memory: 1048576kB End of RAM: 0x20000 (PFN) Delta: 524288kB, PCI start: 524288kB (0x20000 PFN), Balloon 0kB
libxl: debug: libxl_x86.c:201:e820_sanitize: : [0 -> 20000] RAM
libxl: debug: libxl_x86.c:201:e820_sanitize: : [20000 -> 20200] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [20200 -> 40000] Unusable
libxl: debug: libxl_x86.c:201:e820_sanitize: : [40000 -> 40200] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [40200 -> db9f0] Unusable
libxl: debug: libxl_x86.c:201:e820_sanitize: : [db9f0 -> dc0da] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dc0da -> dc1f9] ACPI NVS
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dc1f9 -> dc651] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dc651 -> dc652] Unusable
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dc652 -> dc695] ACPI NVS
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dc695 -> dcdba] Unusable
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dcdba -> dcff2] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dcff2 -> dd000] Unusable
libxl: debug: libxl_x86.c:201:e820_sanitize: : [dd800 -> dfa00] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [f8000 -> fc000] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [fec00 -> fec01] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [fed00 -> fed04] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [fed1c -> fed20] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [fee00 -> fee01] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [fee01 -> fef00] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [ff000 -> 100000] Reserved
libxl: debug: libxl_x86.c:201:e820_sanitize: : [100000 -> 120000] RAM
domainbuilder: detail: xc_dom_allocate: cmdline="", features="(null)"
libxl: debug: libxl_dom.c:341:libxl__build_pv: pv kernel mapped 0 path /usr/lib/xen/boot/pv-grub-x86_64.gz
domainbuilder: detail: xc_dom_kernel_file: filename="/usr/lib/xen/boot/pv-grub-x86_64.gz"
domainbuilder: detail: xc_dom_malloc_filemap : 1240 kB
domainbuilder: detail: xc_dom_malloc : 15110 kB
domainbuilder: detail: xc_dom_do_gunzip: unzip ok, 0x13631e -> 0xec1ae1
domainbuilder: detail: xc_dom_ramdisk_file: filename="/etc/xen/guests/grub.d/voip.grub"
domainbuilder: detail: xc_dom_boot_xen_init: ver 4.3, caps xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
domainbuilder: detail: xc_dom_parse_image: called
domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ...
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying Linux bzImage loader ...
domainbuilder: detail: xc_dom_probe_bzimage_kernel: kernel is not a bzImage
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying ELF-generic loader ...
domainbuilder: detail: loader probe OK
xc: detail: elf_parse_binary: phdr: paddr=0x0 memsz=0x9a96e0
xc: detail: elf_parse_binary: memory: 0x0 -> 0x9a96e0
xc: detail: elf_xen_parse: __xen_guest: "GUEST_OS=Mini-OS,XEN_VER=xen-3.0,VIRT_BASE=0x0,ELF_PADDR_OFFSET=0x0,HYPERCALL_PAGE=0x2,LOADER=generic"
xc: detail: elf_xen_parse_guest_info: GUEST_OS="Mini-OS"
xc: detail: elf_xen_parse_guest_info: XEN_VER="xen-3.0"
xc: detail: elf_xen_parse_guest_info: VIRT_BASE="0x0"
xc: detail: elf_xen_parse_guest_info: ELF_PADDR_OFFSET="0x0"
xc: detail: elf_xen_parse_guest_info: HYPERCALL_PAGE="0x2"
xc: detail: elf_xen_parse_guest_info: LOADER="generic"
xc: detail: elf_xen_addr_calc_check: addresses:
xc: detail: virt_base = 0x0
xc: detail: elf_paddr_offset = 0x0
xc: detail: virt_offset = 0x0
xc: detail: virt_kstart = 0x0
xc: detail: virt_kend = 0x9a96e0
xc: detail: virt_entry = 0x0
xc: detail: p2m_base = 0xffffffffffffffff
domainbuilder: detail: xc_dom_parse_elf_kernel: xen-3.0-x86_64: 0x0 -> 0x9a96e0
domainbuilder: detail: xc_dom_mem_init: mem 1024 MB, pages 0x40000 pages, 4k each
domainbuilder: detail: xc_dom_mem_init: 0x40000 pages
domainbuilder: detail: xc_dom_boot_mem_init: called
domainbuilder: detail: x86_compat: guest xen-3.0-x86_64, address size 64
domainbuilder: detail: xc_dom_malloc : 2048 kB
domainbuilder: detail: xc_dom_build_image: called
domainbuilder: detail: xc_dom_alloc_segment: kernel : 0x0 -> 0x9aa000 (pfn 0x0 + 0x9aa pages)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x0+0x9aa at 0x7feea282a000
xc: detail: elf_load_binary: phdr 0 at 0x7feea282a000 -> 0x7feea31d36e0
domainbuilder: detail: xc_dom_alloc_segment: ramdisk : 0x9aa000 -> 0x9ab000 (pfn 0x9aa + 0x1 pages)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x9aa+0x1 at 0x7feea67e1000
domainbuilder: detail: xc_dom_alloc_segment: phys2mach : 0x9ab000 -> 0xbab000 (pfn 0x9ab + 0x200 pages)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x9ab+0x200 at 0x7feea262a000
domainbuilder: detail: xc_dom_alloc_page : start info : 0xbab000 (pfn 0xbab)
domainbuilder: detail: xc_dom_alloc_page : xenstore : 0xbac000 (pfn 0xbac)
domainbuilder: detail: xc_dom_alloc_page : console : 0xbad000 (pfn 0xbad)
domainbuilder: detail: nr_page_tables: 0x0000ffffffffffff/48: 0x0000000000000000 -> 0x0000ffffffffffff, 1 table(s)
domainbuilder: detail: nr_page_tables: 0x0000007fffffffff/39: 0x0000000000000000 -> 0x0000007fffffffff, 1 table(s)
domainbuilder: detail: nr_page_tables: 0x000000003fffffff/30: 0x0000000000000000 -> 0x000000003fffffff, 1 table(s)
domainbuilder: detail: nr_page_tables: 0x00000000001fffff/21: 0x0000000000000000 -> 0x0000000000ffffff, 8 table(s)
domainbuilder: detail: xc_dom_alloc_segment: page tables : 0xbae000 -> 0xbb9000 (pfn 0xbae + 0xb pages)
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0xbae+0xb at 0x7feea6693000
domainbuilder: detail: xc_dom_alloc_page : boot stack : 0xbb9000 (pfn 0xbb9)
domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0xbba000
domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0x1000000
domainbuilder: detail: xc_dom_boot_image: called
domainbuilder: detail: arch_setup_bootearly: doing nothing
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_64 <= matches
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-x86_32p
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_32p
domainbuilder: detail: xc_dom_compat_check: supported guest type: hvm-3.0-x86_64
domainbuilder: detail: xc_dom_update_guest_p2m: dst 64bit, pages 0x40000
domainbuilder: detail: clear_page: pfn 0xbad, mfn 0x7533d8
domainbuilder: detail: clear_page: pfn 0xbac, mfn 0x7533d9
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0xbab+0x1 at 0x7feea67de000
domainbuilder: detail: start_info_x86_64: called
domainbuilder: detail: setup_hypercall_page: vaddr=0x2000 pfn=0x2
domainbuilder: detail: domain builder memory footprint
domainbuilder: detail: allocated
domainbuilder: detail: malloc : 17231 kB
domainbuilder: detail: anon mmap : 0 bytes
domainbuilder: detail: mapped
domainbuilder: detail: file mmap : 1241 kB
domainbuilder: detail: domU mmap : 11996 kB
domainbuilder: detail: arch_setup_bootlate: shared_info: pfn 0x0, mfn 0xdb9e8
domainbuilder: detail: shared_info_x86_64: called
domainbuilder: detail: vcpu_x86_64: called
domainbuilder: detail: vcpu_x86_64: cr3: pfn 0xbae mfn 0x7533d7
domainbuilder: detail: launch_vm: called, ctxt=0x7fffadabe320
domainbuilder: detail: xc_dom_release: called
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda1 spec.backend=phy
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7feea6a17098 wpath=/local/domain/0/backend/vbd/4/51713/state token=3/0: register slotnum=3
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda2 spec.backend=phy
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda2 spec.backend=phy
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7feea6a180b8 wpath=/local/domain/0/backend/vbd/4/51714/state token=2/1: register slotnum=2
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda3 spec.backend=phy
libxl: debug: libxl_device.c:257:libxl__device_disk_set_backend: Disk vdev=xvda3 spec.backend=phy
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7feea6a178a8 wpath=/local/domain/0/backend/vbd/4/51715/state token=1/2: register slotnum=1
libxl: debug: libxl_create.c:1243:do_domain_create: ao 0x7feea6a15850: inprogress: poller=0x7feea6a16d70, flags=i
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7feea6a17098 wpath=/local/domain/0/backend/vbd/4/51713/state token=3/0: event epath=/local/domain/0/backend/vbd/4/51713/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51713/state wanted state 2 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7feea6a17098 wpath=/local/domain/0/backend/vbd/4/51713/state token=3/0: deregister slotnum=3
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7feea6a17098: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block add
libxl: debug: libxl_event.c:472:watchfd_callback: watch epath=/local/domain/0/backend/vbd/4/51713/state token=3/0: empty slot
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7feea6a180b8 wpath=/local/domain/0/backend/vbd/4/51714/state token=2/1: event epath=/local/domain/0/backend/vbd/4/51714/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51714/state wanted state 2 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7feea6a180b8 wpath=/local/domain/0/backend/vbd/4/51714/state token=2/1: deregister slotnum=2
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7feea6a180b8: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block add
libxl: debug: libxl_event.c:472:watchfd_callback: watch epath=/local/domain/0/backend/vbd/4/51714/state token=2/1: empty slot
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7feea6a178a8 wpath=/local/domain/0/backend/vbd/4/51715/state token=1/2: event epath=/local/domain/0/backend/vbd/4/51715/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51715/state wanted state 2 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7feea6a178a8 wpath=/local/domain/0/backend/vbd/4/51715/state token=1/2: deregister slotnum=1
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7feea6a178a8: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block add
libxl: debug: libxl_event.c:472:watchfd_callback: watch epath=/local/domain/0/backend/vbd/4/51715/state token=1/2: empty slot
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7feea6a1c1e8 wpath=/local/domain/0/backend/vif/4/0/state token=1/3: register slotnum=1
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7feea6a1c1e8 wpath=/local/domain/0/backend/vif/4/0/state token=1/3: event epath=/local/domain/0/backend/vif/4/0/state
libxl: debug: libxl_event.c:647:devstate_watch_callback: backend /local/domain/0/backend/vif/4/0/state wanted state 2 still waiting state 1
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7feea6a1c1e8 wpath=/local/domain/0/backend/vif/4/0/state token=1/3: event epath=/local/domain/0/backend/vif/4/0/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vif/4/0/state wanted state 2 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7feea6a1c1e8 wpath=/local/domain/0/backend/vif/4/0/state token=1/3: deregister slotnum=1
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7feea6a1c1e8: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/vif-bridge online
libxl: error: libxl_pci.c:992:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:09:02.0
libxl: debug: libxl_pci.c:81:libxl__create_pci_backend: Creating pci backend
libxl: debug: libxl_event.c:1737:libxl__ao_progress_report: ao 0x7feea6a15850: progress report: ignored
libxl: debug: libxl_event.c:1569:libxl__ao_complete: ao 0x7feea6a15850: complete, rc=0
libxl: debug: libxl_event.c:1541:libxl__ao__destroy: ao 0x7feea6a15850: destroy
Waiting for domain voip (domid 4) to die [pid 7532]
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7feea6a137e0 wpath=@releaseDomain token=1/4: register slotnum=1
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7feea6a137e0 wpath=@releaseDomain token=1/4: event epath=@releaseDomain
libxl: debug: libxl.c:1000:domain_death_xswatch_callback: [evg=0x7feea6a1b900:4] from domid=4 nentries=1 rc=1
libxl: debug: libxl.c:1011:domain_death_xswatch_callback: [evg=0x7feea6a1b900:4] got=domaininfos[0] got->domain=4
libxl: debug: libxl.c:1038:domain_death_xswatch_callback: exists shutdown_reported=0 dominf.flags=ffff0020
libxl: debug: libxl.c:1004:domain_death_xswatch_callback: [evg=0] all reported
libxl: debug: libxl.c:1068:domain_death_xswatch_callback: domain death search done
<NOTE: this is after the start - no further output until shutdown -h now from ssh connection>
Domain 4 has shut down, reason code 0 0x0
Action for shutdown reason code 0 is destroy
Domain 4 needs to be cleaned up: destroying the domain
libxl: debug: libxl.c:1252:libxl_domain_destroy: ao 0x7feea6a15850: create: how=(nil) callback=(nil) poller=0x7feea6a16d70
libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission irq=17
<NOTE: at this point a 10s pause happens>
libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated: Backend /local/domain/0/backend/pci/4/0 not ready (state 7)
libxl: error: libxl_device.c:1138:libxl__wait_for_backend_deprecated: FE /local/domain/4/device/pci/0 state 6
libxl: debug: libxl_pci.c:204:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/4/0 is not ready
libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission irq=16
<NOTE: at this point a 10s pause happens>
libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated: Backend /local/domain/0/backend/pci/4/0 not ready (state 7)
libxl: error: libxl_device.c:1138:libxl__wait_for_backend_deprecated: FE /local/domain/4/device/pci/0 state 6
libxl: debug: libxl_pci.c:174:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/4/0 is not ready
libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission irq=18
libxl: error: libxl_pci.c:992:libxl__device_pci_reset: The kernel doesn't support reset from sysfs for PCI device 0000:09:02.0
<NOTE: at this point a 10s pause happens>
libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated: Backend /local/domain/0/backend/pci/4/0 not ready (state 7)
libxl: error: libxl_device.c:1138:libxl__wait_for_backend_deprecated: FE /local/domain/4/device/pci/0 state 6
libxl: debug: libxl_pci.c:174:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/4/0 is not ready
libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission irq=23
<NOTE: at this point a 10s pause happens>
libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated: Backend /local/domain/0/backend/pci/4/0 not ready (state 7)
libxl: error: libxl_device.c:1138:libxl__wait_for_backend_deprecated: FE /local/domain/4/device/pci/0 state 6
libxl: debug: libxl_pci.c:174:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/4/0 is not ready
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7feea6a178a8 wpath=/local/domain/0/backend/vbd/4/51713/state token=2/5: register slotnum=2
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7feea6a1c1e8 wpath=/local/domain/0/backend/vbd/4/51714/state token=3/6: register slotnum=3
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7feea6a1f3d8 wpath=/local/domain/0/backend/vbd/4/51715/state token=0/7: register slotnum=0
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7feea6a173c8 wpath=/local/domain/0/backend/vif/4/0/state token=19/8: register slotnum=19
libxl: debug: libxl_event.c:559:libxl__ev_xswatch_register: watch w=0x7feea6a18408 wpath=/local/domain/0/backend/pci/4/0/state token=18/9: register slotnum=18
libxl: debug: libxl.c:1261:libxl_domain_destroy: ao 0x7feea6a15850: inprogress: poller=0x7feea6a16d70, flags=i
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7feea6a178a8 wpath=/local/domain/0/backend/vbd/4/51713/state token=2/5: event epath=/local/domain/0/backend/vbd/4/51713/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51713/state wanted state 6 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7feea6a178a8 wpath=/local/domain/0/backend/vbd/4/51713/state token=2/5: deregister slotnum=2
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7feea6a178a8: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block remove
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7feea6a1c1e8 wpath=/local/domain/0/backend/vbd/4/51714/state token=3/6: event epath=/local/domain/0/backend/vbd/4/51714/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51714/state wanted state 6 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7feea6a1c1e8 wpath=/local/domain/0/backend/vbd/4/51714/state token=3/6: deregister slotnum=3
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7feea6a1c1e8: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block remove
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7feea6a1f3d8 wpath=/local/domain/0/backend/vbd/4/51715/state token=0/7: event epath=/local/domain/0/backend/vbd/4/51715/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vbd/4/51715/state wanted state 6 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7feea6a1f3d8 wpath=/local/domain/0/backend/vbd/4/51715/state token=0/7: deregister slotnum=0
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7feea6a1f3d8: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/block remove
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7feea6a173c8 wpath=/local/domain/0/backend/vif/4/0/state token=19/8: event epath=/local/domain/0/backend/vif/4/0/state
libxl: debug: libxl_event.c:643:devstate_watch_callback: backend /local/domain/0/backend/vif/4/0/state wanted state 6 ok
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7feea6a173c8 wpath=/local/domain/0/backend/vif/4/0/state token=19/8: deregister slotnum=19
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7feea6a173c8: deregister unregistered
libxl: debug: libxl_device.c:959:device_hotplug: calling hotplug script: /etc/xen/scripts/vif-bridge offline
libxl: debug: libxl_event.c:503:watchfd_callback: watch w=0x7feea6a18408 wpath=/local/domain/0/backend/pci/4/0/state token=18/9: event epath=/local/domain/0/backend/pci/4/0/state
libxl: debug: libxl_event.c:647:devstate_watch_callback: backend /local/domain/0/backend/pci/4/0/state wanted state 6 still waiting state 5
<NOTE: at this point a 10s pause happens - but prompt in shutdown shell is back>
libxl: debug: libxl_event.c:661:devstate_timeout: backend /local/domain/0/backend/pci/4/0/state wanted state 6 timed out
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7feea6a18408 wpath=/local/domain/0/backend/pci/4/0/state token=18/9: deregister slotnum=18
libxl: debug: libxl_event.c:608:libxl__ev_xswatch_deregister: watch w=0x7feea6a18408: deregister unregistered
libxl: error: libxl_device.c:894:device_backend_callback: unable to remove device with path /local/domain/0/backend/pci/4/0
libxl: error: libxl.c:1452:devices_destroy_cb: libxl__devices_destroy failed for 4
libxl: debug: libxl_event.c:1569:libxl__ao_complete: ao 0x7feea6a15850: complete, rc=0
libxl: debug: libxl_event.c:1541:libxl__ao__destroy: ao 0x7feea6a15850: destroy
Done. Exiting now
libxl: debug: libxl_event.c:596:libxl__ev_xswatch_deregister: watch w=0x7feea6a137e0 wpath=@releaseDomain token=1/4: deregister slotnum=1
xc: debug: hypercall buffer: total allocations:555 total releases:555
xc: debug: hypercall buffer: current allocations:0 maximum allocations:2
xc: debug: hypercall buffer: cache current size:2
xc: debug: hypercall buffer: cache hits:549 misses:2 toobig:4
[-- Attachment #3: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-03-21 19:39 ` Atom2
@ 2014-04-02 14:44 ` Ian Jackson
2014-04-02 15:17 ` Atom2
0 siblings, 1 reply; 23+ messages in thread
From: Ian Jackson @ 2014-04-02 14:44 UTC (permalink / raw)
To: Atom2, Konrad Rzeszutek Wilk, Boris Ostrovsky, David Vrabel
Cc: xen-users, xen-devel, Ian Campbell, Roger Pau Monne
Atom2 writes ("Re: [Xen-devel] [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough"):
> Am 21.03.14 19:11, schrieb Ian Jackson:
> > Can you run it again with this, on top of the previous patch, please ?
>
> Sure, the new output of xl -vvv create -F domain is again attached to
> this e-Mail.
Sorry for the delay replying. I have been ill :-(.
> <NOTE: at this point a 10s pause happens>
> libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated: Backend /local/domain/0/backend/pci/4/0 not ready (state 7)
> libxl: error: libxl_device.c:1138:libxl__wait_for_backend_deprecated: FE /local/domain/4/device/pci/0 state 6
> libxl: debug: libxl_pci.c:204:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/4/0 is not ready
> libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission irq=16
So the backend here is in state 7 (Reconfiguring), but the frontend is
in state 6 (Closed). I think this is a bug in pciback.
I looked at drivers/xen/xen-pciback/xenbus.c in Linux 3.13 and found
xen_pcibk_frontend_changed which seems to do roughly what I would
expect.
Has this changed at some point ?
Atom, what kernel are you using ?
Thanks,
Ian.
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-04-02 14:44 ` Ian Jackson
@ 2014-04-02 15:17 ` Atom2
2014-04-18 21:47 ` Atom2
0 siblings, 1 reply; 23+ messages in thread
From: Atom2 @ 2014-04-02 15:17 UTC (permalink / raw)
To: xen-devel
Am 02.04.14 16:44, schrieb Ian Jackson:
> Atom2 writes ("Re: [Xen-devel] [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough"):
>> Am 21.03.14 19:11, schrieb Ian Jackson:
>>> Can you run it again with this, on top of the previous patch, please ?
>>
>> Sure, the new output of xl -vvv create -F domain is again attached to
>> this e-Mail.
>
> Sorry for the delay replying. I have been ill :-(.
Sorry to hear that. Though I noticed your absence from the list I simply
assumed that you were off on vacation. In any case good to see you back.
>
>> <NOTE: at this point a 10s pause happens>
>> libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated: Backend /local/domain/0/backend/pci/4/0 not ready (state 7)
>> libxl: error: libxl_device.c:1138:libxl__wait_for_backend_deprecated: FE /local/domain/4/device/pci/0 state 6
>> libxl: debug: libxl_pci.c:204:libxl__device_pci_remove_xenstore: pci backend at /local/domain/0/backend/pci/4/0 is not ready
>> libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission irq=16
>
> So the backend here is in state 7 (Reconfiguring), but the frontend is
> in state 6 (Closed). I think this is a bug in pciback.
>
> I looked at drivers/xen/xen-pciback/xenbus.c in Linux 3.13 and found
> xen_pcibk_frontend_changed which seems to do roughly what I would
> expect.
>
> Has this changed at some point ?
>
> Atom, what kernel are you using ?
All the error messages stem from kernel 3.11.7. In the meantime 3.13.2
became stable for gentoo and I installed that a few days ago. I have not
run the debug output yet or timed the shutdown process, but there's
still a delay with that kernel and it feels as long as before. I you
want, I can clearly provide new debug output or timing information.
Thanks Atom2
>
> Thanks,
> Ian.
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-04-02 15:17 ` Atom2
@ 2014-04-18 21:47 ` Atom2
2014-04-19 0:12 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 23+ messages in thread
From: Atom2 @ 2014-04-18 21:47 UTC (permalink / raw)
To: xen-devel
This is just a (very) gentle ping ... or have I missed out on a reply?
Thanks Atom2
Am 02.04.14 17:17, schrieb Atom2:
>
> Am 02.04.14 16:44, schrieb Ian Jackson:
>> Atom2 writes ("Re: [Xen-devel] [Xen-users] substantial shutdown delay
>> for PV guests with PCI -passthrough"):
>>> Am 21.03.14 19:11, schrieb Ian Jackson:
>>>> Can you run it again with this, on top of the previous patch, please ?
>>>
>>> Sure, the new output of xl -vvv create -F domain is again attached to
>>> this e-Mail.
>>
>> Sorry for the delay replying. I have been ill :-(.
> Sorry to hear that. Though I noticed your absence from the list I simply
> assumed that you were off on vacation. In any case good to see you back.
>>
>>> <NOTE: at this point a 10s pause happens>
>>> libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated:
>>> Backend /local/domain/0/backend/pci/4/0 not ready (state 7)
>>> libxl: error: libxl_device.c:1138:libxl__wait_for_backend_deprecated:
>>> FE /local/domain/4/device/pci/0 state 6
>>> libxl: debug: libxl_pci.c:204:libxl__device_pci_remove_xenstore: pci
>>> backend at /local/domain/0/backend/pci/4/0 is not ready
>>> libxl: error: libxl_pci.c:1250:do_pci_remove:
>>> xc_domain_irq_permission irq=16
>>
>> So the backend here is in state 7 (Reconfiguring), but the frontend is
>> in state 6 (Closed). I think this is a bug in pciback.
>>
>> I looked at drivers/xen/xen-pciback/xenbus.c in Linux 3.13 and found
>> xen_pcibk_frontend_changed which seems to do roughly what I would
>> expect.
>>
>> Has this changed at some point ?
>>
>> Atom, what kernel are you using ?
> All the error messages stem from kernel 3.11.7. In the meantime 3.13.2
> became stable for gentoo and I installed that a few days ago. I have not
> run the debug output yet or timed the shutdown process, but there's
> still a delay with that kernel and it feels as long as before. I you
> want, I can clearly provide new debug output or timing information.
>
> Thanks Atom2
>>
>> Thanks,
>> Ian.
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel
>>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-04-18 21:47 ` Atom2
@ 2014-04-19 0:12 ` Konrad Rzeszutek Wilk
2014-04-19 18:59 ` Atom2
0 siblings, 1 reply; 23+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-04-19 0:12 UTC (permalink / raw)
To: Atom2; +Cc: xen-devel
On Fri, Apr 18, 2014 at 11:47:46PM +0200, Atom2 wrote:
> This is just a (very) gentle ping ... or have I missed out on a reply?
I ran an PV guest with PCI passthrough this week and it had no trouble -
didn't see 10 seconds or so. But I did the shutdown from within the
guest (poweroff).
What is the kernel you are running as your dom0? Is it the same as
frontend?
>
> Thanks Atom2
>
> Am 02.04.14 17:17, schrieb Atom2:
> >
> >Am 02.04.14 16:44, schrieb Ian Jackson:
> >>Atom2 writes ("Re: [Xen-devel] [Xen-users] substantial shutdown delay
> >>for PV guests with PCI -passthrough"):
> >>>Am 21.03.14 19:11, schrieb Ian Jackson:
> >>>>Can you run it again with this, on top of the previous patch, please ?
> >>>
> >>>Sure, the new output of xl -vvv create -F domain is again attached to
> >>>this e-Mail.
> >>
> >>Sorry for the delay replying. I have been ill :-(.
> >Sorry to hear that. Though I noticed your absence from the list I simply
> >assumed that you were off on vacation. In any case good to see you back.
> >>
> >>><NOTE: at this point a 10s pause happens>
> >>>libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated:
> >>>Backend /local/domain/0/backend/pci/4/0 not ready (state 7)
> >>>libxl: error: libxl_device.c:1138:libxl__wait_for_backend_deprecated:
> >>>FE /local/domain/4/device/pci/0 state 6
> >>>libxl: debug: libxl_pci.c:204:libxl__device_pci_remove_xenstore: pci
> >>>backend at /local/domain/0/backend/pci/4/0 is not ready
> >>>libxl: error: libxl_pci.c:1250:do_pci_remove:
> >>>xc_domain_irq_permission irq=16
> >>
> >>So the backend here is in state 7 (Reconfiguring), but the frontend is
> >>in state 6 (Closed). I think this is a bug in pciback.
> >>
> >>I looked at drivers/xen/xen-pciback/xenbus.c in Linux 3.13 and found
> >>xen_pcibk_frontend_changed which seems to do roughly what I would
> >>expect.
> >>
> >>Has this changed at some point ?
> >>
> >>Atom, what kernel are you using ?
> >All the error messages stem from kernel 3.11.7. In the meantime 3.13.2
> >became stable for gentoo and I installed that a few days ago. I have not
> >run the debug output yet or timed the shutdown process, but there's
> >still a delay with that kernel and it feels as long as before. I you
> >want, I can clearly provide new debug output or timing information.
> >
> >Thanks Atom2
> >>
> >>Thanks,
> >>Ian.
> >>
> >>_______________________________________________
> >>Xen-devel mailing list
> >>Xen-devel@lists.xen.org
> >>http://lists.xen.org/xen-devel
> >>
> >
> >_______________________________________________
> >Xen-devel mailing list
> >Xen-devel@lists.xen.org
> >http://lists.xen.org/xen-devel
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-04-19 0:12 ` Konrad Rzeszutek Wilk
@ 2014-04-19 18:59 ` Atom2
2014-04-22 10:44 ` George Dunlap
0 siblings, 1 reply; 23+ messages in thread
From: Atom2 @ 2014-04-19 18:59 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk, Ian Jackson, boris.ostrovsky, david.vrabel
Cc: Roger Pau Monne, xen-users, Ian Campbell, xen-devel
Hi Konrad,
thanks for your quick reply. I have re-added the other recipients that
were in the list prior to my reply from 2 April as I just saw that I
somehow have managed to drop those guys - which might also explain their
silence to my reply.
All: sorry for dropping you from my earlier reply. For your convenience
I have added my reply from 2 April at the end of this mail.
Am 19.04.14 02:12, schrieb Konrad Rzeszutek Wilk:
> On Fri, Apr 18, 2014 at 11:47:46PM +0200, Atom2 wrote:
>> This is just a (very) gentle ping ... or have I missed out on a reply?
>
> I ran an PV guest with PCI passthrough this week and it had no trouble -
> didn't see 10 seconds or so. But I did the shutdown from within the
> guest (poweroff).
For me it makes no difference timewise whether I issue a
xl shutdown guest
from dom0 or whether I issue
shutdown -h now
from a connection (i.e. ssh or screen or console) to the guest. The main
difference being that for the latter the delay is visible whereas for
the former, the delay is not so obvious because 'xl shutdown guest' from
dom0 due to its asynchronous nature returns immediately even when the
guest is still alive.
One difference that I have noticed however is that for the shutdown from
_within_ the guest (i.e. shutdown -h now) the state of the guest remains
's' in 'xl list' from the time the "system halted" message appears on
screen until the prompt returns in dom0 whereas for a shutdown from dom0
with 'xl shutdown guest' the state changes from 's' to 'ps' for a number
of seconds before it is finally gone.
>
> What is the kernel you are running as your dom0? Is it the same as
> frontend?
Frontend and Backend are both running the same kernel version, albeit
obviously with different configurations. The current version of both
kernels is 13.3.2-r3 and I am using the gentoo-hardened sources. The
same thing also happend with my previous kernel version which was
3.11.7-r2 (also gentoo-hardened sources).
Thanks Atom2
========================================================================
===== Am 02.04.14 17:17, schrieb Atom2: =======
Am 02.04.14 16:44, schrieb Ian Jackson:
> Atom2 writes ("Re: [Xen-devel] [Xen-users] substantial shutdown delay
for PV guests with PCI -passthrough"):
>> Am 21.03.14 19:11, schrieb Ian Jackson:
>>> Can you run it again with this, on top of the previous patch, please ?
>>
>> Sure, the new output of xl -vvv create -F domain is again attached to
>> this e-Mail.
>
> Sorry for the delay replying. I have been ill .
Sorry to hear that. Though I noticed your absence from the list I simply
assumed that you were off on vacation. In any case good to see you back.
>
>> <NOTE: at this point a 10s pause happens>
>> libxl: error:
libxl_device.c:1134:libxl__wait_for_backend_deprecated: Backend
/local/domain/0/backend/pci/4/0 not ready (state 7)
>> libxl: error:
libxl_device.c:1138:libxl__wait_for_backend_deprecated: FE
/local/domain/4/device/pci/0 state 6
>> libxl: debug: libxl_pci.c:204:libxl__device_pci_remove_xenstore: pci
backend at /local/domain/0/backend/pci/4/0 is not ready
>> libxl: error: libxl_pci.c:1250:do_pci_remove:
xc_domain_irq_permission irq=16
>
> So the backend here is in state 7 (Reconfiguring), but the frontend is
> in state 6 (Closed). I think this is a bug in pciback.
>
> I looked at drivers/xen/xen-pciback/xenbus.c in Linux 3.13 and found
> xen_pcibk_frontend_changed which seems to do roughly what I would
> expect.
>
> Has this changed at some point ?
>
> Atom, what kernel are you using ?
All the error messages stem from kernel 3.11.7. In the meantime 3.13.2
became stable for gentoo and I installed that a few days ago. I have not
run the debug output yet or timed the shutdown process, but there's
still a delay with that kernel and it feels as long as before. I you
want, I can clearly provide new debug output or timing information.
Thanks Atom2
>
> Thanks,
> Ian.
=======================================================================
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-04-19 18:59 ` Atom2
@ 2014-04-22 10:44 ` George Dunlap
2014-04-22 12:02 ` Atom2
0 siblings, 1 reply; 23+ messages in thread
From: George Dunlap @ 2014-04-22 10:44 UTC (permalink / raw)
To: Atom2
Cc: Ian Campbell, Ian Jackson, xen-devel@lists.xen.org, David Vrabel,
xen-users, Boris Ostrovsky, Roger Pau Monne
On Sat, Apr 19, 2014 at 7:59 PM, Atom2 <ariel.atom2@web2web.at> wrote:
> Hi Konrad,
> thanks for your quick reply. I have re-added the other recipients that were
> in the list prior to my reply from 2 April as I just saw that I somehow have
> managed to drop those guys - which might also explain their silence to my
> reply.
> All: sorry for dropping you from my earlier reply. For your convenience I
> have added my reply from 2 April at the end of this mail.
>
> Am 19.04.14 02:12, schrieb Konrad Rzeszutek Wilk:
>
>> On Fri, Apr 18, 2014 at 11:47:46PM +0200, Atom2 wrote:
>>>
>>> This is just a (very) gentle ping ... or have I missed out on a reply?
>>
>>
>> I ran an PV guest with PCI passthrough this week and it had no trouble -
>> didn't see 10 seconds or so. But I did the shutdown from within the
>> guest (poweroff).
>
> For me it makes no difference timewise whether I issue a
> xl shutdown guest
> from dom0 or whether I issue
> shutdown -h now
> from a connection (i.e. ssh or screen or console) to the guest. The main
> difference being that for the latter the delay is visible whereas for the
> former, the delay is not so obvious because 'xl shutdown guest' from dom0
> due to its asynchronous nature returns immediately even when the guest is
> still alive.
>
> One difference that I have noticed however is that for the shutdown from
> _within_ the guest (i.e. shutdown -h now) the state of the guest remains 's'
> in 'xl list' from the time the "system halted" message appears on screen
> until the prompt returns in dom0 whereas for a shutdown from dom0 with 'xl
> shutdown guest' the state changes from 's' to 'ps' for a number of seconds
> before it is finally gone.
Does it look anything like this?
marc.info/?i=<CAFLBxZbOdU=uSwNBVRTB7_7yhPyRShda0asSOvv9J+xfoGxRnA@mail.gmail.com>
(the log in question is /var/log/xen/xl-$DOMAINNAME.log)
-George
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough
2014-04-22 10:44 ` George Dunlap
@ 2014-04-22 12:02 ` Atom2
0 siblings, 0 replies; 23+ messages in thread
From: Atom2 @ 2014-04-22 12:02 UTC (permalink / raw)
To: George Dunlap
Cc: Ian Campbell, Ian Jackson, xen-devel@lists.xen.org, David Vrabel,
xen-users, Boris Ostrovsky, Roger Pau Monne
Am 22.04.14 12:44, schrieb George Dunlap:
> On Sat, Apr 19, 2014 at 7:59 PM, Atom2 <ariel.atom2@web2web.at> wrote:
>> Am 19.04.14 02:12, schrieb Konrad Rzeszutek Wilk:
>>> I ran an PV guest with PCI passthrough this week and it had no trouble -
>>> didn't see 10 seconds or so. But I did the shutdown from within the
>>> guest (poweroff).
>>
>> For me it makes no difference timewise whether I issue a
>> xl shutdown guest
>> from dom0 or whether I issue
>> shutdown -h now
>> from a connection (i.e. ssh or screen or console) to the guest. The main
>> difference being that for the latter the delay is visible whereas for the
>> former, the delay is not so obvious because 'xl shutdown guest' from dom0
>> due to its asynchronous nature returns immediately even when the guest is
>> still alive.
>>
>> One difference that I have noticed however is that for the shutdown from
>> _within_ the guest (i.e. shutdown -h now) the state of the guest remains 's'
>> in 'xl list' from the time the "system halted" message appears on screen
>> until the prompt returns in dom0 whereas for a shutdown from dom0 with 'xl
>> shutdown guest' the state changes from 's' to 'ps' for a number of seconds
>> before it is finally gone.
>
> Does it look anything like this?
>
> marc.info/?i=<CAFLBxZbOdU=uSwNBVRTB7_7yhPyRShda0asSOvv9J+xfoGxRnA@mail.gmail.com>
>
> (the log in question is /var/log/xen/xl-$DOMAINNAME.log)
>
> -George
Not really (unless there's a specific command line option required to
get your output) - at least to my eye the messages in my log file look
rather different:
_______________________________________________________________________
Waiting for domain voip (domid 3) to die [pid 2274]
Domain 3 has shut down, reason code 0 0x0
Action for shutdown reason code 0 is destroy
Domain 3 needs to be cleaned up: destroying the domain
libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission
irq=17
libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated:
Backend /local/domain/0/backend/pci/3/0 not ready (state 7)
libxl: error: libxl_device.c:1138:libxl__wait_for_backend_deprecated: FE
/local/domain/3/device/pci/0 state 6
libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission
irq=16
libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated:
Backend /local/domain/0/backend/pci/3/0 not ready (state 7)
libxl: error: libxl_device.c:1138:libxl__wait_for_backend_deprecated: FE
/local/domain/3/device/pci/0 state 6
libxl: error: libxl_pci.c:1250:do_pci_remove: xc_domain_irq_permission
irq=23
libxl: error: libxl_device.c:1134:libxl__wait_for_backend_deprecated:
Backend /local/domain/0/backend/pci/3/0 not ready (state 7)
libxl: error: libxl_device.c:1138:libxl__wait_for_backend_deprecated: FE
/local/domain/3/device/pci/0 state 6
libxl: error: libxl_device.c:894:device_backend_callback: unable to
remove device with path /local/domain/0/backend/pci/3/0
libxl: error: libxl.c:1452:devices_destroy_cb: libxl__devices_destroy
failed for 3
Done. Exiting now
Please note: some of my messages may be the result of Ian Jackson's
(debug) patches which are still active in my environment.
Thanks Atom2
^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2014-04-22 12:02 UTC | newest]
Thread overview: 23+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <5325B828.1060303@web2web.at>
[not found] ` <1395050430.4122.29.camel@kazak.uk.xensource.com>
[not found] ` <53273B3C.40707@web2web.at>
2014-03-18 10:15 ` [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough Ian Campbell
2014-03-18 13:01 ` Atom2
2014-03-18 15:07 ` Ian Campbell
2014-03-19 0:25 ` Atom2
2014-03-19 11:26 ` Ian Campbell
2014-03-19 13:00 ` Konrad Rzeszutek Wilk
2014-03-19 14:03 ` Atom2
2014-03-19 15:45 ` Ian Jackson
2014-03-20 2:31 ` Atom2
2014-03-20 11:52 ` Ian Jackson
2014-03-20 13:53 ` Pasi Kärkkäinen
2014-03-20 15:28 ` Ian Jackson
2014-03-20 19:34 ` Atom2
2014-03-20 19:32 ` Atom2
2014-03-21 18:11 ` Ian Jackson
2014-03-21 19:39 ` Atom2
2014-04-02 14:44 ` Ian Jackson
2014-04-02 15:17 ` Atom2
2014-04-18 21:47 ` Atom2
2014-04-19 0:12 ` Konrad Rzeszutek Wilk
2014-04-19 18:59 ` Atom2
2014-04-22 10:44 ` George Dunlap
2014-04-22 12:02 ` Atom2
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).