* kdump with xen-unstable on efi machine @ 2014-11-26 12:15 Juergen Gross 2014-11-26 12:41 ` Andrew Cooper 0 siblings, 1 reply; 12+ messages in thread From: Juergen Gross @ 2014-11-26 12:15 UTC (permalink / raw) To: xen-devel@lists.xensource.com Hi, I tried to enable kdump on my test-machine with actual xen-unstable booting via EFI. The kdump kernel is not being loaded. I'm seeing the memory being reserved: (XEN) EFI RAM map: (XEN) 0000000000000000 - 00000000000a0000 (usable) (XEN) 0000000000100000 - 000000004bc00000 (usable) (XEN) 000000004bc00000 - 000000005bc00000 (reserved) (XEN) 000000005bc00000 - 000000005bfec000 (usable) (XEN) 000000005bfec000 - 000000005c000000 (ACPI NVS) (XEN) 000000005c000000 - 000000006a429000 (usable) (XEN) 000000006a429000 - 000000006a42c000 (reserved) (XEN) 000000006a42c000 - 000000006a7a2000 (usable) (XEN) 000000006a7a2000 - 000000006a7a8000 (reserved) (XEN) 000000006a7a8000 - 000000006a987000 (usable) (XEN) 000000006a987000 - 000000006a98d000 (reserved) (XEN) 000000006a98d000 - 000000006aa63000 (usable) (XEN) 000000006aa63000 - 000000006aa73000 (reserved) (XEN) 000000006aa73000 - 000000006ac60000 (usable) (XEN) 000000006ac60000 - 000000006ac61000 (reserved) (XEN) 000000006ac61000 - 000000006ac9b000 (ACPI data) (XEN) 000000006ac9b000 - 000000006acac000 (reserved) (XEN) 000000006acac000 - 000000006acad000 (usable) (XEN) 000000006acad000 - 000000006acae000 (reserved) (XEN) 000000006acae000 - 000000007189c000 (usable) (XEN) 000000007189c000 - 0000000071946000 (reserved) (XEN) 0000000071946000 - 0000000072d76000 (ACPI NVS) (XEN) 0000000072d76000 - 0000000072db2000 (ACPI data) (XEN) 0000000072db2000 - 0000000072edc000 (usable) (XEN) 0000000080000000 - 0000000090000000 (reserved) (XEN) 0000000100000000 - 0000002080000000 (usable) (XEN) Kdump: 256MB (262144kB) at 0x206dff4000 I'd expect this area being visible in the efi or e820 map presented to dom0, but I can't see anything: [ 0.000000] e820: BIOS-provided physical RAM map: [ 0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable [ 0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved [ 0.000000] Xen: [mem 0x0000000000100000-0x000000004bbfffff] usable [ 0.000000] Xen: [mem 0x000000004bc00000-0x000000005bbfffff] reserved [ 0.000000] Xen: [mem 0x000000005bc00000-0x000000005bfebfff] usable [ 0.000000] Xen: [mem 0x000000005bfec000-0x000000005bffffff] ACPI NVS [ 0.000000] Xen: [mem 0x000000005c000000-0x000000006a428fff] usable [ 0.000000] Xen: [mem 0x000000006a429000-0x000000006a42bfff] reserved [ 0.000000] Xen: [mem 0x000000006a42c000-0x000000006a7a1fff] usable [ 0.000000] Xen: [mem 0x000000006a7a2000-0x000000006a7a7fff] reserved [ 0.000000] Xen: [mem 0x000000006a7a8000-0x000000006a986fff] usable [ 0.000000] Xen: [mem 0x000000006a987000-0x000000006a98cfff] reserved [ 0.000000] Xen: [mem 0x000000006a98d000-0x000000006aa62fff] usable [ 0.000000] Xen: [mem 0x000000006aa63000-0x000000006aa72fff] reserved [ 0.000000] Xen: [mem 0x000000006aa73000-0x000000006ac5ffff] usable [ 0.000000] Xen: [mem 0x000000006ac60000-0x000000006ac60fff] reserved [ 0.000000] Xen: [mem 0x000000006ac61000-0x000000006ac9afff] ACPI data [ 0.000000] Xen: [mem 0x000000006ac9b000-0x000000006acabfff] reserved [ 0.000000] Xen: [mem 0x000000006acac000-0x000000006acacfff] usable [ 0.000000] Xen: [mem 0x000000006acad000-0x000000006acadfff] reserved [ 0.000000] Xen: [mem 0x000000006acae000-0x000000007189bfff] usable [ 0.000000] Xen: [mem 0x000000007189c000-0x0000000071945fff] reserved [ 0.000000] Xen: [mem 0x0000000071946000-0x0000000072d75fff] ACPI NVS [ 0.000000] Xen: [mem 0x0000000072d76000-0x0000000072db1fff] ACPI data [ 0.000000] Xen: [mem 0x0000000072db2000-0x0000000072edbfff] usable [ 0.000000] Xen: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] Xen: [mem 0x00000000fec00000-0x00000000fec00fff] reserved [ 0.000000] Xen: [mem 0x00000000fecc0000-0x00000000fecc0fff] reserved [ 0.000000] Xen: [mem 0x00000000fee00000-0x00000000feefffff] reserved [ 0.000000] Xen: [mem 0x0000000100000000-0x000000207fffffff] usable [ 0.000000] bootconsole [xenboot0] enabled [ 0.000000] movable_node option not supported [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] e820: user-defined physical RAM map: [ 0.000000] user: [mem 0x0000000000000000-0x000000000009ffff] usable [ 0.000000] user: [mem 0x00000000000a0000-0x00000000000fffff] reserved [ 0.000000] user: [mem 0x0000000000100000-0x000000004bbfffff] usable [ 0.000000] user: [mem 0x000000004bc00000-0x000000005bbfffff] reserved [ 0.000000] user: [mem 0x000000005bc00000-0x000000005bfebfff] usable [ 0.000000] user: [mem 0x000000005bfec000-0x000000005bffffff] ACPI NVS [ 0.000000] user: [mem 0x000000005c000000-0x000000006a428fff] usable [ 0.000000] user: [mem 0x000000006a429000-0x000000006a42bfff] reserved [ 0.000000] user: [mem 0x000000006a42c000-0x000000006a7a1fff] usable [ 0.000000] user: [mem 0x000000006a7a2000-0x000000006a7a7fff] reserved [ 0.000000] user: [mem 0x000000006a7a8000-0x000000006a986fff] usable [ 0.000000] user: [mem 0x000000006a987000-0x000000006a98cfff] reserved [ 0.000000] user: [mem 0x000000006a98d000-0x000000006aa62fff] usable [ 0.000000] user: [mem 0x000000006aa63000-0x000000006aa72fff] reserved [ 0.000000] user: [mem 0x000000006aa73000-0x000000006ac5ffff] usable [ 0.000000] user: [mem 0x000000006ac60000-0x000000006ac60fff] reserved [ 0.000000] user: [mem 0x000000006ac61000-0x000000006ac9afff] ACPI data [ 0.000000] user: [mem 0x000000006ac9b000-0x000000006acabfff] reserved [ 0.000000] user: [mem 0x000000006acac000-0x000000006acacfff] usable [ 0.000000] user: [mem 0x000000006acad000-0x000000006acadfff] reserved [ 0.000000] user: [mem 0x000000006acae000-0x000000007189bfff] usable [ 0.000000] user: [mem 0x000000007189c000-0x0000000071945fff] reserved [ 0.000000] user: [mem 0x0000000071946000-0x0000000072d75fff] ACPI NVS [ 0.000000] user: [mem 0x0000000072d76000-0x0000000072db1fff] ACPI data [ 0.000000] user: [mem 0x0000000072db2000-0x0000000072edbfff] usable [ 0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] user: [mem 0x00000000fec00000-0x00000000fec00fff] reserved [ 0.000000] user: [mem 0x00000000fecc0000-0x00000000fecc0fff] reserved [ 0.000000] user: [mem 0x00000000fee00000-0x00000000feefffff] reserved [ 0.000000] user: [mem 0x0000000100000000-0x000000207fffffff] usable [ 0.000000] efi: EFI v2.31 by FUJITSU LIMITED Juergen ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kdump with xen-unstable on efi machine 2014-11-26 12:15 kdump with xen-unstable on efi machine Juergen Gross @ 2014-11-26 12:41 ` Andrew Cooper 2014-11-26 14:01 ` Juergen Gross 0 siblings, 1 reply; 12+ messages in thread From: Andrew Cooper @ 2014-11-26 12:41 UTC (permalink / raw) To: Juergen Gross, xen-devel@lists.xensource.com On 26/11/14 12:15, Juergen Gross wrote: > Hi, > > I tried to enable kdump on my test-machine with actual xen-unstable > booting via EFI. > > The kdump kernel is not being loaded. > > I'm seeing the memory being reserved: > > (XEN) EFI RAM map: > (XEN) 0000000000000000 - 00000000000a0000 (usable) > (XEN) 0000000000100000 - 000000004bc00000 (usable) > (XEN) 000000004bc00000 - 000000005bc00000 (reserved) > (XEN) 000000005bc00000 - 000000005bfec000 (usable) > (XEN) 000000005bfec000 - 000000005c000000 (ACPI NVS) > (XEN) 000000005c000000 - 000000006a429000 (usable) > (XEN) 000000006a429000 - 000000006a42c000 (reserved) > (XEN) 000000006a42c000 - 000000006a7a2000 (usable) > (XEN) 000000006a7a2000 - 000000006a7a8000 (reserved) > (XEN) 000000006a7a8000 - 000000006a987000 (usable) > (XEN) 000000006a987000 - 000000006a98d000 (reserved) > (XEN) 000000006a98d000 - 000000006aa63000 (usable) > (XEN) 000000006aa63000 - 000000006aa73000 (reserved) > (XEN) 000000006aa73000 - 000000006ac60000 (usable) > (XEN) 000000006ac60000 - 000000006ac61000 (reserved) > (XEN) 000000006ac61000 - 000000006ac9b000 (ACPI data) > (XEN) 000000006ac9b000 - 000000006acac000 (reserved) > (XEN) 000000006acac000 - 000000006acad000 (usable) > (XEN) 000000006acad000 - 000000006acae000 (reserved) > (XEN) 000000006acae000 - 000000007189c000 (usable) > (XEN) 000000007189c000 - 0000000071946000 (reserved) > (XEN) 0000000071946000 - 0000000072d76000 (ACPI NVS) > (XEN) 0000000072d76000 - 0000000072db2000 (ACPI data) > (XEN) 0000000072db2000 - 0000000072edc000 (usable) > (XEN) 0000000080000000 - 0000000090000000 (reserved) > (XEN) 0000000100000000 - 0000002080000000 (usable) > (XEN) Kdump: 256MB (262144kB) at 0x206dff4000 > > I'd expect this area being visible in the efi or e820 map presented to > dom0, but I can't see anything: This is expected. The dom0 kernel now has nothing at all do with loading crash kernel. Loading happens via hypercalls straight from the kexec utility. You need kexec-tools 2.0.4 (I think) or later, compiled with Xen support, but it should JustWork. ~Andrew ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kdump with xen-unstable on efi machine 2014-11-26 12:41 ` Andrew Cooper @ 2014-11-26 14:01 ` Juergen Gross 2014-11-26 14:26 ` Andrew Cooper 2014-11-26 14:30 ` Konrad Rzeszutek Wilk 0 siblings, 2 replies; 12+ messages in thread From: Juergen Gross @ 2014-11-26 14:01 UTC (permalink / raw) To: Andrew Cooper, xen-devel@lists.xensource.com On 11/26/2014 01:41 PM, Andrew Cooper wrote: > On 26/11/14 12:15, Juergen Gross wrote: >> Hi, >> >> I tried to enable kdump on my test-machine with actual xen-unstable >> booting via EFI. >> >> The kdump kernel is not being loaded. >> >> I'm seeing the memory being reserved: >> >> (XEN) EFI RAM map: >> (XEN) 0000000000000000 - 00000000000a0000 (usable) >> (XEN) 0000000000100000 - 000000004bc00000 (usable) >> (XEN) 000000004bc00000 - 000000005bc00000 (reserved) >> (XEN) 000000005bc00000 - 000000005bfec000 (usable) >> (XEN) 000000005bfec000 - 000000005c000000 (ACPI NVS) >> (XEN) 000000005c000000 - 000000006a429000 (usable) >> (XEN) 000000006a429000 - 000000006a42c000 (reserved) >> (XEN) 000000006a42c000 - 000000006a7a2000 (usable) >> (XEN) 000000006a7a2000 - 000000006a7a8000 (reserved) >> (XEN) 000000006a7a8000 - 000000006a987000 (usable) >> (XEN) 000000006a987000 - 000000006a98d000 (reserved) >> (XEN) 000000006a98d000 - 000000006aa63000 (usable) >> (XEN) 000000006aa63000 - 000000006aa73000 (reserved) >> (XEN) 000000006aa73000 - 000000006ac60000 (usable) >> (XEN) 000000006ac60000 - 000000006ac61000 (reserved) >> (XEN) 000000006ac61000 - 000000006ac9b000 (ACPI data) >> (XEN) 000000006ac9b000 - 000000006acac000 (reserved) >> (XEN) 000000006acac000 - 000000006acad000 (usable) >> (XEN) 000000006acad000 - 000000006acae000 (reserved) >> (XEN) 000000006acae000 - 000000007189c000 (usable) >> (XEN) 000000007189c000 - 0000000071946000 (reserved) >> (XEN) 0000000071946000 - 0000000072d76000 (ACPI NVS) >> (XEN) 0000000072d76000 - 0000000072db2000 (ACPI data) >> (XEN) 0000000072db2000 - 0000000072edc000 (usable) >> (XEN) 0000000080000000 - 0000000090000000 (reserved) >> (XEN) 0000000100000000 - 0000002080000000 (usable) >> (XEN) Kdump: 256MB (262144kB) at 0x206dff4000 >> >> I'd expect this area being visible in the efi or e820 map presented to >> dom0, but I can't see anything: > > This is expected. The dom0 kernel now has nothing at all do with > loading crash kernel. Loading happens via hypercalls straight from the > kexec utility. > > You need kexec-tools 2.0.4 (I think) or later, compiled with Xen > support, but it should JustWork. Should. I have kexec 2.0.5 with Xen support. Doesn't work: Excerpt form strace: "sysctl operation failed -- need to rebuild the user-space tool set?\n" My personal translation: kexec is tightly coupled to the Xen version (this one was built against Xen 4.4.1 AFAIK). Perhaps we should add kexec to the tools directory? Juergen ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kdump with xen-unstable on efi machine 2014-11-26 14:01 ` Juergen Gross @ 2014-11-26 14:26 ` Andrew Cooper 2014-11-27 9:49 ` Ian Campbell 2014-11-26 14:30 ` Konrad Rzeszutek Wilk 1 sibling, 1 reply; 12+ messages in thread From: Andrew Cooper @ 2014-11-26 14:26 UTC (permalink / raw) To: Juergen Gross, xen-devel@lists.xensource.com On 26/11/14 14:01, Juergen Gross wrote: > On 11/26/2014 01:41 PM, Andrew Cooper wrote: >> On 26/11/14 12:15, Juergen Gross wrote: >>> Hi, >>> >>> I tried to enable kdump on my test-machine with actual xen-unstable >>> booting via EFI. >>> >>> The kdump kernel is not being loaded. >>> >>> I'm seeing the memory being reserved: >>> >>> (XEN) EFI RAM map: >>> (XEN) 0000000000000000 - 00000000000a0000 (usable) >>> (XEN) 0000000000100000 - 000000004bc00000 (usable) >>> (XEN) 000000004bc00000 - 000000005bc00000 (reserved) >>> (XEN) 000000005bc00000 - 000000005bfec000 (usable) >>> (XEN) 000000005bfec000 - 000000005c000000 (ACPI NVS) >>> (XEN) 000000005c000000 - 000000006a429000 (usable) >>> (XEN) 000000006a429000 - 000000006a42c000 (reserved) >>> (XEN) 000000006a42c000 - 000000006a7a2000 (usable) >>> (XEN) 000000006a7a2000 - 000000006a7a8000 (reserved) >>> (XEN) 000000006a7a8000 - 000000006a987000 (usable) >>> (XEN) 000000006a987000 - 000000006a98d000 (reserved) >>> (XEN) 000000006a98d000 - 000000006aa63000 (usable) >>> (XEN) 000000006aa63000 - 000000006aa73000 (reserved) >>> (XEN) 000000006aa73000 - 000000006ac60000 (usable) >>> (XEN) 000000006ac60000 - 000000006ac61000 (reserved) >>> (XEN) 000000006ac61000 - 000000006ac9b000 (ACPI data) >>> (XEN) 000000006ac9b000 - 000000006acac000 (reserved) >>> (XEN) 000000006acac000 - 000000006acad000 (usable) >>> (XEN) 000000006acad000 - 000000006acae000 (reserved) >>> (XEN) 000000006acae000 - 000000007189c000 (usable) >>> (XEN) 000000007189c000 - 0000000071946000 (reserved) >>> (XEN) 0000000071946000 - 0000000072d76000 (ACPI NVS) >>> (XEN) 0000000072d76000 - 0000000072db2000 (ACPI data) >>> (XEN) 0000000072db2000 - 0000000072edc000 (usable) >>> (XEN) 0000000080000000 - 0000000090000000 (reserved) >>> (XEN) 0000000100000000 - 0000002080000000 (usable) >>> (XEN) Kdump: 256MB (262144kB) at 0x206dff4000 >>> >>> I'd expect this area being visible in the efi or e820 map presented to >>> dom0, but I can't see anything: >> >> This is expected. The dom0 kernel now has nothing at all do with >> loading crash kernel. Loading happens via hypercalls straight from the >> kexec utility. >> >> You need kexec-tools 2.0.4 (I think) or later, compiled with Xen >> support, but it should JustWork. > > Should. I have kexec 2.0.5 with Xen support. Doesn't work: > > Excerpt form strace: > > "sysctl operation failed -- need to rebuild the user-space tool set?\n" > > My personal translation: kexec is tightly coupled to the Xen version > (this one was built against Xen 4.4.1 AFAIK). It uses libxc, so needs to be built from the same source. > > Perhaps we should add kexec to the tools directory? The tools directory, and this willingness to fork other projects and keep a local copy, is the primary driver behind this situation being as dire as it is. libxc (or some new alternative) should suck it up and gain some notion of a stable API or ABI (like the rest of the world appears to be able to manage), such that it is possible to compile with an older header and use a newer .so at runtime. ~Andrew ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kdump with xen-unstable on efi machine 2014-11-26 14:26 ` Andrew Cooper @ 2014-11-27 9:49 ` Ian Campbell 2014-11-27 11:05 ` Andrew Cooper 0 siblings, 1 reply; 12+ messages in thread From: Ian Campbell @ 2014-11-27 9:49 UTC (permalink / raw) To: Andrew Cooper; +Cc: Juergen Gross, xen-devel@lists.xensource.com On Wed, 2014-11-26 at 14:26 +0000, Andrew Cooper wrote: > libxc (or some new alternative) should suck it up and gain some notion > of a stable API or ABI (like the rest of the world appears to be able to > manage), such that it is possible to compile with an older header and > use a newer .so at runtime. Retrofitting a stable API/ABI to the melting pot which is libxc simply isn't going to work in practice. IMO the most likely to succeed approach would be to split off the bits of libxc which 3rd party's can/should/need to rely on into one of more libraries, probably by functional area. So far I'm aware of plans (or at least desires) to do that for: * Interfaces used by device-models/qemu. * The bits which are useful inside a guest (i.e. the various /dev/xen/* related helpers). So it sounds like libxenkexec should be added to that list. Ian. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kdump with xen-unstable on efi machine 2014-11-27 9:49 ` Ian Campbell @ 2014-11-27 11:05 ` Andrew Cooper 2014-11-27 11:32 ` Ian Campbell 0 siblings, 1 reply; 12+ messages in thread From: Andrew Cooper @ 2014-11-27 11:05 UTC (permalink / raw) To: Ian Campbell; +Cc: Juergen Gross, xen-devel@lists.xensource.com On 27/11/14 09:49, Ian Campbell wrote: > On Wed, 2014-11-26 at 14:26 +0000, Andrew Cooper wrote: >> libxc (or some new alternative) should suck it up and gain some notion >> of a stable API or ABI (like the rest of the world appears to be able to >> manage), such that it is possible to compile with an older header and >> use a newer .so at runtime. > Retrofitting a stable API/ABI to the melting pot which is libxc simply > isn't going to work in practice. > > IMO the most likely to succeed approach would be to split off the bits > of libxc which 3rd party's can/should/need to rely on into one of more > libraries, probably by functional area. > > So far I'm aware of plans (or at least desires) to do that for: > > * Interfaces used by device-models/qemu. > * The bits which are useful inside a guest (i.e. the > various /dev/xen/* related helpers). > > So it sounds like libxenkexec should be added to that list. > > Ian. > Agreed. For a domU, I think we need libxenevt, libxengnt and libxenstore with stable API and ABIs. This in turn will permit libvchan to work without needing libxenctrl. For dom0, each of the libraries is going to need basic hypercall functionality. It might be worth considering making libxenbasic (name looking for improvement) which is more along the lines of a privcmd driver, providing do_hypercall() and bounce buffering. libxenctrl and others can then avoid reimplementing the wheel many times. ~Andrew ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kdump with xen-unstable on efi machine 2014-11-27 11:05 ` Andrew Cooper @ 2014-11-27 11:32 ` Ian Campbell 2014-11-27 11:51 ` Andrew Cooper 0 siblings, 1 reply; 12+ messages in thread From: Ian Campbell @ 2014-11-27 11:32 UTC (permalink / raw) To: Andrew Cooper; +Cc: Juergen Gross, xen-devel@lists.xensource.com On Thu, 2014-11-27 at 11:05 +0000, Andrew Cooper wrote: > On 27/11/14 09:49, Ian Campbell wrote: > > On Wed, 2014-11-26 at 14:26 +0000, Andrew Cooper wrote: > >> libxc (or some new alternative) should suck it up and gain some notion > >> of a stable API or ABI (like the rest of the world appears to be able to > >> manage), such that it is possible to compile with an older header and > >> use a newer .so at runtime. > > Retrofitting a stable API/ABI to the melting pot which is libxc simply > > isn't going to work in practice. > > > > IMO the most likely to succeed approach would be to split off the bits > > of libxc which 3rd party's can/should/need to rely on into one of more > > libraries, probably by functional area. > > > > So far I'm aware of plans (or at least desires) to do that for: > > > > * Interfaces used by device-models/qemu. > > * The bits which are useful inside a guest (i.e. the > > various /dev/xen/* related helpers). > > > > So it sounds like libxenkexec should be added to that list. > > > > Ian. > > > > Agreed. > > For a domU, I think we need libxenevt, libxengnt and libxenstore with > stable API and ABIs. This in turn will permit libvchan to work without > needing libxenctrl. > > For dom0, each of the libraries is going to need basic hypercall > functionality. It might be worth considering making libxenbasic (name > looking for improvement) which is more along the lines of a privcmd > driver, providing do_hypercall() and bounce buffering. > libxenctrl and > others can then avoid reimplementing the wheel many times. NB that if such a library contains the h/call wrappers for any non-stable hypercall interface (I'm not sure if that was what you were suggesting or not) then it would itself remain ABI/API unstable. But anyway there's nothing stopping libxen* using interfaces exposed by an unstable libxenctrl or libxenbasic and remaining interface stable themselves, so you can avoid reimplementing the wheel that way. Those libraries just need to do the impedance matching and link the correct SONAME for the unstable library, the linker will take care of the rest. I don't think it is going to be realistic in practice to support keeping the exact same libxenfoo.so.N binary working across multiple releases, but rather each release will provide its own libxenfoo.so.N with the same N and supporting the same ABI, such that it can just be swapped in under the feet of the application. The alternative is to do dlopen stuff on the unstable backend library, which is going to be a nightmare, from the test matrix PoV if nothing else. Ian. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kdump with xen-unstable on efi machine 2014-11-27 11:32 ` Ian Campbell @ 2014-11-27 11:51 ` Andrew Cooper 0 siblings, 0 replies; 12+ messages in thread From: Andrew Cooper @ 2014-11-27 11:51 UTC (permalink / raw) To: Ian Campbell; +Cc: Juergen Gross, xen-devel@lists.xensource.com On 27/11/14 11:32, Ian Campbell wrote: > On Thu, 2014-11-27 at 11:05 +0000, Andrew Cooper wrote: >> On 27/11/14 09:49, Ian Campbell wrote: >>> On Wed, 2014-11-26 at 14:26 +0000, Andrew Cooper wrote: >>>> libxc (or some new alternative) should suck it up and gain some notion >>>> of a stable API or ABI (like the rest of the world appears to be able to >>>> manage), such that it is possible to compile with an older header and >>>> use a newer .so at runtime. >>> Retrofitting a stable API/ABI to the melting pot which is libxc simply >>> isn't going to work in practice. >>> >>> IMO the most likely to succeed approach would be to split off the bits >>> of libxc which 3rd party's can/should/need to rely on into one of more >>> libraries, probably by functional area. >>> >>> So far I'm aware of plans (or at least desires) to do that for: >>> >>> * Interfaces used by device-models/qemu. >>> * The bits which are useful inside a guest (i.e. the >>> various /dev/xen/* related helpers). >>> >>> So it sounds like libxenkexec should be added to that list. >>> >>> Ian. >>> >> Agreed. >> >> For a domU, I think we need libxenevt, libxengnt and libxenstore with >> stable API and ABIs. This in turn will permit libvchan to work without >> needing libxenctrl. >> >> For dom0, each of the libraries is going to need basic hypercall >> functionality. It might be worth considering making libxenbasic (name >> looking for improvement) which is more along the lines of a privcmd >> driver, providing do_hypercall() and bounce buffering. >> libxenctrl and >> others can then avoid reimplementing the wheel many times. > NB that if such a library contains the h/call wrappers for any > non-stable hypercall interface (I'm not sure if that was what you were > suggesting or not) then it would itself remain ABI/API unstable. I was suggesting do_hypercall() and specifically not things like do_domctl(). do_hypercall() is a think wrapper around ioctl(), which will be stable going forwards. This allows a consumer to use stable hypercalls and keep stability. libxenctrl/libxenguest can implement do_domctl() in terms of do_hypercall(), and gain the inherent instability that comes with this. However, I think all of these are design decisions to be considered by whomever tackles this chunk of work :) > > But anyway there's nothing stopping libxen* using interfaces exposed by > an unstable libxenctrl or libxenbasic and remaining interface stable > themselves, so you can avoid reimplementing the wheel that way. libxenctrl, being an explicit unstable API and ABI cannot reasonably be used by something claiming to provide a stable ABI, without libxenctrl changing its stance on what is considered stable and what is not. I suppose the advantage of this is that it can be sorted with some paperwork. > > Those libraries just need to do the impedance matching and link the > correct SONAME for the unstable library, the linker will take care of > the rest. > > I don't think it is going to be realistic in practice to support keeping > the exact same libxenfoo.so.N binary working across multiple releases, > but rather each release will provide its own libxenfoo.so.N with the > same N and supporting the same ABI, such that it can just be swapped in > under the feet of the application. I believe that this covers our intentions, and appears to be the common way of doing things. ~Andrew ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kdump with xen-unstable on efi machine 2014-11-26 14:01 ` Juergen Gross 2014-11-26 14:26 ` Andrew Cooper @ 2014-11-26 14:30 ` Konrad Rzeszutek Wilk 2014-11-26 14:49 ` Juergen Gross 1 sibling, 1 reply; 12+ messages in thread From: Konrad Rzeszutek Wilk @ 2014-11-26 14:30 UTC (permalink / raw) To: Juergen Gross, daniel.kiper; +Cc: Andrew Cooper, xen-devel@lists.xensource.com On Wed, Nov 26, 2014 at 03:01:51PM +0100, Juergen Gross wrote: > On 11/26/2014 01:41 PM, Andrew Cooper wrote: > >On 26/11/14 12:15, Juergen Gross wrote: > >>Hi, > >> > >>I tried to enable kdump on my test-machine with actual xen-unstable > >>booting via EFI. > >> > >>The kdump kernel is not being loaded. > >> > >>I'm seeing the memory being reserved: > >> > >>(XEN) EFI RAM map: > >>(XEN) 0000000000000000 - 00000000000a0000 (usable) > >>(XEN) 0000000000100000 - 000000004bc00000 (usable) > >>(XEN) 000000004bc00000 - 000000005bc00000 (reserved) > >>(XEN) 000000005bc00000 - 000000005bfec000 (usable) > >>(XEN) 000000005bfec000 - 000000005c000000 (ACPI NVS) > >>(XEN) 000000005c000000 - 000000006a429000 (usable) > >>(XEN) 000000006a429000 - 000000006a42c000 (reserved) > >>(XEN) 000000006a42c000 - 000000006a7a2000 (usable) > >>(XEN) 000000006a7a2000 - 000000006a7a8000 (reserved) > >>(XEN) 000000006a7a8000 - 000000006a987000 (usable) > >>(XEN) 000000006a987000 - 000000006a98d000 (reserved) > >>(XEN) 000000006a98d000 - 000000006aa63000 (usable) > >>(XEN) 000000006aa63000 - 000000006aa73000 (reserved) > >>(XEN) 000000006aa73000 - 000000006ac60000 (usable) > >>(XEN) 000000006ac60000 - 000000006ac61000 (reserved) > >>(XEN) 000000006ac61000 - 000000006ac9b000 (ACPI data) > >>(XEN) 000000006ac9b000 - 000000006acac000 (reserved) > >>(XEN) 000000006acac000 - 000000006acad000 (usable) > >>(XEN) 000000006acad000 - 000000006acae000 (reserved) > >>(XEN) 000000006acae000 - 000000007189c000 (usable) > >>(XEN) 000000007189c000 - 0000000071946000 (reserved) > >>(XEN) 0000000071946000 - 0000000072d76000 (ACPI NVS) > >>(XEN) 0000000072d76000 - 0000000072db2000 (ACPI data) > >>(XEN) 0000000072db2000 - 0000000072edc000 (usable) > >>(XEN) 0000000080000000 - 0000000090000000 (reserved) > >>(XEN) 0000000100000000 - 0000002080000000 (usable) > >>(XEN) Kdump: 256MB (262144kB) at 0x206dff4000 > >> > >>I'd expect this area being visible in the efi or e820 map presented to > >>dom0, but I can't see anything: > > > >This is expected. The dom0 kernel now has nothing at all do with > >loading crash kernel. Loading happens via hypercalls straight from the > >kexec utility. > > > >You need kexec-tools 2.0.4 (I think) or later, compiled with Xen > >support, but it should JustWork. > > Should. I have kexec 2.0.5 with Xen support. Doesn't work: > > Excerpt form strace: > > "sysctl operation failed -- need to rebuild the user-space tool set?\n" > > My personal translation: kexec is tightly coupled to the Xen version > (this one was built against Xen 4.4.1 AFAIK). Odd, the hypercall interface did not change in Xen 4.5 for kexec? Perhaps it is making some other hypercalls that are tied in to the version of Xen (like sysctl ones?). I presume with recompiling it works? > > Perhaps we should add kexec to the tools directory? Gosh no. > > > Juergen > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kdump with xen-unstable on efi machine 2014-11-26 14:30 ` Konrad Rzeszutek Wilk @ 2014-11-26 14:49 ` Juergen Gross 2014-11-27 0:19 ` Daniel Kiper 0 siblings, 1 reply; 12+ messages in thread From: Juergen Gross @ 2014-11-26 14:49 UTC (permalink / raw) To: Konrad Rzeszutek Wilk, Daniel Kiper, Andrew Cooper, xen-devel@lists.xensource.com On 11/26/2014 03:30 PM, Konrad Rzeszutek Wilk wrote: > On Wed, Nov 26, 2014 at 03:01:51PM +0100, Juergen Gross wrote: >> On 11/26/2014 01:41 PM, Andrew Cooper wrote: >>> On 26/11/14 12:15, Juergen Gross wrote: >>>> Hi, >>>> >>>> I tried to enable kdump on my test-machine with actual xen-unstable >>>> booting via EFI. >>>> >>>> The kdump kernel is not being loaded. >>>> >>>> I'm seeing the memory being reserved: >>>> >>>> (XEN) EFI RAM map: >>>> (XEN) 0000000000000000 - 00000000000a0000 (usable) >>>> (XEN) 0000000000100000 - 000000004bc00000 (usable) >>>> (XEN) 000000004bc00000 - 000000005bc00000 (reserved) >>>> (XEN) 000000005bc00000 - 000000005bfec000 (usable) >>>> (XEN) 000000005bfec000 - 000000005c000000 (ACPI NVS) >>>> (XEN) 000000005c000000 - 000000006a429000 (usable) >>>> (XEN) 000000006a429000 - 000000006a42c000 (reserved) >>>> (XEN) 000000006a42c000 - 000000006a7a2000 (usable) >>>> (XEN) 000000006a7a2000 - 000000006a7a8000 (reserved) >>>> (XEN) 000000006a7a8000 - 000000006a987000 (usable) >>>> (XEN) 000000006a987000 - 000000006a98d000 (reserved) >>>> (XEN) 000000006a98d000 - 000000006aa63000 (usable) >>>> (XEN) 000000006aa63000 - 000000006aa73000 (reserved) >>>> (XEN) 000000006aa73000 - 000000006ac60000 (usable) >>>> (XEN) 000000006ac60000 - 000000006ac61000 (reserved) >>>> (XEN) 000000006ac61000 - 000000006ac9b000 (ACPI data) >>>> (XEN) 000000006ac9b000 - 000000006acac000 (reserved) >>>> (XEN) 000000006acac000 - 000000006acad000 (usable) >>>> (XEN) 000000006acad000 - 000000006acae000 (reserved) >>>> (XEN) 000000006acae000 - 000000007189c000 (usable) >>>> (XEN) 000000007189c000 - 0000000071946000 (reserved) >>>> (XEN) 0000000071946000 - 0000000072d76000 (ACPI NVS) >>>> (XEN) 0000000072d76000 - 0000000072db2000 (ACPI data) >>>> (XEN) 0000000072db2000 - 0000000072edc000 (usable) >>>> (XEN) 0000000080000000 - 0000000090000000 (reserved) >>>> (XEN) 0000000100000000 - 0000002080000000 (usable) >>>> (XEN) Kdump: 256MB (262144kB) at 0x206dff4000 >>>> >>>> I'd expect this area being visible in the efi or e820 map presented to >>>> dom0, but I can't see anything: >>> >>> This is expected. The dom0 kernel now has nothing at all do with >>> loading crash kernel. Loading happens via hypercalls straight from the >>> kexec utility. >>> >>> You need kexec-tools 2.0.4 (I think) or later, compiled with Xen >>> support, but it should JustWork. >> >> Should. I have kexec 2.0.5 with Xen support. Doesn't work: >> >> Excerpt form strace: >> >> "sysctl operation failed -- need to rebuild the user-space tool set?\n" >> >> My personal translation: kexec is tightly coupled to the Xen version >> (this one was built against Xen 4.4.1 AFAIK). > > Odd, the hypercall interface did not change in Xen 4.5 for kexec? > > Perhaps it is making some other hypercalls that are tied in > to the version of Xen (like sysctl ones?). The error message above suggests that, yes. :-) Grepping for xc_ in kexec sources finds e.g. xc_get_max_cpus() which in turn calls xc_physinfo() doing a sysctl. > > I presume with recompiling it works? Didn't check up to now, but I think it should. >> >> Perhaps we should add kexec to the tools directory? > > Gosh no. Oops, did I forget the smiley? ;-) I think we should look what kexec is really needing and put this in a stable interface set (perhaps an own library?). This might require some new sub functions of e.g. the KEXEC hypercall, but this is better than making kexec depending on the Xen version. Juergen ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kdump with xen-unstable on efi machine 2014-11-26 14:49 ` Juergen Gross @ 2014-11-27 0:19 ` Daniel Kiper 2014-11-27 5:40 ` Juergen Gross 0 siblings, 1 reply; 12+ messages in thread From: Daniel Kiper @ 2014-11-27 0:19 UTC (permalink / raw) To: Juergen Gross; +Cc: Andrew Cooper, xen-devel@lists.xensource.com On Wed, Nov 26, 2014 at 03:49:54PM +0100, Juergen Gross wrote: > On 11/26/2014 03:30 PM, Konrad Rzeszutek Wilk wrote: > >On Wed, Nov 26, 2014 at 03:01:51PM +0100, Juergen Gross wrote: > >>On 11/26/2014 01:41 PM, Andrew Cooper wrote: > >>>On 26/11/14 12:15, Juergen Gross wrote: > >>>>Hi, > >>>> > >>>>I tried to enable kdump on my test-machine with actual xen-unstable > >>>>booting via EFI. > >>>> > >>>>The kdump kernel is not being loaded. > >>>> > >>>>I'm seeing the memory being reserved: > >>>> > >>>>(XEN) EFI RAM map: > >>>>(XEN) 0000000000000000 - 00000000000a0000 (usable) > >>>>(XEN) 0000000000100000 - 000000004bc00000 (usable) > >>>>(XEN) 000000004bc00000 - 000000005bc00000 (reserved) > >>>>(XEN) 000000005bc00000 - 000000005bfec000 (usable) > >>>>(XEN) 000000005bfec000 - 000000005c000000 (ACPI NVS) > >>>>(XEN) 000000005c000000 - 000000006a429000 (usable) > >>>>(XEN) 000000006a429000 - 000000006a42c000 (reserved) > >>>>(XEN) 000000006a42c000 - 000000006a7a2000 (usable) > >>>>(XEN) 000000006a7a2000 - 000000006a7a8000 (reserved) > >>>>(XEN) 000000006a7a8000 - 000000006a987000 (usable) > >>>>(XEN) 000000006a987000 - 000000006a98d000 (reserved) > >>>>(XEN) 000000006a98d000 - 000000006aa63000 (usable) > >>>>(XEN) 000000006aa63000 - 000000006aa73000 (reserved) > >>>>(XEN) 000000006aa73000 - 000000006ac60000 (usable) > >>>>(XEN) 000000006ac60000 - 000000006ac61000 (reserved) > >>>>(XEN) 000000006ac61000 - 000000006ac9b000 (ACPI data) > >>>>(XEN) 000000006ac9b000 - 000000006acac000 (reserved) > >>>>(XEN) 000000006acac000 - 000000006acad000 (usable) > >>>>(XEN) 000000006acad000 - 000000006acae000 (reserved) > >>>>(XEN) 000000006acae000 - 000000007189c000 (usable) > >>>>(XEN) 000000007189c000 - 0000000071946000 (reserved) > >>>>(XEN) 0000000071946000 - 0000000072d76000 (ACPI NVS) > >>>>(XEN) 0000000072d76000 - 0000000072db2000 (ACPI data) > >>>>(XEN) 0000000072db2000 - 0000000072edc000 (usable) > >>>>(XEN) 0000000080000000 - 0000000090000000 (reserved) > >>>>(XEN) 0000000100000000 - 0000002080000000 (usable) > >>>>(XEN) Kdump: 256MB (262144kB) at 0x206dff4000 > >>>> > >>>>I'd expect this area being visible in the efi or e820 map presented to > >>>>dom0, but I can't see anything: > >>> > >>>This is expected. The dom0 kernel now has nothing at all do with > >>>loading crash kernel. Loading happens via hypercalls straight from the > >>>kexec utility. > >>> > >>>You need kexec-tools 2.0.4 (I think) or later, compiled with Xen > >>>support, but it should JustWork. > >> > >>Should. I have kexec 2.0.5 with Xen support. Doesn't work: > >> > >>Excerpt form strace: > >> > >>"sysctl operation failed -- need to rebuild the user-space tool set?\n" > >> > >>My personal translation: kexec is tightly coupled to the Xen version > >>(this one was built against Xen 4.4.1 AFAIK). > > > >Odd, the hypercall interface did not change in Xen 4.5 for kexec? > > > >Perhaps it is making some other hypercalls that are tied in > >to the version of Xen (like sysctl ones?). > > The error message above suggests that, yes. :-) > > Grepping for xc_ in kexec sources finds e.g. xc_get_max_cpus() which > in turn calls xc_physinfo() doing a sysctl. > > > > >I presume with recompiling it works? > > Didn't check up to now, but I think it should. Are you sure that kexec-tools configure script discovered Xen headers and development libraries? Please check that. "ldd kexec" is your friend. Do not forget to use kexec-tools version 2.0.5 or newer. > >> > >>Perhaps we should add kexec to the tools directory? > > > >Gosh no. > > Oops, did I forget the smiley? ;-) > > I think we should look what kexec is really needing and put this in a > stable interface set (perhaps an own library?). This might require some David did the work. > new sub functions of e.g. the KEXEC hypercall, but this is better than > making kexec depending on the Xen version. Maybe we need some things which are specific for EFI platforms. I am going to investigate that after finishing EFI + multiboot2 work. Probably it will happen at the beginning of next year. Daniel ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: kdump with xen-unstable on efi machine 2014-11-27 0:19 ` Daniel Kiper @ 2014-11-27 5:40 ` Juergen Gross 0 siblings, 0 replies; 12+ messages in thread From: Juergen Gross @ 2014-11-27 5:40 UTC (permalink / raw) To: Daniel Kiper; +Cc: Andrew Cooper, xen-devel@lists.xensource.com On 11/27/2014 01:19 AM, Daniel Kiper wrote: > On Wed, Nov 26, 2014 at 03:49:54PM +0100, Juergen Gross wrote: >> On 11/26/2014 03:30 PM, Konrad Rzeszutek Wilk wrote: >>> On Wed, Nov 26, 2014 at 03:01:51PM +0100, Juergen Gross wrote: >>>> On 11/26/2014 01:41 PM, Andrew Cooper wrote: >>>>> On 26/11/14 12:15, Juergen Gross wrote: >>>>>> Hi, >>>>>> >>>>>> I tried to enable kdump on my test-machine with actual xen-unstable >>>>>> booting via EFI. >>>>>> >>>>>> The kdump kernel is not being loaded. >>>>>> >>>>>> I'm seeing the memory being reserved: >>>>>> >>>>>> (XEN) EFI RAM map: >>>>>> (XEN) 0000000000000000 - 00000000000a0000 (usable) >>>>>> (XEN) 0000000000100000 - 000000004bc00000 (usable) >>>>>> (XEN) 000000004bc00000 - 000000005bc00000 (reserved) >>>>>> (XEN) 000000005bc00000 - 000000005bfec000 (usable) >>>>>> (XEN) 000000005bfec000 - 000000005c000000 (ACPI NVS) >>>>>> (XEN) 000000005c000000 - 000000006a429000 (usable) >>>>>> (XEN) 000000006a429000 - 000000006a42c000 (reserved) >>>>>> (XEN) 000000006a42c000 - 000000006a7a2000 (usable) >>>>>> (XEN) 000000006a7a2000 - 000000006a7a8000 (reserved) >>>>>> (XEN) 000000006a7a8000 - 000000006a987000 (usable) >>>>>> (XEN) 000000006a987000 - 000000006a98d000 (reserved) >>>>>> (XEN) 000000006a98d000 - 000000006aa63000 (usable) >>>>>> (XEN) 000000006aa63000 - 000000006aa73000 (reserved) >>>>>> (XEN) 000000006aa73000 - 000000006ac60000 (usable) >>>>>> (XEN) 000000006ac60000 - 000000006ac61000 (reserved) >>>>>> (XEN) 000000006ac61000 - 000000006ac9b000 (ACPI data) >>>>>> (XEN) 000000006ac9b000 - 000000006acac000 (reserved) >>>>>> (XEN) 000000006acac000 - 000000006acad000 (usable) >>>>>> (XEN) 000000006acad000 - 000000006acae000 (reserved) >>>>>> (XEN) 000000006acae000 - 000000007189c000 (usable) >>>>>> (XEN) 000000007189c000 - 0000000071946000 (reserved) >>>>>> (XEN) 0000000071946000 - 0000000072d76000 (ACPI NVS) >>>>>> (XEN) 0000000072d76000 - 0000000072db2000 (ACPI data) >>>>>> (XEN) 0000000072db2000 - 0000000072edc000 (usable) >>>>>> (XEN) 0000000080000000 - 0000000090000000 (reserved) >>>>>> (XEN) 0000000100000000 - 0000002080000000 (usable) >>>>>> (XEN) Kdump: 256MB (262144kB) at 0x206dff4000 >>>>>> >>>>>> I'd expect this area being visible in the efi or e820 map presented to >>>>>> dom0, but I can't see anything: >>>>> >>>>> This is expected. The dom0 kernel now has nothing at all do with >>>>> loading crash kernel. Loading happens via hypercalls straight from the >>>>> kexec utility. >>>>> >>>>> You need kexec-tools 2.0.4 (I think) or later, compiled with Xen >>>>> support, but it should JustWork. >>>> >>>> Should. I have kexec 2.0.5 with Xen support. Doesn't work: >>>> >>>> Excerpt form strace: >>>> >>>> "sysctl operation failed -- need to rebuild the user-space tool set?\n" >>>> >>>> My personal translation: kexec is tightly coupled to the Xen version >>>> (this one was built against Xen 4.4.1 AFAIK). >>> >>> Odd, the hypercall interface did not change in Xen 4.5 for kexec? >>> >>> Perhaps it is making some other hypercalls that are tied in >>> to the version of Xen (like sysctl ones?). >> >> The error message above suggests that, yes. :-) >> >> Grepping for xc_ in kexec sources finds e.g. xc_get_max_cpus() which >> in turn calls xc_physinfo() doing a sysctl. >> >>> >>> I presume with recompiling it works? >> >> Didn't check up to now, but I think it should. > > Are you sure that kexec-tools configure script discovered > Xen headers and development libraries? Please check that. > "ldd kexec" is your friend. Aah, here is the problem: The kexec installed on my test system has been linked statically with libxenctrl to be usable on non-Xen systems, too. The proper solution, however, would have been to check for running on Xen dynamically and then load the current library via dlopen(). For now I'll just rebuild kexec. I put above on my todo list. As I expect to modify kexec in the future for support of the linear mapped p2m list I'll do the dlopen() stuff then. > > Do not forget to use kexec-tools version 2.0.5 or newer. > >>>> >>>> Perhaps we should add kexec to the tools directory? >>> >>> Gosh no. >> >> Oops, did I forget the smiley? ;-) >> >> I think we should look what kexec is really needing and put this in a >> stable interface set (perhaps an own library?). This might require some > > David did the work. Okay. > >> new sub functions of e.g. the KEXEC hypercall, but this is better than >> making kexec depending on the Xen version. > > Maybe we need some things which are specific for EFI platforms. I am > going to investigate that after finishing EFI + multiboot2 work. > Probably it will happen at the beginning of next year. I'll come back to you when I'm going to change kexec. Juergen ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2014-11-27 11:51 UTC | newest] Thread overview: 12+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-11-26 12:15 kdump with xen-unstable on efi machine Juergen Gross 2014-11-26 12:41 ` Andrew Cooper 2014-11-26 14:01 ` Juergen Gross 2014-11-26 14:26 ` Andrew Cooper 2014-11-27 9:49 ` Ian Campbell 2014-11-27 11:05 ` Andrew Cooper 2014-11-27 11:32 ` Ian Campbell 2014-11-27 11:51 ` Andrew Cooper 2014-11-26 14:30 ` Konrad Rzeszutek Wilk 2014-11-26 14:49 ` Juergen Gross 2014-11-27 0:19 ` Daniel Kiper 2014-11-27 5:40 ` Juergen Gross
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.