* [PATCH 1/2] x86/xen: fix memory setup for PVH dom0
2014-06-02 17:09 [RFC PATCHv1 0/2] Linux PVH dom0 memory setup David Vrabel
@ 2014-06-02 17:09 ` David Vrabel
2014-06-02 17:28 ` Boris Ostrovsky
2014-06-03 12:18 ` Roger Pau Monné
2014-06-02 17:09 ` [PATCH 2/2] Revert "xen/pvh: Update E820 to work with PVH (v2)" David Vrabel
2014-06-02 22:46 ` [RFC PATCHv1 0/2] Linux PVH dom0 memory setup Mukesh Rathor
2 siblings, 2 replies; 6+ messages in thread
From: David Vrabel @ 2014-06-02 17:09 UTC (permalink / raw)
To: xen-devel; +Cc: Boris Ostrovsky, David Vrabel
Since af06d66ee32b (x86: fix setup of PVH Dom0 memory map) in Xen, PVH
dom0 need only use the memory memory provided by Xen which has already
setup all the correct holes.
xen_memory_setup() then ends up being trivial for a PVH guest so
introduce a new function (xen_auto_xlated_memory_setup()).
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
---
arch/x86/xen/enlighten.c | 5 ++++-
arch/x86/xen/setup.c | 23 +++++++++++++++++++++++
arch/x86/xen/xen-ops.h | 1 +
3 files changed, 28 insertions(+), 1 deletion(-)
diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index c34bfc4..7dffa8c 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -1536,7 +1536,10 @@ asmlinkage __visible void __init xen_start_kernel(void)
if (!xen_pvh_domain())
pv_cpu_ops = xen_cpu_ops;
- x86_init.resources.memory_setup = xen_memory_setup;
+ if (xen_feature(XENFEAT_auto_translated_physmap))
+ x86_init.resources.memory_setup = xen_auto_xlated_memory_setup;
+ else
+ x86_init.resources.memory_setup = xen_memory_setup;
x86_init.oem.arch_setup = xen_arch_setup;
x86_init.oem.banner = xen_banner;
diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c
index 0982233..e7ef388 100644
--- a/arch/x86/xen/setup.c
+++ b/arch/x86/xen/setup.c
@@ -509,6 +509,29 @@ char * __init xen_memory_setup(void)
}
/*
+ * Machine specific memory setup for auto-translated guests.
+ */
+char * __init xen_auto_xlated_memory_setup(void)
+{
+ static struct e820entry map[E820MAX] __initdata;
+
+ struct xen_memory_map memmap;
+ int i;
+ int rc;
+
+ memmap.nr_entries = E820MAX;
+ set_xen_guest_handle(memmap.buffer, map);
+
+ rc = HYPERVISOR_memory_op(XENMEM_memory_map, &memmap);
+ BUG_ON(rc);
+
+ for (i = 0; i < memmap.nr_entries; i++)
+ e820_add_region(map[i].addr, map[i].size, map[i].type);
+
+ return "Xen";
+}
+
+/*
* Set the bit indicating "nosegneg" library variants should be used.
* We only need to bother in pure 32-bit mode; compat 32-bit processes
* can have un-truncated segments, so wrapping around is allowed.
diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h
index 1cb6f4c..b371d18 100644
--- a/arch/x86/xen/xen-ops.h
+++ b/arch/x86/xen/xen-ops.h
@@ -34,6 +34,7 @@ extern unsigned long xen_max_p2m_pfn;
void xen_set_pat(u64);
char * __init xen_memory_setup(void);
+char * xen_auto_xlated_memory_setup(void);
void __init xen_arch_setup(void);
void xen_enable_sysenter(void);
void xen_enable_syscall(void);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH 1/2] x86/xen: fix memory setup for PVH dom0
2014-06-02 17:09 ` [PATCH 1/2] x86/xen: fix memory setup for PVH dom0 David Vrabel
@ 2014-06-02 17:28 ` Boris Ostrovsky
2014-06-03 12:18 ` Roger Pau Monné
1 sibling, 0 replies; 6+ messages in thread
From: Boris Ostrovsky @ 2014-06-02 17:28 UTC (permalink / raw)
To: David Vrabel; +Cc: xen-devel
On 06/02/2014 01:09 PM, David Vrabel wrote:
> /*
> + * Machine specific memory setup for auto-translated guests.
> + */
> +char * __init xen_auto_xlated_memory_setup(void)
> +{
> + static struct e820entry map[E820MAX] __initdata;
> +
> + struct xen_memory_map memmap;
> + int i;
> + int rc;
> +
> + memmap.nr_entries = E820MAX;
> + set_xen_guest_handle(memmap.buffer, map);
> +
> + rc = HYPERVISOR_memory_op(XENMEM_memory_map, &memmap);
> + BUG_ON(rc);
Do we want to know what the error was?
-boris
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH 1/2] x86/xen: fix memory setup for PVH dom0
2014-06-02 17:09 ` [PATCH 1/2] x86/xen: fix memory setup for PVH dom0 David Vrabel
2014-06-02 17:28 ` Boris Ostrovsky
@ 2014-06-03 12:18 ` Roger Pau Monné
1 sibling, 0 replies; 6+ messages in thread
From: Roger Pau Monné @ 2014-06-03 12:18 UTC (permalink / raw)
To: David Vrabel, xen-devel; +Cc: Boris Ostrovsky
On 02/06/14 19:09, David Vrabel wrote:
> Since af06d66ee32b (x86: fix setup of PVH Dom0 memory map) in Xen, PVH
> dom0 need only use the memory memory provided by Xen which has already
> setup all the correct holes.
>
> xen_memory_setup() then ends up being trivial for a PVH guest so
> introduce a new function (xen_auto_xlated_memory_setup()).
>
> Signed-off-by: David Vrabel <david.vrabel@citrix.com>
Hello,
Thanks for the patch, as Mukesh pointed out, this is not working as-is.
> ---
> arch/x86/xen/enlighten.c | 5 ++++-
> arch/x86/xen/setup.c | 23 +++++++++++++++++++++++
> arch/x86/xen/xen-ops.h | 1 +
> 3 files changed, 28 insertions(+), 1 deletion(-)
>
> diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
> index c34bfc4..7dffa8c 100644
> --- a/arch/x86/xen/enlighten.c
> +++ b/arch/x86/xen/enlighten.c
> @@ -1536,7 +1536,10 @@ asmlinkage __visible void __init xen_start_kernel(void)
> if (!xen_pvh_domain())
> pv_cpu_ops = xen_cpu_ops;
>
> - x86_init.resources.memory_setup = xen_memory_setup;
> + if (xen_feature(XENFEAT_auto_translated_physmap))
> + x86_init.resources.memory_setup = xen_auto_xlated_memory_setup;
> + else
> + x86_init.resources.memory_setup = xen_memory_setup;
> x86_init.oem.arch_setup = xen_arch_setup;
> x86_init.oem.banner = xen_banner;
>
> diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c
> index 0982233..e7ef388 100644
> --- a/arch/x86/xen/setup.c
> +++ b/arch/x86/xen/setup.c
> @@ -509,6 +509,29 @@ char * __init xen_memory_setup(void)
> }
>
> /*
> + * Machine specific memory setup for auto-translated guests.
> + */
> +char * __init xen_auto_xlated_memory_setup(void)
> +{
> + static struct e820entry map[E820MAX] __initdata;
> +
> + struct xen_memory_map memmap;
> + int i;
> + int rc;
> +
> + memmap.nr_entries = E820MAX;
> + set_xen_guest_handle(memmap.buffer, map);
> +
> + rc = HYPERVISOR_memory_op(XENMEM_memory_map, &memmap);
> + BUG_ON(rc);
Although not strictly necessary, I think it would be good to add the
following here (it's also done in bare metal Linux):
sanitize_e820_map(map, ARRAY_SIZE(map), &memmap.nr_entries);
> +
> + for (i = 0; i < memmap.nr_entries; i++)
> + e820_add_region(map[i].addr, map[i].size, map[i].type);
And I think here we need to reserve the start_info area:
memblock_reserve(__pa(xen_start_info->mfn_list),
xen_start_info->pt_base - xen_start_info->mfn_list);
With those changes I've been able to boot fine, but I'm still testing.
> + return "Xen";
> +}
> +
> +/*
> * Set the bit indicating "nosegneg" library variants should be used.
> * We only need to bother in pure 32-bit mode; compat 32-bit processes
> * can have un-truncated segments, so wrapping around is allowed.
> diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h
> index 1cb6f4c..b371d18 100644
> --- a/arch/x86/xen/xen-ops.h
> +++ b/arch/x86/xen/xen-ops.h
> @@ -34,6 +34,7 @@ extern unsigned long xen_max_p2m_pfn;
> void xen_set_pat(u64);
>
> char * __init xen_memory_setup(void);
> +char * xen_auto_xlated_memory_setup(void);
> void __init xen_arch_setup(void);
> void xen_enable_sysenter(void);
> void xen_enable_syscall(void);
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 2/2] Revert "xen/pvh: Update E820 to work with PVH (v2)"
2014-06-02 17:09 [RFC PATCHv1 0/2] Linux PVH dom0 memory setup David Vrabel
2014-06-02 17:09 ` [PATCH 1/2] x86/xen: fix memory setup for PVH dom0 David Vrabel
@ 2014-06-02 17:09 ` David Vrabel
2014-06-02 22:46 ` [RFC PATCHv1 0/2] Linux PVH dom0 memory setup Mukesh Rathor
2 siblings, 0 replies; 6+ messages in thread
From: David Vrabel @ 2014-06-02 17:09 UTC (permalink / raw)
To: xen-devel; +Cc: Boris Ostrovsky, David Vrabel
This reverts commit 9103bb0f8240b2a55aac3ff7ecba9c7dcf66b08b.
Now than xen_memory_setup() is not called for auto-translated guests,
we can remove this commit.
Signed-off-by: David Vrabel <david.vrabel@citrix.com>
---
arch/x86/xen/setup.c | 22 ++--------------------
1 file changed, 2 insertions(+), 20 deletions(-)
diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c
index e7ef388..40373d7 100644
--- a/arch/x86/xen/setup.c
+++ b/arch/x86/xen/setup.c
@@ -27,7 +27,6 @@
#include <xen/interface/memory.h>
#include <xen/interface/physdev.h>
#include <xen/features.h>
-#include "mmu.h"
#include "xen-ops.h"
#include "vdso.h"
@@ -82,9 +81,6 @@ static void __init xen_add_extra_mem(u64 start, u64 size)
memblock_reserve(start, size);
- if (xen_feature(XENFEAT_auto_translated_physmap))
- return;
-
xen_max_p2m_pfn = PFN_DOWN(start + size);
for (pfn = PFN_DOWN(start); pfn < xen_max_p2m_pfn; pfn++) {
unsigned long mfn = pfn_to_mfn(pfn);
@@ -107,7 +103,6 @@ static unsigned long __init xen_do_chunk(unsigned long start,
.domid = DOMID_SELF
};
unsigned long len = 0;
- int xlated_phys = xen_feature(XENFEAT_auto_translated_physmap);
unsigned long pfn;
int ret;
@@ -121,7 +116,7 @@ static unsigned long __init xen_do_chunk(unsigned long start,
continue;
frame = mfn;
} else {
- if (!xlated_phys && mfn != INVALID_P2M_ENTRY)
+ if (mfn != INVALID_P2M_ENTRY)
continue;
frame = pfn;
}
@@ -159,13 +154,6 @@ static unsigned long __init xen_do_chunk(unsigned long start,
static unsigned long __init xen_release_chunk(unsigned long start,
unsigned long end)
{
- /*
- * Xen already ballooned out the E820 non RAM regions for us
- * and set them up properly in EPT.
- */
- if (xen_feature(XENFEAT_auto_translated_physmap))
- return end - start;
-
return xen_do_chunk(start, end, true);
}
@@ -234,13 +222,7 @@ static void __init xen_set_identity_and_release_chunk(
* (except for the ISA region which must be 1:1 mapped) to
* release the refcounts (in Xen) on the original frames.
*/
-
- /*
- * PVH E820 matches the hypervisor's P2M which means we need to
- * account for the proper values of *release and *identity.
- */
- for (pfn = start_pfn; !xen_feature(XENFEAT_auto_translated_physmap) &&
- pfn <= max_pfn_mapped && pfn < end_pfn; pfn++) {
+ for (pfn = start_pfn; pfn <= max_pfn_mapped && pfn < end_pfn; pfn++) {
pte_t pte = __pte_ma(0);
if (pfn < PFN_UP(ISA_END_ADDRESS))
--
1.7.10.4
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [RFC PATCHv1 0/2] Linux PVH dom0 memory setup
2014-06-02 17:09 [RFC PATCHv1 0/2] Linux PVH dom0 memory setup David Vrabel
2014-06-02 17:09 ` [PATCH 1/2] x86/xen: fix memory setup for PVH dom0 David Vrabel
2014-06-02 17:09 ` [PATCH 2/2] Revert "xen/pvh: Update E820 to work with PVH (v2)" David Vrabel
@ 2014-06-02 22:46 ` Mukesh Rathor
2 siblings, 0 replies; 6+ messages in thread
From: Mukesh Rathor @ 2014-06-02 22:46 UTC (permalink / raw)
To: David Vrabel; +Cc: xen-devel, Boris Ostrovsky
On Mon, 2 Jun 2014 18:09:41 +0100
David Vrabel <david.vrabel@citrix.com> wrote:
> This is an RFC patch fixing up Linux's memory setup for PVH dom0 after
> the recent Xen change (x86: fix setup of PVH Dom0 memory map).
>
> I've not tested it but this what I would want it to look like (i.e.,
> really simple). Mukesh, Roger can you give it a test, please?
Hi David,
Sorry, it didn't work:
[ 25.546053] audit: type=1404 audit(1401745598.015:2): selinux=0 auid=4294967295 ses=4294967295
[ 25.573154] systemd[1]: segfault at 0 ip 00007f5a9d772210 sp 00007fff0c1b5a28 error 6 in libc-2.17.so[7f5a9d6c3000+1b6000]
[ 25.595626] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[ 25.595626]
[ 25.614155] CPU: 0 PID: 1 Comm: systemd Not tainted 3.15.0-rc3-latest #2
[ 25.627636] Hardware name: Supermicro X8DTN/X8DTN, BIOS 4.6.3 02/26/2010
[ 25.641154] ffffffff81c44600 ffff8800444d1c08 ffffffff816d2501 000000000000fffe
[ 25.656047] ffffffff8196bf88 ffff8800444d1c88 ffffffff816d2131 ffff880000000010
[ 25.670955] ffff8800444d1c98 ffff8800444d1c38 ffff8800444cf270 ffff8800444d1c88
[ 25.685860] Call Trace:
[ 25.690896] [<ffffffff816d2501>] dump_stack+0x51/0x6b
[ 25.701298] [<ffffffff816d2131>] panic+0xbc/0x1f1
[ 25.711007] [<ffffffff8109e4c6>] do_exit+0xb96/0xba0
[ 25.721260] [<ffffffff8109e5e1>] do_group_exit+0x51/0x130
[ 25.732357] [<ffffffff810af9bf>] get_signal_to_deliver+0x23f/0x730
[ 25.745012] [<ffffffff8104c45f>] do_signal+0x4f/0x630
[ 25.755411] [<ffffffff810d4830>] ? set_next_entity+0x90/0xa0
[ 25.767020] [<ffffffff810d9a32>] ? pick_next_task_fair+0x452/0x460
[ 25.779675] [<ffffffff811b987f>] ? kmem_cache_free+0xdf/0x260
[ 25.791460] [<ffffffff816d2377>] ? printk+0x72/0x74
[ 25.801514] [<ffffffff810a51cb>] ? ns_capable+0x2b/0x60
[ 25.812257] [<ffffffff810cbabd>] ? finish_task_switch+0x4d/0x150
[ 25.824566] [<ffffffff816d2f14>] ? __schedule+0x354/0x7f0
[ 25.835658] [<ffffffff81043278>] ? xen_clocksource_read+0x28/0x40
[ 25.848138] [<ffffffff8104ca9d>] do_notify_resume+0x5d/0x80
[ 25.859579] [<ffffffff816db58c>] ? do_page_fault+0x2c/0x40
[ 25.870843] [<ffffffff816d79bc>] retint_signal+0x48/0x8c
I am in the middle of another debug right now, but here's some info that
might help:
BEFORE:
[ 0.000000] Released 99 pages of unused memory
[ 0.000000] Set 265132 page(s) to 1-1 mapping
[ 0.000000] Populating 4b000-4b063 pfn range: 99 pages added
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] Xen: [mem 0x0000000000000000-0x000000000009cfff] usable
[ 0.000000] Xen: [mem 0x000000000009d800-0x00000000000fffff] reserved
[ 0.000000] Xen: [mem 0x0000000000100000-0x00000000bf30cfff] usable
[ 0.000000] Xen: [mem 0x00000000bf30d000-0x00000000bf38cfff] ACPI NVS
[ 0.000000] Xen: [mem 0x00000000bf38d000-0x00000000bf3a2fff] reserved
[ 0.000000] Xen: [mem 0x00000000bf3a3000-0x00000000bf3a3fff] ACPI NVS
[ 0.000000] Xen: [mem 0x00000000bf3a4000-0x00000000bf3b4fff] reserved
[ 0.000000] Xen: [mem 0x00000000bf3b5000-0x00000000bf3b7fff] ACPI NVS
[ 0.000000] Xen: [mem 0x00000000bf3b8000-0x00000000bf3defff] reserved
[ 0.000000] Xen: [mem 0x00000000bf3df000-0x00000000bf3dffff] usable
[ 0.000000] Xen: [mem 0x00000000bf3e0000-0x00000000bf3e0fff] ACPI NVS
[ 0.000000] Xen: [mem 0x00000000bf3e1000-0x00000000bf415fff] reserved
[ 0.000000] Xen: [mem 0x00000000bf416000-0x00000000bf41ffff] ACPI data
[ 0.000000] Xen: [mem 0x00000000bf420000-0x00000000bf420fff] ACPI NVS
[ 0.000000] Xen: [mem 0x00000000bf421000-0x00000000bf422fff] ACPI data
[ 0.000000] Xen: [mem 0x00000000bf423000-0x00000000bf42afff] ACPI NVS
[ 0.000000] Xen: [mem 0x00000000bf42b000-0x00000000bf453fff] reserved
[ 0.000000] Xen: [mem 0x00000000bf454000-0x00000000bf656fff] ACPI NVS
[ 0.000000] Xen: [mem 0x00000000bf657000-0x00000000bf7fffff] usable
[ 0.000000] Xen: [mem 0x00000000c0000000-0x00000000cfffffff] reserved
[ 0.000000] Xen: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
[ 0.000000] Xen: [mem 0x00000000fee00000-0x00000000feefffff] reserved
[ 0.000000] Xen: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[ 0.000000] Xen: [mem 0x0000000100000000-0x00000002bfffffff] usable
[ 0.000000] Malformed early option 'loglevel'
[ 0.000000] debug: ignoring loglevel setting.
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 2.6 present.
[ 0.000000] DMI: Supermicro X8DTN/X8DTN, BIOS 4.6.3 02/26/2010
[ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000000] e820: last_pfn = 0x2c0000 max_arch_pfn = 0x400000000
[ 0.000000] e820: last_pfn = 0xbf800 max_arch_pfn = 0x400000000
[ 0.000000] Scanning 1 areas for low memory corruption
AFTER:
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] Xen: [mem 0x0000000000000000-0x000000000009d7ff] usable
[ 0.000000] Xen: [mem 0x000000000009d800-0x000000000009ffff] reserved
[ 0.000000] Xen: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[ 0.000000] Xen: [mem 0x0000000000100000-0x000000004b061fff] usable
[ 0.000000] Xen: [mem 0x00000000bf30d000-0x00000000bf38cfff] ACPI NVS
[ 0.000000] Xen: [mem 0x00000000bf38d000-0x00000000bf3a2fff] reserved
[ 0.000000] Xen: [mem 0x00000000bf3a3000-0x00000000bf3a3fff] ACPI NVS
[ 0.000000] Xen: [mem 0x00000000bf3a4000-0x00000000bf3b4fff] reserved
[ 0.000000] Xen: [mem 0x00000000bf3b5000-0x00000000bf3b7fff] ACPI NVS
[ 0.000000] Xen: [mem 0x00000000bf3b8000-0x00000000bf3defff] reserved
[ 0.000000] Xen: [mem 0x00000000bf3e0000-0x00000000bf3e0fff] ACPI NVS
[ 0.000000] Xen: [mem 0x00000000bf3e1000-0x00000000bf415fff] reserved
[ 0.000000] Xen: [mem 0x00000000bf416000-0x00000000bf41ffff] ACPI data
[ 0.000000] Xen: [mem 0x00000000bf420000-0x00000000bf420fff] ACPI NVS
[ 0.000000] Xen: [mem 0x00000000bf421000-0x00000000bf422fff] ACPI data
[ 0.000000] Xen: [mem 0x00000000bf423000-0x00000000bf42afff] ACPI NVS
[ 0.000000] Xen: [mem 0x00000000bf42b000-0x00000000bf453fff] reserved
[ 0.000000] Xen: [mem 0x00000000bf454000-0x00000000bf656fff] ACPI NVS
[ 0.000000] Xen: [mem 0x00000000c0000000-0x00000000cfffffff] reserved
[ 0.000000] Xen: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
[ 0.000000] Xen: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[ 0.000000] Xen: [mem 0x0000000180000000-0x00000002bfffffff] unusable
[ 0.000000] Malformed early option 'loglevel'
[ 0.000000] debug: ignoring loglevel setting.
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 2.6 present.
[ 0.000000] DMI: Supermicro X8DTN/X8DTN, BIOS 4.6.3 02/26/2010
[ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000000] e820: last_pfn = 0x4b062 max_arch_pfn = 0x400000000
[ 0.000000] Scanning 1 areas for low memory corruption
I noticed the last_pfn is different, I wonder if that has something
to do with it.
thanks,
Mukesh
^ permalink raw reply [flat|nested] 6+ messages in thread