Re: Xen not working with stock Debian Wheezy 3.2 kernel on a Core 2 Duo box

[David Vrabel <david.vrabel@citrix.com>]

On 08/07/13 12:29, Jan Beulich wrote:
>>>> On 08.07.13 at 12:31, Wei Liu <wei.liu2@citrix.com> wrote:
> 
> $subject is probably the wrong way round, since ...
> 
>> (XEN)  0000000000000000 - 000000000008f000 (usable)
>> (XEN)  000000000008f000 - 00000000000a0000 (reserved)
>> (XEN)  00000000000e0000 - 0000000000100000 (reserved)
>> (XEN)  0000000000100000 - 00000000ce69a000 (usable)
>> (XEN)  00000000ce69a000 - 00000000ce6f1000 (ACPI NVS)
>> (XEN)  00000000ce6f1000 - 00000000cf5fb000 (usable)
>> (XEN)  00000000cf5fb000 - 00000000cf608000 (reserved)
>> (XEN)  00000000cf608000 - 00000000cf6a5000 (usable)
>> (XEN)  00000000cf6a5000 - 00000000cf6aa000 (ACPI data)
>> (XEN)  00000000cf6aa000 - 00000000cf6ab000 (usable)
>> (XEN)  00000000cf6ab000 - 00000000cf6f2000 (ACPI NVS)
>> (XEN)  00000000cf6f2000 - 00000000cf6ff000 (ACPI data)
>> (XEN)  00000000cf6ff000 - 00000000cf700000 (usable)
>> (XEN)  00000000cf700000 - 00000000d0000000 (reserved)
>> (XEN)  00000000fff00000 - 0000000100000000 (reserved)
>> (XEN)  0000000100000000 - 0000000228000000 (usable)
>> (XEN)  0000000228000000 - 000000022c000000 (unusable)
> 
> .. this and ...
> 
>> [    0.000000]  Xen: 0000000000000000 - 000000000008f000 (usable)
>> [    0.000000]  Xen: 000000000008f000 - 0000000000100000 (reserved)
>> [    0.000000]  Xen: 0000000000100000 - 00000000ce69a000 (usable)
>> [    0.000000]  Xen: 00000000ce69a000 - 00000000ce6f1000 (ACPI NVS)
>> [    0.000000]  Xen: 00000000ce6f1000 - 00000000cf5fb000 (usable)
>> [    0.000000]  Xen: 00000000cf5fb000 - 00000000cf608000 (reserved)
>> [    0.000000]  Xen: 00000000cf608000 - 00000000cf6a5000 (usable)
>> [    0.000000]  Xen: 00000000cf6a5000 - 00000000cf6aa000 (ACPI data)
>> [    0.000000]  Xen: 00000000cf6aa000 - 00000000cf6ab000 (usable)
>> [    0.000000]  Xen: 00000000cf6ab000 - 00000000cf6f2000 (ACPI NVS)
>> [    0.000000]  Xen: 00000000cf6f2000 - 00000000cf6ff000 (ACPI data)
>> [    0.000000]  Xen: 00000000cf6ff000 - 00000000cf700000 (usable)
>> [    0.000000]  Xen: 00000000cf700000 - 00000000d0000000 (reserved)
>> [    0.000000]  Xen: 00000000fee00000 - 00000000fee01000 (reserved)
>> [    0.000000]  Xen: 00000000fff00000 - 0000000100000000 (reserved)
>> [    0.000000]  Xen: 0000000100000000 - 0000000228000000 (usable)
>> [    0.000000]  Xen: 0000000228000000 - 000000022c000000 (unusable)
> 
> ... this show that the kernel should be well aware that it shouldn't
> map (or use in any other way) this region.

This came up before when tboot (?) was incorrectly marking a region as
UNUSABLE instead of RESERVED.

Does the following (untested) patch help?

8<---------------------------------------
x86/xen: do not identity map UNUSABLE regions in the machine E820

If there are UNUSABLE regions in the machine memory map, dom0 will
attempt to map them 1:1 which is not permitted by Xen and the kernel
will crash.

There isn't anything interesting in the UNUSABLE region that the dom0
kernel needs access to so we can avoid making the 1:1 mapping and
treat it as RAM.

Signed-off-by: David Vrabel <david.vrabel@citrix.com>
---
 arch/x86/xen/setup.c |   24 ++++++++++++++++++++++++
 1 files changed, 24 insertions(+), 0 deletions(-)

diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c
index 94eac5c..73f621c 100644
--- a/arch/x86/xen/setup.c
+++ b/arch/x86/xen/setup.c
@@ -313,6 +313,17 @@ static void xen_align_and_add_e820_region(u64
start, u64 size, int type)
 	e820_add_region(start, end - start, type);
 }

+void xen_ignore_unusable(struct e820entry *list, size_t map_size)
+{
+	struct e820entry *entry;
+	unsigned int i;
+
+	for (i = 0, entry = list; i < map_size; i++, entry++) {
+		if (entry->type == E820_UNUSABLE)
+			entry->type = E820_RAM;
+	}
+}
+
 /**
  * machine_specific_memory_setup - Hook for machine specific memory setup.
  **/
@@ -353,6 +364,19 @@ char * __init xen_memory_setup(void)
 	}
 	BUG_ON(rc);

+	/*
+	 * Xen won't allow a 1:1 mapping to be created to UNUSABLE
+	 * regions, so if we're using the machine memory map convert
+	 * UNUSABLE to RAM.
+	 *
+	 * This might look odd but what we're really doing here is
+	 * taking the psuedo-physical memory map and punching 1:1
+	 * holes through to interesting bits found in the machine
+	 * memory map.
+	 */
+	if (xen_initial_domain())
+		xen_ignore_unusable(map, memmap.nr_entries);
+
 	/* Make sure the Xen-supplied memory map is well-ordered. */
 	sanitize_e820_map(map, memmap.nr_entries, &memmap.nr_entries);