[PATCH 00/12] xen: initial domain support

[PATCH 00/12] xen: initial domain support

[Stefano Stabellini]

Hi all,
this series implements the basic support needed to boot Linux as initial
domain on Xen: the target is not to add full featured dom0 support in
the kernel but to be able to boot Linux on Xen on native.
We tried to minimize the impact to generic x86 code and interfaces and
as you can see from the diffstat below we were able to reduce them
significantly. The only generic code modifications regard mtrr setups
(see the last two patches in the series).


This series depends on two other patch series not yet applied but
already sent to the list for comments a little while back:

Konrad's "Xen PCI frontend driver" series
http://lkml.org/lkml/2010/8/4/374

my "PV on HVM: receive interrupts as xen events" series
http://lkml.org/lkml/2010/8/30/170


The list of patches with a diffstat follows:

Ian Campbell (1):
      xen: use host E820 map for dom0

Jeremy Fitzhardinge (3):
      xen: remap GSIs as pirqs when running as initial domain
      xen: map a dummy page for local apic and ioapic in xen_set_fixmap
      xen: make hvc_xen console work for dom0.

Juan Quintela (2):
      xen: Initialize xenbus for dom0.
      xen: add the direct mapping area for ISA bus access

Qing He (1):
      xen: remap MSIs into pirqs when running as initial domain

Stefano Stabellini (2):
      xen: use vcpu_ops to setup cpu masks
      xen: introduce XEN_DOM0 as a silent option

Stephen Tweedie (3):
      xen: add support for the platform_ops hypercall
      x86/mtrr: Extend mtrr_if to include num_var_ranges
      xen/mtrr: Add mtrr_if support for Xen mtrr

 arch/x86/include/asm/xen/hypercall.h |    8 ++
 arch/x86/kernel/cpu/mtrr/Makefile    |    2 +-
 arch/x86/kernel/cpu/mtrr/amd.c       |    6 +
 arch/x86/kernel/cpu/mtrr/centaur.c   |    6 +
 arch/x86/kernel/cpu/mtrr/cyrix.c     |    6 +
 arch/x86/kernel/cpu/mtrr/generic.c   |    8 ++
 arch/x86/kernel/cpu/mtrr/main.c      |   20 +---
 arch/x86/kernel/cpu/mtrr/mtrr.h      |    8 ++
 arch/x86/kernel/cpu/mtrr/xen.c       |  110 +++++++++++++++++++
 arch/x86/pci/xen.c                   |  199 ++++++++++++++++++++++++++++++----
 arch/x86/xen/Kconfig                 |    5 +
 arch/x86/xen/enlighten.c             |    1 +
 arch/x86/xen/mmu.c                   |   47 ++++++++-
 arch/x86/xen/setup.c                 |   46 ++++++++-
 arch/x86/xen/smp.c                   |    8 ++-
 drivers/char/hvc_xen.c               |   98 +++++++++++------
 drivers/xen/events.c                 |   73 ++++++++++++-
 drivers/xen/xenbus/xenbus_probe.c    |   29 +++++-
 include/xen/events.h                 |    5 +
 include/xen/interface/memory.h       |   28 +++++
 include/xen/interface/physdev.h      |   10 ++
 include/xen/interface/platform.h     |   96 ++++++++++++++++
 22 files changed, 741 insertions(+), 78 deletions(-)


A git tree with this and the other two series on top of Linux 2.6.36-rc1
is available here:

git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git 2.6.36-rc1-initial-domain

Cheers,

Stefano Stabellini

[PATCH 01/12] xen: remap GSIs as pirqs when running as initial domain

[stefano.stabellini]

From: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>

Implement xen_register_gsi to setup the correct triggering and polarity
properties of a gsi.
Implement xen_register_pirq to register a particular gsi as pirq and
receive interrupts as events.
Call xen_setup_pirqs to register all the legacy ISA irqs as pirqs.

Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 arch/x86/pci/xen.c              |  144 ++++++++++++++++++++++++++++++++++++++-
 drivers/xen/events.c            |   13 ++++
 include/xen/events.h            |    2 +
 include/xen/interface/physdev.h |   10 +++
 4 files changed, 168 insertions(+), 1 deletions(-)

diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
index 6fbc81a..cb8d5be 100644
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -18,6 +18,108 @@
 #include <xen/events.h>
 #include <asm/xen/pci.h>
 
+static int xen_register_pirq(u32 gsi, int triggering)
+{
+	int rc, irq;
+	struct physdev_map_pirq map_irq;
+	int shareable = 0;
+	char *name;
+
+	if (!xen_pv_domain())
+		return -1;
+
+	if (triggering == ACPI_EDGE_SENSITIVE) {
+		shareable = 0;
+		name = "ioapic-edge";
+	} else {
+		shareable = 1;
+		name = "ioapic-level";
+	}
+
+	irq = xen_allocate_pirq(gsi, shareable, name);
+
+	printk(KERN_DEBUG "xen: --> irq=%d\n", irq);
+
+	if (irq < 0)
+		goto out;
+
+	map_irq.domid = DOMID_SELF;
+	map_irq.type = MAP_PIRQ_TYPE_GSI;
+	map_irq.index = gsi;
+	map_irq.pirq = irq;
+
+	rc = HYPERVISOR_physdev_op(PHYSDEVOP_map_pirq, &map_irq);
+	if (rc) {
+		printk(KERN_WARNING "xen map irq failed %d\n", rc);
+		return -1;
+	}
+
+out:
+	return irq;
+}
+
+static int xen_register_gsi(u32 gsi, int triggering, int polarity)
+{
+	int rc, irq;
+	struct physdev_setup_gsi setup_gsi;
+
+	if (!xen_pv_domain())
+		return -1;
+
+	printk(KERN_DEBUG "xen: registering gsi %u triggering %d polarity %d\n",
+			gsi, triggering, polarity);
+
+	irq = xen_register_pirq(gsi, triggering);
+
+	setup_gsi.gsi = gsi;
+	setup_gsi.triggering = (triggering == ACPI_EDGE_SENSITIVE ? 0 : 1);
+	setup_gsi.polarity = (polarity == ACPI_ACTIVE_HIGH ? 0 : 1);
+
+	rc = HYPERVISOR_physdev_op(PHYSDEVOP_setup_gsi, &setup_gsi);
+	if (rc == -EEXIST)
+		printk(KERN_INFO "Already setup the GSI :%d\n", gsi);
+	else if (rc) {
+		printk(KERN_ERR "Failed to setup GSI :%d, err_code:%d\n",
+				gsi, rc);
+	}
+
+	return irq;
+}
+
+#ifdef CONFIG_ACPI
+static __init void xen_setup_acpi_sci(void)
+{
+	int rc;
+	int trigger, polarity;
+	int gsi = acpi_sci_override_gsi;
+
+	if (!gsi)
+		return;
+
+	rc = acpi_get_override_irq(gsi, &trigger, &polarity);
+	if (rc)
+		return;
+	trigger = trigger ? ACPI_LEVEL_SENSITIVE : ACPI_EDGE_SENSITIVE;
+	polarity = polarity ? ACPI_ACTIVE_LOW : ACPI_ACTIVE_HIGH;
+	
+	printk("xen: sci override: global_irq=%d trigger=%d polarity=%d\n",
+			gsi, trigger, polarity);
+
+	gsi = xen_register_gsi(gsi,	trigger, polarity);
+	/*
+	 * stash over-ride to indicate we've been here
+	 * and for later update of acpi_gbl_FADT
+	 */
+	printk("xen: acpi sci %d\n", gsi);
+
+	return;
+}
+#else
+static __init void xen_setup_acpi_sci(void)
+{
+}
+#endif
+
 int xen_hvm_register_pirq(u32 gsi, int triggering)
 {
 	int rc, irq;
@@ -211,10 +313,15 @@ static int acpi_register_gsi_xen_hvm(struct device *dev, u32 gsi,
 	return xen_hvm_register_pirq(gsi, trigger);
 }
 
+static int acpi_register_gsi_xen(struct device *dev, u32 gsi,
+				 int trigger, int polarity)
+{
+	return xen_register_gsi(gsi, trigger, polarity);
+}
 
 int __init pci_xen_init(void)
 {
-	if (!xen_pv_domain() || xen_initial_domain())
+ 	if (!xen_pv_domain() || xen_initial_domain())
 		return -ENODEV;
 
 	printk(KERN_INFO "PCI: setting up Xen PCI frontend stub\n");
@@ -242,6 +349,41 @@ int __init pci_xen_init(void)
 	return 0;
 }
 
+static int __init pci_xen_initial_domain(void)
+{
+	xen_setup_acpi_sci();
+	__acpi_register_gsi = acpi_register_gsi_xen;
+
+	return 0;
+}
+
+void __init xen_setup_pirqs(void)
+{
+	int irq;
+#ifndef CONFIG_SMP
+	int nr_ioapics = 1;
+#endif
+
+	pci_xen_initial_domain();
+
+	if (0 == nr_ioapics) {
+		for (irq = 0; irq < NR_IRQS_LEGACY; irq++)
+			xen_allocate_pirq(irq, 0, "xt-pic");
+		return;
+	}
+
+	/* Pre-allocate legacy irqs */
+	for (irq = 0; irq < NR_IRQS_LEGACY; irq++) {
+		int trigger, polarity;
+
+		if (acpi_get_override_irq(irq, &trigger, &polarity) == -1)
+			continue;
+
+		xen_register_pirq(irq,
+			trigger ? ACPI_LEVEL_SENSITIVE : ACPI_EDGE_SENSITIVE);
+	}
+}
+
 int __init pci_xen_hvm_init(void)
 {
 	if (!xen_feature(XENFEAT_hvm_pirqs))
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index a31677e..27e4b70 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -676,6 +676,8 @@ out:
 int xen_destroy_irq(int irq)
 {
 	struct irq_desc *desc;
+	struct physdev_unmap_pirq unmap_irq;
+	struct irq_info *info = info_for_irq(irq);
 	int rc = -ENOENT;
 
 	spin_lock(&irq_mapping_update_lock);
@@ -684,6 +686,15 @@ int xen_destroy_irq(int irq)
 	if (!desc)
 		goto out;
 
+	if (xen_initial_domain()) {
+		unmap_irq.pirq = info->u.pirq.gsi;
+		unmap_irq.domid = DOMID_SELF;
+		rc = HYPERVISOR_physdev_op(PHYSDEVOP_unmap_pirq, &unmap_irq);
+		if (rc) {
+			printk(KERN_WARNING "unmap irq failed %d\n", rc);
+			goto out;
+		}
+	}
 	irq_info[irq] = mk_unbound_info();
 
 	dynamic_irq_cleanup(irq);
@@ -1410,5 +1421,7 @@ void __init xen_init_IRQ(void)
 		pci_xen_hvm_init();
 	} else {
 		irq_ctx_init(smp_processor_id());
+		if (xen_initial_domain())
+			xen_setup_pirqs();
 	}
 }
diff --git a/include/xen/events.h b/include/xen/events.h
index 5e67be7..9853fce 100644
--- a/include/xen/events.h
+++ b/include/xen/events.h
@@ -84,4 +84,6 @@ int xen_vector_from_irq(unsigned pirq);
 /* Return gsi allocated to pirq */
 int xen_gsi_from_irq(unsigned pirq);
 
+void xen_setup_pirqs(void);
+
 #endif	/* _XEN_EVENTS_H */
diff --git a/include/xen/interface/physdev.h b/include/xen/interface/physdev.h
index 69a72b9..a85d76c 100644
--- a/include/xen/interface/physdev.h
+++ b/include/xen/interface/physdev.h
@@ -151,6 +151,16 @@ struct physdev_op {
 	} u;
 };
 
+#define PHYSDEVOP_setup_gsi    21
+struct physdev_setup_gsi {
+    int gsi;
+    /* IN */
+    uint8_t triggering;
+    /* IN */
+    uint8_t polarity;
+    /* IN */
+};
+
 #define PHYSDEVOP_get_nr_pirqs    22
 struct physdev_nr_pirqs {
     /* OUT */
-- 
1.5.6.5

[PATCH 02/12] xen: remap MSIs into pirqs when running as initial domain

[stefano.stabellini]

From: Qing He <qing.he@intel.com>

Implement xen_create_msi_irq to create an msi and remap it as pirq.
Use xen_create_msi_irq to implement an initial domain specific version
of setup_msi_irqs.

Signed-off-by: Qing He <qing.he@intel.com>
Signed-off-by: Yunhong Jiang <yunhong.jiang@intel.com>
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 arch/x86/pci/xen.c   |   55 +++++++++++++++++++++++++++++++---------------
 drivers/xen/events.c |   58 ++++++++++++++++++++++++++++++++++++++++++++++++++
 include/xen/events.h |    2 +
 3 files changed, 97 insertions(+), 18 deletions(-)

diff --git a/arch/x86/pci/xen.c b/arch/x86/pci/xen.c
index cb8d5be..8453e36 100644
--- a/arch/x86/pci/xen.c
+++ b/arch/x86/pci/xen.c
@@ -229,14 +229,12 @@ static int xen_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
 	if (!v)
 		return -ENOMEM;
 
-	if (!xen_initial_domain()) {
-		if (type == PCI_CAP_ID_MSIX)
-			ret = xen_pci_frontend_enable_msix(dev, &v, nvec);
-		else
-			ret = xen_pci_frontend_enable_msi(dev, &v);
-		if (ret)
-			goto error;
-	}
+	if (type == PCI_CAP_ID_MSIX)
+		ret = xen_pci_frontend_enable_msix(dev, &v, nvec);
+	else
+		ret = xen_pci_frontend_enable_msi(dev, &v);
+	if (ret)
+		goto error;
 	i = 0;
 	list_for_each_entry(msidesc, &dev->msi_list, list) {
 		irq = xen_allocate_pirq(v[i], 0, /* not sharable */
@@ -266,23 +264,40 @@ error:
 
 static void xen_teardown_msi_irqs(struct pci_dev *dev)
 {
-	/* Only do this when were are in non-privileged mode.*/
-	if (!xen_initial_domain()) {
-		struct msi_desc *msidesc;
-
-		msidesc = list_entry(dev->msi_list.next, struct msi_desc, list);
-		if (msidesc->msi_attrib.is_msix)
-			xen_pci_frontend_disable_msix(dev);
-		else
-			xen_pci_frontend_disable_msi(dev);
-	}
+	struct msi_desc *msidesc;
 
+	msidesc = list_entry(dev->msi_list.next, struct msi_desc, list);
+	if (msidesc->msi_attrib.is_msix)
+		xen_pci_frontend_disable_msix(dev);
+	else
+		xen_pci_frontend_disable_msi(dev);
 }
 
 static void xen_teardown_msi_irq(unsigned int irq)
 {
 	xen_destroy_irq(irq);
 }
+
+int xen_initdom_setup_msi_irqs(struct pci_dev *dev, int nvec, int type)
+{
+	int irq, ret;
+	struct msi_desc *msidesc;
+
+	list_for_each_entry(msidesc, &dev->msi_list, list) {
+		irq = xen_create_msi_irq(dev, msidesc, type);
+		if (irq < 0)
+			return -1;
+
+		ret = set_irq_msi(irq, msidesc);
+		if (ret)
+			goto error;
+	}
+	return 0;
+
+error:
+	xen_destroy_irq(irq);
+	return ret;
+}
 #endif
 
 static int xen_pcifront_enable_irq(struct pci_dev *dev)
@@ -351,6 +366,10 @@ int __init pci_xen_init(void)
 
 static int __init pci_xen_initial_domain(void)
 {
+#ifdef CONFIG_PCI_MSI
+	x86_msi.setup_msi_irqs = xen_initdom_setup_msi_irqs;
+	x86_msi.teardown_msi_irq = xen_teardown_msi_irq;
+#endif
 	xen_setup_acpi_sci();
 	__acpi_register_gsi = acpi_register_gsi_xen;
 
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index 27e4b70..b83918a 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -29,6 +29,8 @@
 #include <linux/bootmem.h>
 #include <linux/slab.h>
 #include <linux/irqnr.h>
+#include <linux/pci.h>
+#include <linux/msi.h>
 
 #include <asm/desc.h>
 #include <asm/ptrace.h>
@@ -49,6 +51,8 @@
 #include <xen/interface/hvm/hvm_op.h>
 #include <xen/interface/hvm/params.h>
 
+#include "../pci/msi.h"
+
 /*
  * This lock protects updates to the following mapping and reference-count
  * arrays. The lock does not need to be acquired to read the mapping tables.
@@ -704,6 +708,60 @@ out:
 	return rc;
 }
 
+int xen_create_msi_irq(struct pci_dev *dev, struct msi_desc *msidesc, int type)
+{
+	int irq = 0;
+	struct physdev_map_pirq map_irq;
+	int rc;
+	int pos;
+	u32 table_offset, bir;
+
+	memset(&map_irq, 0, sizeof(map_irq));
+	map_irq.domid = DOMID_SELF;
+	map_irq.type = MAP_PIRQ_TYPE_MSI;
+	map_irq.index = -1;
+	map_irq.pirq = -1;
+	map_irq.bus = dev->bus->number;
+	map_irq.devfn = dev->devfn;
+
+	if (type == PCI_CAP_ID_MSIX) {
+		pos = pci_find_capability(dev, PCI_CAP_ID_MSIX);
+
+		pci_read_config_dword(dev, msix_table_offset_reg(pos),
+					&table_offset);
+		bir = (u8)(table_offset & PCI_MSIX_FLAGS_BIRMASK);
+
+		map_irq.table_base = pci_resource_start(dev, bir);
+		map_irq.entry_nr = msidesc->msi_attrib.entry_nr;
+	}
+
+	spin_lock(&irq_mapping_update_lock);
+
+	irq = find_unbound_irq();
+
+	if (irq == -1)
+		goto out;
+
+	rc = HYPERVISOR_physdev_op(PHYSDEVOP_map_pirq, &map_irq);
+	if (rc) {
+		printk(KERN_WARNING "xen map irq failed %d\n", rc);
+
+		dynamic_irq_cleanup(irq);
+
+		irq = -1;
+		goto out;
+	}
+	irq_info[irq] = mk_pirq_info(0, map_irq.pirq, 0, map_irq.index);
+
+	set_irq_chip_and_handler_name(irq, &xen_pirq_chip,
+			handle_level_irq,
+			(type == PCI_CAP_ID_MSIX) ? "msi-x":"msi");
+
+out:
+	spin_unlock(&irq_mapping_update_lock);
+	return irq;
+}
+
 int xen_vector_from_irq(unsigned irq)
 {
 	return vector_from_irq(irq);
diff --git a/include/xen/events.h b/include/xen/events.h
index 9853fce..8cd18a5 100644
--- a/include/xen/events.h
+++ b/include/xen/events.h
@@ -78,6 +78,8 @@ void xen_allocate_pirq_msi(char *name, int *irq, int *pirq);
 /* De-allocates the above mentioned physical interrupt. */
 int xen_destroy_irq(int irq);
 
+int xen_create_msi_irq(struct pci_dev *dev, struct msi_desc *msidesc, int type);
+
 /* Return vector allocated to pirq */
 int xen_vector_from_irq(unsigned pirq);
 
-- 
1.5.6.5

[PATCH 03/12] xen: map a dummy page for local apic and ioapic in xen_set_fixmap

[stefano.stabellini]

From: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>

Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 arch/x86/xen/mmu.c |   23 ++++++++++++++++++++---
 1 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
index 42086ac..ffc5e24 100644
--- a/arch/x86/xen/mmu.c
+++ b/arch/x86/xen/mmu.c
@@ -1861,6 +1861,8 @@ __init pgd_t *xen_setup_kernel_pagetable(pgd_t *pgd,
 }
 #endif	/* CONFIG_X86_64 */
 
+static unsigned char dummy_mapping[PAGE_SIZE] __page_aligned_bss;
+
 static void xen_set_fixmap(unsigned idx, phys_addr_t phys, pgprot_t prot)
 {
 	pte_t pte;
@@ -1881,15 +1883,28 @@ static void xen_set_fixmap(unsigned idx, phys_addr_t phys, pgprot_t prot)
 #else
 	case VSYSCALL_LAST_PAGE ... VSYSCALL_FIRST_PAGE:
 #endif
-#ifdef CONFIG_X86_LOCAL_APIC
-	case FIX_APIC_BASE:	/* maps dummy local APIC */
-#endif
 	case FIX_TEXT_POKE0:
 	case FIX_TEXT_POKE1:
 		/* All local page mappings */
 		pte = pfn_pte(phys, prot);
 		break;
 
+#ifdef CONFIG_X86_LOCAL_APIC
+	case FIX_APIC_BASE:	/* maps dummy local APIC */
+		pte = pfn_pte(PFN_DOWN(__pa(dummy_mapping)), PAGE_KERNEL);
+		break;
+#endif
+
+#ifdef CONFIG_X86_IO_APIC
+	case FIX_IO_APIC_BASE_0 ... FIX_IO_APIC_BASE_END:
+		/*
+		 * We just don't map the IO APIC - all access is via
+		 * hypercalls.  Keep the address in the pte for reference.
+		 */
+		pte = pfn_pte(PFN_DOWN(__pa(dummy_mapping)), PAGE_KERNEL);
+		break;
+#endif
+
 	case FIX_PARAVIRT_BOOTMAP:
 		/* This is an MFN, but it isn't an IO mapping from the
 		   IO domain */
@@ -2027,6 +2042,8 @@ void __init xen_init_mmu_ops(void)
 	pv_mmu_ops = xen_mmu_ops;
 
 	vmap_lazy_unmap = false;
+
+	memset(dummy_mapping, 0xff, PAGE_SIZE);
 }
 
 /* Protected by xen_reservation_lock. */
-- 
1.5.6.5

[PATCH 04/12] xen: use vcpu_ops to setup cpu masks

[stefano.stabellini]

From: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 arch/x86/xen/smp.c |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)

diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index 25f232b..1386767 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -156,11 +156,16 @@ static void __init xen_fill_possible_map(void)
 {
 	int i, rc;
 
+	num_processors = 0;
+	disabled_cpus = 0;
 	for (i = 0; i < nr_cpu_ids; i++) {
 		rc = HYPERVISOR_vcpu_op(VCPUOP_is_up, i, NULL);
 		if (rc >= 0) {
 			num_processors++;
 			set_cpu_possible(i, true);
+		} else {
+			set_cpu_possible(i, false);
+			set_cpu_present(i, false);
 		}
 	}
 }
@@ -190,6 +195,8 @@ static void __init xen_smp_prepare_cpus(unsigned int max_cpus)
 	if (xen_smp_intr_init(0))
 		BUG();
 
+	xen_fill_possible_map();
+
 	if (!alloc_cpumask_var(&xen_cpu_initialized_map, GFP_KERNEL))
 		panic("could not allocate xen_cpu_initialized_map\n");
 
@@ -480,6 +487,5 @@ static const struct smp_ops xen_smp_ops __initdata = {
 void __init xen_smp_init(void)
 {
 	smp_ops = xen_smp_ops;
-	xen_fill_possible_map();
 	xen_init_spinlocks();
 }
-- 
1.5.6.5

[PATCH 05/12] xen: Initialize xenbus for dom0.

[stefano.stabellini]

From: Juan Quintela <quintela@redhat.com>

Do initial xenbus/xenstore setup in dom0.  In dom0 we need to actually
allocate the xenstore resources, rather than being given them from
outside.

[ Impact: initialize Xenbus ]

Signed-off-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 drivers/xen/xenbus/xenbus_probe.c |   29 ++++++++++++++++++++++++++++-
 1 files changed, 28 insertions(+), 1 deletions(-)

diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c
index 29bac51..df0049e 100644
--- a/drivers/xen/xenbus/xenbus_probe.c
+++ b/drivers/xen/xenbus/xenbus_probe.c
@@ -798,6 +798,7 @@ device_initcall(xenbus_probe_initcall);
 static int __init xenbus_init(void)
 {
 	int err = 0;
+	unsigned long page = 0;
 
 	DPRINTK("");
 
@@ -818,7 +819,31 @@ static int __init xenbus_init(void)
 	 * Domain0 doesn't have a store_evtchn or store_mfn yet.
 	 */
 	if (xen_initial_domain()) {
-		/* dom0 not yet supported */
+		struct evtchn_alloc_unbound alloc_unbound;
+
+		/* Allocate Xenstore page */
+		page = get_zeroed_page(GFP_KERNEL);
+		if (!page)
+			goto out_error;
+
+		xen_store_mfn = xen_start_info->store_mfn =
+			pfn_to_mfn(virt_to_phys((void *)page) >>
+				   PAGE_SHIFT);
+
+		/* Next allocate a local port which xenstored can bind to */
+		alloc_unbound.dom        = DOMID_SELF;
+		alloc_unbound.remote_dom = 0;
+
+		err = HYPERVISOR_event_channel_op(EVTCHNOP_alloc_unbound,
+						  &alloc_unbound);
+		if (err == -ENOSYS)
+			goto out_error;
+
+		BUG_ON(err);
+		xen_store_evtchn = xen_start_info->store_evtchn =
+			alloc_unbound.port;
+
+		xen_store_interface = mfn_to_virt(xen_store_mfn);
 	} else {
 		if (xen_hvm_domain()) {
 			uint64_t v = 0;
@@ -864,6 +889,8 @@ static int __init xenbus_init(void)
 	bus_unregister(&xenbus_frontend.bus);
 
   out_error:
+	if (page != 0)
+		free_page(page);
 	return err;
 }
 
-- 
1.5.6.5

[PATCH 06/12] xen: add the direct mapping area for ISA bus access

[stefano.stabellini]

From: Juan Quintela <quintela@redhat.com>

add the direct mapping area for ISA bus access when running as initial
domain

Signed-off-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 arch/x86/xen/enlighten.c |    1 +
 arch/x86/xen/mmu.c       |   24 ++++++++++++++++++++++++
 arch/x86/xen/setup.c     |    3 +++
 3 files changed, 28 insertions(+), 0 deletions(-)

diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index 1ccfa1b..9efb004 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -1186,6 +1186,7 @@ asmlinkage void __init xen_start_kernel(void)
 
 	xen_raw_console_write("mapping kernel into physical memory\n");
 	pgd = xen_setup_kernel_pagetable(pgd, xen_start_info->nr_pages);
+	xen_ident_map_ISA();
 
 	init_mm.pgd = pgd;
 
diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
index ffc5e24..eed9c7c 100644
--- a/arch/x86/xen/mmu.c
+++ b/arch/x86/xen/mmu.c
@@ -1682,6 +1682,7 @@ static void *m2v(phys_addr_t maddr)
 	return __ka(m2p(maddr));
 }
 
+/* Set the page permissions on an identity-mapped pages */
 static void set_page_prot(void *addr, pgprot_t prot)
 {
 	unsigned long pfn = __pa(addr) >> PAGE_SHIFT;
@@ -1929,6 +1930,29 @@ static void xen_set_fixmap(unsigned idx, phys_addr_t phys, pgprot_t prot)
 #endif
 }
 
+__init void xen_ident_map_ISA(void)
+{
+	unsigned long pa;
+
+	/*
+	 * If we're dom0, then linear map the ISA machine addresses into
+	 * the kernel's address space.
+	 */
+	if (!xen_initial_domain())
+		return;
+
+	xen_raw_printk("Xen: setup ISA identity maps\n");
+
+	for (pa = ISA_START_ADDRESS; pa < ISA_END_ADDRESS; pa += PAGE_SIZE) {
+		pte_t pte = mfn_pte(PFN_DOWN(pa), PAGE_KERNEL_IO);
+
+		if (HYPERVISOR_update_va_mapping(PAGE_OFFSET + pa, pte, 0))
+			BUG();
+	}
+
+	xen_flush_tlb();
+}
+
 static __init void xen_post_allocator_init(void)
 {
 	pv_mmu_ops.set_pte = xen_set_pte;
diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c
index c413132..62ceb78 100644
--- a/arch/x86/xen/setup.c
+++ b/arch/x86/xen/setup.c
@@ -119,6 +119,9 @@ char * __init xen_memory_setup(void)
 	 * Even though this is normal, usable memory under Xen, reserve
 	 * ISA memory anyway because too many things think they can poke
 	 * about in there.
+	 *
+	 * In a dom0 kernel, this region is identity mapped with the
+	 * hardware ISA area, so it really is out of bounds.
 	 */
 	e820_add_region(ISA_START_ADDRESS, ISA_END_ADDRESS - ISA_START_ADDRESS,
 			E820_RESERVED);
-- 
1.5.6.5

[PATCH 07/12] xen: introduce XEN_DOM0 as a silent option

[stefano.stabellini]

From: Stefano Stabellini <stefano.stabellini@eu.citrix.com>

Add XEN_DOM0 to arch/x86/xen/Kconfig as a silent compile time option
that gets enabled when xen and basic x86, acpi and pci support are
selected.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 arch/x86/xen/Kconfig |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/arch/x86/xen/Kconfig b/arch/x86/xen/Kconfig
index 68128a1..9bbb06b 100644
--- a/arch/x86/xen/Kconfig
+++ b/arch/x86/xen/Kconfig
@@ -13,6 +13,11 @@ config XEN
 	  kernel to boot in a paravirtualized environment under the
 	  Xen hypervisor.
 
+config XEN_DOM0
+	def_bool y
+	depends on XEN && PCI_XEN && SWIOTLB_XEN
+	depends on X86_LOCAL_APIC && X86_IO_APIC && ACPI && PCI
+
 config XEN_PVHVM
 	def_bool y
 	depends on XEN
-- 
1.5.6.5

[PATCH 08/12] xen: use host E820 map for dom0

[stefano.stabellini]

From: Ian Campbell <ian.campbell@citrix.com>

When running as initial domain, get the real physical memory map from
xen using the XENMEM_machine_memory_map hypercall and use it to setup
the e820 regions.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 arch/x86/xen/setup.c           |   43 +++++++++++++++++++++++++++++++++++++--
 include/xen/interface/memory.h |   28 ++++++++++++++++++++++++++
 2 files changed, 68 insertions(+), 3 deletions(-)

diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c
index 62ceb78..b08aac2 100644
--- a/arch/x86/xen/setup.c
+++ b/arch/x86/xen/setup.c
@@ -107,14 +107,51 @@ static unsigned long __init xen_return_unused_memory(unsigned long max_pfn,
 
 char * __init xen_memory_setup(void)
 {
+	static __initdata struct e820entry map[E820MAX];
+
 	unsigned long max_pfn = xen_start_info->nr_pages;
+	struct xen_memory_map memmap;
+	unsigned long long mem_end;
+	int op;
+	int rc;
+	int i;
 
 	max_pfn = min(MAX_DOMAIN_PAGES, max_pfn);
+	mem_end = PFN_PHYS((u64)max_pfn);
+
+	memmap.nr_entries = E820MAX;
+	set_xen_guest_handle(memmap.buffer, map);
+
+	op = xen_initial_domain() ?
+		XENMEM_machine_memory_map :
+		XENMEM_memory_map;
+	rc = HYPERVISOR_memory_op(op, &memmap);
+	if (rc == -ENOSYS) {
+		memmap.nr_entries = 1;
+		map[0].addr = 0ULL;
+		map[0].size = mem_end;
+		/* 8MB slack (to balance backend allocations). */
+		map[0].size += 8ULL << 20;
+		map[0].type = E820_RAM;
+		rc = 0;
+	}
+	BUG_ON(rc);
 
 	e820.nr_map = 0;
-
-	e820_add_region(0, PFN_PHYS((u64)max_pfn), E820_RAM);
-
+	for (i = 0; i < memmap.nr_entries; i++) {
+		unsigned long long end = map[i].addr + map[i].size;
+		if (map[i].type == E820_RAM) {
+			if (map[i].addr > mem_end)
+				continue;
+			if (end > mem_end) {
+				/* Truncate region to max_mem. */
+				map[i].size -= end - mem_end;
+			}
+		}
+		if (map[i].size > 0)
+			e820_add_region(map[i].addr, map[i].size, map[i].type);
+	}
+	
 	/*
 	 * Even though this is normal, usable memory under Xen, reserve
 	 * ISA memory anyway because too many things think they can poke
diff --git a/include/xen/interface/memory.h b/include/xen/interface/memory.h
index d3938d3..4c4b817 100644
--- a/include/xen/interface/memory.h
+++ b/include/xen/interface/memory.h
@@ -186,6 +186,34 @@ struct xen_translate_gpfn_list {
 };
 DEFINE_GUEST_HANDLE_STRUCT(xen_translate_gpfn_list);
 
+/*
+ * Returns the pseudo-physical memory map as it was when the domain
+ * was started (specified by XENMEM_set_memory_map).
+ * arg == addr of struct xen_memory_map.
+ */
+#define XENMEM_memory_map           9
+struct xen_memory_map {
+    /*
+     * On call the number of entries which can be stored in buffer. On
+     * return the number of entries which have been stored in
+     * buffer.
+     */
+    unsigned int nr_entries;
+
+    /*
+     * Entries in the buffer are in the same format as returned by the
+     * BIOS INT 0x15 EAX=0xE820 call.
+     */
+    GUEST_HANDLE(void) buffer;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_memory_map);
+
+/*
+ * Returns the real physical memory map. Passes the same structure as
+ * XENMEM_memory_map.
+ * arg == addr of struct xen_memory_map.
+ */
+#define XENMEM_machine_memory_map   10
 
 /*
  * Prevent the balloon driver from changing the memory reservation
-- 
1.5.6.5

[PATCH 09/12] xen: make hvc_xen console work for dom0.

[stefano.stabellini]

From: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>

Use the console hypercalls for dom0 console.

[ Impact: Add Xen dom0 console ]
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 drivers/char/hvc_xen.c |   98 ++++++++++++++++++++++++++++++++----------------
 drivers/xen/events.c   |    2 +-
 include/xen/events.h   |    1 +
 3 files changed, 67 insertions(+), 34 deletions(-)

diff --git a/drivers/char/hvc_xen.c b/drivers/char/hvc_xen.c
index 60446f8..cc21c97 100644
--- a/drivers/char/hvc_xen.c
+++ b/drivers/char/hvc_xen.c
@@ -78,7 +78,7 @@ static int __write_console(const char *data, int len)
 	return sent;
 }
 
-static int write_console(uint32_t vtermno, const char *data, int len)
+static int domU_write_console(uint32_t vtermno, const char *data, int len)
 {
 	int ret = len;
 
@@ -101,7 +101,7 @@ static int write_console(uint32_t vtermno, const char *data, int len)
 	return ret;
 }
 
-static int read_console(uint32_t vtermno, char *buf, int len)
+static int domU_read_console(uint32_t vtermno, char *buf, int len)
 {
 	struct xencons_interface *intf = xencons_interface();
 	XENCONS_RING_IDX cons, prod;
@@ -122,28 +122,62 @@ static int read_console(uint32_t vtermno, char *buf, int len)
 	return recv;
 }
 
-static const struct hv_ops hvc_ops = {
-	.get_chars = read_console,
-	.put_chars = write_console,
+static struct hv_ops domU_hvc_ops = {
+	.get_chars = domU_read_console,
+	.put_chars = domU_write_console,
 	.notifier_add = notifier_add_irq,
 	.notifier_del = notifier_del_irq,
 	.notifier_hangup = notifier_hangup_irq,
 };
 
-static int __init xen_init(void)
+static int dom0_read_console(uint32_t vtermno, char *buf, int len)
+{
+	return HYPERVISOR_console_io(CONSOLEIO_read, len, buf);
+}
+
+/*
+ * Either for a dom0 to write to the system console, or a domU with a
+ * debug version of Xen
+ */
+static int dom0_write_console(uint32_t vtermno, const char *str, int len)
+{
+	int rc = HYPERVISOR_console_io(CONSOLEIO_write, len, (char *)str);
+	if (rc < 0)
+		return 0;
+
+	return len;
+}
+
+static struct hv_ops dom0_hvc_ops = {
+	.get_chars = dom0_read_console,
+	.put_chars = dom0_write_console,
+	.notifier_add = notifier_add_irq,
+	.notifier_del = notifier_del_irq,
+	.notifier_hangup = notifier_hangup_irq,
+};
+
+static int __init xen_hvc_init(void)
 {
 	struct hvc_struct *hp;
+	struct hv_ops *ops;
 
-	if (!xen_pv_domain() ||
-	    xen_initial_domain() ||
-	    !xen_start_info->console.domU.evtchn)
+	if (!xen_pv_domain())
 		return -ENODEV;
 
-	xencons_irq = bind_evtchn_to_irq(xen_start_info->console.domU.evtchn);
+	if (xen_initial_domain()) {
+		ops = &dom0_hvc_ops;
+		xencons_irq = bind_virq_to_irq(VIRQ_CONSOLE, 0);
+	} else {
+		if (!xen_start_info->console.domU.evtchn)
+			return -ENODEV;
+
+		ops = &domU_hvc_ops;
+		xencons_irq = bind_evtchn_to_irq(xen_start_info->console.domU.evtchn);
+	}
 	if (xencons_irq < 0)
 		xencons_irq = 0; /* NO_IRQ */
 
-	hp = hvc_alloc(HVC_COOKIE, xencons_irq, &hvc_ops, 256);
+	hp = hvc_alloc(HVC_COOKIE, xencons_irq, ops, 256);
 	if (IS_ERR(hp))
 		return PTR_ERR(hp);
 
@@ -160,7 +194,7 @@ void xen_console_resume(void)
 		rebind_evtchn_irq(xen_start_info->console.domU.evtchn, xencons_irq);
 }
 
-static void __exit xen_fini(void)
+static void __exit xen_hvc_fini(void)
 {
 	if (hvc)
 		hvc_remove(hvc);
@@ -168,29 +202,24 @@ static void __exit xen_fini(void)
 
 static int xen_cons_init(void)
 {
+	struct hv_ops *ops;
+
 	if (!xen_pv_domain())
 		return 0;
 
-	hvc_instantiate(HVC_COOKIE, 0, &hvc_ops);
+	if (xen_initial_domain())
+		ops = &dom0_hvc_ops;
+	else
+		ops = &domU_hvc_ops;
+
+	hvc_instantiate(HVC_COOKIE, 0, ops);
 	return 0;
 }
 
-module_init(xen_init);
-module_exit(xen_fini);
+module_init(xen_hvc_init);
+module_exit(xen_hvc_fini);
 console_initcall(xen_cons_init);
 
-static void raw_console_write(const char *str, int len)
-{
-	while(len > 0) {
-		int rc = HYPERVISOR_console_io(CONSOLEIO_write, len, (char *)str);
-		if (rc <= 0)
-			break;
-
-		str += rc;
-		len -= rc;
-	}
-}
-
 #ifdef CONFIG_EARLY_PRINTK
 static void xenboot_write_console(struct console *console, const char *string,
 				  unsigned len)
@@ -198,19 +227,22 @@ static void xenboot_write_console(struct console *console, const char *string,
 	unsigned int linelen, off = 0;
 	const char *pos;
 
-	raw_console_write(string, len);
+	dom0_write_console(0, string, len);
+
+	if (xen_initial_domain())
+		return;
 
-	write_console(0, "(early) ", 8);
+	dumU_write_console(0, "(early) ", 8);
 	while (off < len && NULL != (pos = strchr(string+off, '\n'))) {
 		linelen = pos-string+off;
 		if (off + linelen > len)
 			break;
-		write_console(0, string+off, linelen);
-		write_console(0, "\r\n", 2);
+		dumU_write_console(0, string+off, linelen);
+		dumU_write_console(0, "\r\n", 2);
 		off += linelen + 1;
 	}
 	if (off < len)
-		write_console(0, string+off, len-off);
+		dumU_write_console(0, string+off, len-off);
 }
 
 struct console xenboot_console = {
@@ -222,7 +254,7 @@ struct console xenboot_console = {
 
 void xen_raw_console_write(const char *str)
 {
-	raw_console_write(str, strlen(str));
+	dom0_write_console(str, strlen(str));
 }
 
 void xen_raw_printk(const char *fmt, ...)
diff --git a/drivers/xen/events.c b/drivers/xen/events.c
index b83918a..90744d5 100644
--- a/drivers/xen/events.c
+++ b/drivers/xen/events.c
@@ -832,7 +832,7 @@ static int bind_ipi_to_irq(unsigned int ipi, unsigned int cpu)
 }
 
 
-static int bind_virq_to_irq(unsigned int virq, unsigned int cpu)
+int bind_virq_to_irq(unsigned int virq, unsigned int cpu)
 {
 	struct evtchn_bind_virq bind_virq;
 	int evtchn, irq;
diff --git a/include/xen/events.h b/include/xen/events.h
index 8cd18a5..3d2b0af 100644
--- a/include/xen/events.h
+++ b/include/xen/events.h
@@ -12,6 +12,7 @@ int bind_evtchn_to_irqhandler(unsigned int evtchn,
 			      irq_handler_t handler,
 			      unsigned long irqflags, const char *devname,
 			      void *dev_id);
+int bind_virq_to_irq(unsigned int virq, unsigned int cpu);
 int bind_virq_to_irqhandler(unsigned int virq, unsigned int cpu,
 			    irq_handler_t handler,
 			    unsigned long irqflags, const char *devname,
-- 
1.5.6.5

[PATCH 10/12] xen: add support for the platform_ops hypercall

[stefano.stabellini]

From: Stephen Tweedie <sct@redhat.com>

Minimal changes to get platform ops (renamed dom0_ops on pv_ops) working
on pv_ops builds.

[ Impact: add Xen hypercall definitions ]

Signed-off-by: Stephen Tweedie <sct@redhat.com>
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 arch/x86/include/asm/xen/hypercall.h |    8 +++
 include/xen/interface/platform.h     |   96 ++++++++++++++++++++++++++++++++++
 2 files changed, 104 insertions(+), 0 deletions(-)
 create mode 100644 include/xen/interface/platform.h

diff --git a/arch/x86/include/asm/xen/hypercall.h b/arch/x86/include/asm/xen/hypercall.h
index 7fda040..186b13d 100644
--- a/arch/x86/include/asm/xen/hypercall.h
+++ b/arch/x86/include/asm/xen/hypercall.h
@@ -45,6 +45,7 @@
 #include <xen/interface/xen.h>
 #include <xen/interface/sched.h>
 #include <xen/interface/physdev.h>
+#include <xen/interface/platform.h>
 
 /*
  * The hypercall asms have to meet several constraints:
@@ -282,6 +283,13 @@ HYPERVISOR_set_timer_op(u64 timeout)
 }
 
 static inline int
+HYPERVISOR_dom0_op(struct xen_platform_op *platform_op)
+{
+	platform_op->interface_version = XENPF_INTERFACE_VERSION;
+	return _hypercall1(int, dom0_op, platform_op);
+}
+
+static inline int
 HYPERVISOR_set_debugreg(int reg, unsigned long value)
 {
 	return _hypercall2(int, set_debugreg, reg, value);
diff --git a/include/xen/interface/platform.h b/include/xen/interface/platform.h
new file mode 100644
index 0000000..3a8cc09
--- /dev/null
+++ b/include/xen/interface/platform.h
@@ -0,0 +1,96 @@
+/******************************************************************************
+ * platform.h
+ *
+ * Hardware platform operations. Intended for use by domain-0 kernel.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (c) 2002-2006, K Fraser
+ */
+
+#ifndef __XEN_PUBLIC_PLATFORM_H__
+#define __XEN_PUBLIC_PLATFORM_H__
+
+#include "xen.h"
+
+#define XENPF_INTERFACE_VERSION 0x03000001
+
+/*
+ * Request memory range (@mfn, @mfn+@nr_mfns-1) to have type @type.
+ * On x86, @type is an architecture-defined MTRR memory type.
+ * On success, returns the MTRR that was used (@reg) and a handle that can
+ * be passed to XENPF_DEL_MEMTYPE to accurately tear down the new setting.
+ * (x86-specific).
+ */
+#define XENPF_add_memtype         31
+struct xenpf_add_memtype {
+    /* IN variables. */
+    unsigned long mfn;
+    uint64_t nr_mfns;
+    uint32_t type;
+    /* OUT variables. */
+    uint32_t handle;
+    uint32_t reg;
+};
+typedef struct xenpf_add_memtype xenpf_add_memtype_t;
+DEFINE_GUEST_HANDLE_STRUCT(xenpf_add_memtype_t);
+
+/*
+ * Tear down an existing memory-range type. If @handle is remembered then it
+ * should be passed in to accurately tear down the correct setting (in case
+ * of overlapping memory regions with differing types). If it is not known
+ * then @handle should be set to zero. In all cases @reg must be set.
+ * (x86-specific).
+ */
+#define XENPF_del_memtype         32
+struct xenpf_del_memtype {
+    /* IN variables. */
+    uint32_t handle;
+    uint32_t reg;
+};
+typedef struct xenpf_del_memtype xenpf_del_memtype_t;
+DEFINE_GUEST_HANDLE_STRUCT(xenpf_del_memtype_t);
+
+/* Read current type of an MTRR (x86-specific). */
+#define XENPF_read_memtype        33
+struct xenpf_read_memtype {
+    /* IN variables. */
+    uint32_t reg;
+    /* OUT variables. */
+    unsigned long mfn;
+    uint64_t nr_mfns;
+    uint32_t type;
+};
+typedef struct xenpf_read_memtype xenpf_read_memtype_t;
+DEFINE_GUEST_HANDLE_STRUCT(xenpf_read_memtype_t);
+
+struct xen_platform_op {
+	uint32_t cmd;
+	uint32_t interface_version; /* XENPF_INTERFACE_VERSION */
+	union {
+		struct xenpf_add_memtype       add_memtype;
+		struct xenpf_del_memtype       del_memtype;
+		struct xenpf_read_memtype      read_memtype;
+		uint8_t                        pad[128];
+	} u;
+};
+typedef struct xen_platform_op xen_platform_op_t;
+DEFINE_GUEST_HANDLE_STRUCT(xen_platform_op_t);
+
+#endif /* __XEN_PUBLIC_PLATFORM_H__ */
-- 
1.5.6.5

[PATCH 11/12] x86/mtrr: Extend mtrr_if to include num_var_ranges

[stefano.stabellini]

From: Stephen Tweedie <sct@redhat.com>

Reorganise mtrr initialisation slightly to allow the mtrr driver to
set up num_var_ranges.  This cleans things up by making each driver
return the number of ranges rather than having a single function with
magic knowledge.

[ Impact: clean up num_var_ranges operation ]

Signed-off-by: Stephen Tweedie <sct@redhat.com>
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 arch/x86/kernel/cpu/mtrr/amd.c     |    6 ++++++
 arch/x86/kernel/cpu/mtrr/centaur.c |    6 ++++++
 arch/x86/kernel/cpu/mtrr/cyrix.c   |    6 ++++++
 arch/x86/kernel/cpu/mtrr/generic.c |    8 ++++++++
 arch/x86/kernel/cpu/mtrr/main.c    |   17 +----------------
 arch/x86/kernel/cpu/mtrr/mtrr.h    |    1 +
 6 files changed, 28 insertions(+), 16 deletions(-)

diff --git a/arch/x86/kernel/cpu/mtrr/amd.c b/arch/x86/kernel/cpu/mtrr/amd.c
index 92ba9cd..8556894 100644
--- a/arch/x86/kernel/cpu/mtrr/amd.c
+++ b/arch/x86/kernel/cpu/mtrr/amd.c
@@ -108,6 +108,11 @@ amd_validate_add_page(unsigned long base, unsigned long size, unsigned int type)
 	return 0;
 }
 
+static int __init amd_num_var_ranges(void)
+{
+	return 2;
+}
+
 static const struct mtrr_ops amd_mtrr_ops = {
 	.vendor            = X86_VENDOR_AMD,
 	.set               = amd_set_mtrr,
@@ -115,6 +120,7 @@ static const struct mtrr_ops amd_mtrr_ops = {
 	.get_free_region   = generic_get_free_region,
 	.validate_add_page = amd_validate_add_page,
 	.have_wrcomb       = positive_have_wrcomb,
+	.num_var_ranges    = amd_num_var_ranges,
 };
 
 int __init amd_init_mtrr(void)
diff --git a/arch/x86/kernel/cpu/mtrr/centaur.c b/arch/x86/kernel/cpu/mtrr/centaur.c
index 316fe3e..4ac35bb 100644
--- a/arch/x86/kernel/cpu/mtrr/centaur.c
+++ b/arch/x86/kernel/cpu/mtrr/centaur.c
@@ -110,6 +110,11 @@ centaur_validate_add_page(unsigned long base, unsigned long size, unsigned int t
 	return 0;
 }
 
+static int __init centaur_num_var_ranges(void)
+{
+	return 8;
+}
+
 static const struct mtrr_ops centaur_mtrr_ops = {
 	.vendor            = X86_VENDOR_CENTAUR,
 	.set               = centaur_set_mcr,
@@ -117,6 +122,7 @@ static const struct mtrr_ops centaur_mtrr_ops = {
 	.get_free_region   = centaur_get_free_region,
 	.validate_add_page = centaur_validate_add_page,
 	.have_wrcomb       = positive_have_wrcomb,
+	.num_var_ranges    = centaur_num_var_ranges,
 };
 
 int __init centaur_init_mtrr(void)
diff --git a/arch/x86/kernel/cpu/mtrr/cyrix.c b/arch/x86/kernel/cpu/mtrr/cyrix.c
index 68a3343..8320306 100644
--- a/arch/x86/kernel/cpu/mtrr/cyrix.c
+++ b/arch/x86/kernel/cpu/mtrr/cyrix.c
@@ -265,6 +265,11 @@ static void cyrix_set_all(void)
 	post_set();
 }
 
+static int __init cyrix_num_var_ranges(void)
+{
+	return 8;
+}
+
 static const struct mtrr_ops cyrix_mtrr_ops = {
 	.vendor            = X86_VENDOR_CYRIX,
 	.set_all	   = cyrix_set_all,
@@ -273,6 +278,7 @@ static const struct mtrr_ops cyrix_mtrr_ops = {
 	.get_free_region   = cyrix_get_free_region,
 	.validate_add_page = generic_validate_add_page,
 	.have_wrcomb       = positive_have_wrcomb,
+	.num_var_ranges    = cyrix_num_var_ranges,
 };
 
 int __init cyrix_init_mtrr(void)
diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/generic.c
index 7d28d7d..a7d3e8f 100644
--- a/arch/x86/kernel/cpu/mtrr/generic.c
+++ b/arch/x86/kernel/cpu/mtrr/generic.c
@@ -747,6 +747,13 @@ int positive_have_wrcomb(void)
 	return 1;
 }
 
+static int __init generic_num_var_ranges(void)
+{
+	unsigned long config = 0, dummy;
+	rdmsr(MSR_MTRRcap, config, dummy);
+	return config & 0xff;
+}
+
 /*
  * Generic structure...
  */
@@ -758,4 +765,5 @@ const struct mtrr_ops generic_mtrr_ops = {
 	.set			= generic_set_mtrr,
 	.validate_add_page	= generic_validate_add_page,
 	.have_wrcomb		= generic_have_wrcomb,
+	.num_var_ranges    = generic_num_var_ranges,
 };
diff --git a/arch/x86/kernel/cpu/mtrr/main.c b/arch/x86/kernel/cpu/mtrr/main.c
index 01c0f3e..91f8f62 100644
--- a/arch/x86/kernel/cpu/mtrr/main.c
+++ b/arch/x86/kernel/cpu/mtrr/main.c
@@ -111,21 +111,6 @@ static int have_wrcomb(void)
 	return mtrr_if->have_wrcomb ? mtrr_if->have_wrcomb() : 0;
 }
 
-/*  This function returns the number of variable MTRRs  */
-static void __init set_num_var_ranges(void)
-{
-	unsigned long config = 0, dummy;
-
-	if (use_intel())
-		rdmsr(MSR_MTRRcap, config, dummy);
-	else if (is_cpu(AMD))
-		config = 2;
-	else if (is_cpu(CYRIX) || is_cpu(CENTAUR))
-		config = 8;
-
-	num_var_ranges = config & 0xff;
-}
-
 static void __init init_table(void)
 {
 	int i, max;
@@ -743,7 +728,7 @@ void __init mtrr_bp_init(void)
 	}
 
 	if (mtrr_if) {
-		set_num_var_ranges();
+		num_var_ranges = mtrr_if->num_var_ranges();
 		init_table();
 		if (use_intel()) {
 			get_mtrr_state();
diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.h b/arch/x86/kernel/cpu/mtrr/mtrr.h
index df5e41f..add8abe 100644
--- a/arch/x86/kernel/cpu/mtrr/mtrr.h
+++ b/arch/x86/kernel/cpu/mtrr/mtrr.h
@@ -25,6 +25,7 @@ struct mtrr_ops {
 	int	(*validate_add_page)(unsigned long base, unsigned long size,
 				     unsigned int type);
 	int	(*have_wrcomb)(void);
+	int	(*num_var_ranges)(void);
 };
 
 extern int generic_get_free_region(unsigned long base, unsigned long size,
-- 
1.5.6.5

[PATCH 12/12] xen/mtrr: Add mtrr_if support for Xen mtrr

[stefano.stabellini]

From: Stephen Tweedie <sct@redhat.com>

Add a Xen mtrr type, and reorganise mtrr initialisation slightly to
allow the mtrr driver to set up num_var_ranges (Xen needs to do this by
querying the hypervisor itself.)

[ Impact: add basic MTRR support ]

Signed-off-by: Stephen Tweedie <sct@redhat.com>
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
 arch/x86/kernel/cpu/mtrr/Makefile |    2 +-
 arch/x86/kernel/cpu/mtrr/main.c   |    3 +
 arch/x86/kernel/cpu/mtrr/mtrr.h   |    7 ++
 arch/x86/kernel/cpu/mtrr/xen.c    |  110 +++++++++++++++++++++++++++++++++++++
 4 files changed, 121 insertions(+), 1 deletions(-)
 create mode 100644 arch/x86/kernel/cpu/mtrr/xen.c

diff --git a/arch/x86/kernel/cpu/mtrr/Makefile b/arch/x86/kernel/cpu/mtrr/Makefile
index ad9e5ed..e955771 100644
--- a/arch/x86/kernel/cpu/mtrr/Makefile
+++ b/arch/x86/kernel/cpu/mtrr/Makefile
@@ -1,3 +1,3 @@
 obj-y		:= main.o if.o generic.o cleanup.o
 obj-$(CONFIG_X86_32) += amd.o cyrix.o centaur.o
-
+obj-$(CONFIG_XEN) += xen.o
diff --git a/arch/x86/kernel/cpu/mtrr/main.c b/arch/x86/kernel/cpu/mtrr/main.c
index 91f8f62..fcfa520 100644
--- a/arch/x86/kernel/cpu/mtrr/main.c
+++ b/arch/x86/kernel/cpu/mtrr/main.c
@@ -727,6 +727,9 @@ void __init mtrr_bp_init(void)
 		}
 	}
 
+	/* Let Xen code override the above if it wants */
+	xen_init_mtrr();
+
 	if (mtrr_if) {
 		num_var_ranges = mtrr_if->num_var_ranges();
 		init_table();
diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.h b/arch/x86/kernel/cpu/mtrr/mtrr.h
index add8abe..8f0693e 100644
--- a/arch/x86/kernel/cpu/mtrr/mtrr.h
+++ b/arch/x86/kernel/cpu/mtrr/mtrr.h
@@ -74,6 +74,13 @@ void mtrr_wrmsr(unsigned, unsigned, unsigned);
 int amd_init_mtrr(void);
 int cyrix_init_mtrr(void);
 int centaur_init_mtrr(void);
+#ifdef CONFIG_XEN
+void xen_init_mtrr(void);
+#else
+static inline void xen_init_mtrr(void)
+{
+}
+#endif
 
 extern int changed_by_mtrr_cleanup;
 extern int mtrr_cleanup(unsigned address_bits);
diff --git a/arch/x86/kernel/cpu/mtrr/xen.c b/arch/x86/kernel/cpu/mtrr/xen.c
new file mode 100644
index 0000000..fec3b0a
--- /dev/null
+++ b/arch/x86/kernel/cpu/mtrr/xen.c
@@ -0,0 +1,110 @@
+#include <linux/init.h>
+#include <linux/mm.h>
+
+#include <asm/pat.h>
+#include <asm/mtrr.h>
+
+#include "mtrr.h"
+
+#include <xen/xen.h>
+#include <xen/interface/platform.h>
+#include <asm/xen/hypervisor.h>
+#include <asm/xen/hypercall.h>
+
+static void xen_set_mtrr(unsigned int reg, unsigned long base,
+			 unsigned long size, mtrr_type type)
+{
+	struct xen_platform_op op;
+	int error;
+
+	/* mtrr_ops->set() is called once per CPU,
+	 * but Xen's ops apply to all CPUs.
+	 */
+	if (smp_processor_id())
+		return;
+
+	if (size == 0) {
+		op.cmd = XENPF_del_memtype;
+		op.u.del_memtype.handle = 0;
+		op.u.del_memtype.reg    = reg;
+	} else {
+		op.cmd = XENPF_add_memtype;
+		op.u.add_memtype.mfn     = base;
+		op.u.add_memtype.nr_mfns = size;
+		op.u.add_memtype.type    = type;
+	}
+
+	error = HYPERVISOR_dom0_op(&op);
+	BUG_ON(error != 0);
+}
+
+static void xen_get_mtrr(unsigned int reg, unsigned long *base,
+			 unsigned long *size, mtrr_type *type)
+{
+	struct xen_platform_op op;
+
+	op.cmd = XENPF_read_memtype;
+	op.u.read_memtype.reg = reg;
+	if (HYPERVISOR_dom0_op(&op) != 0) {
+		*base = 0;
+		*size = 0;
+		*type = 0;
+		return;
+	}
+
+	*size = op.u.read_memtype.nr_mfns;
+	*base = op.u.read_memtype.mfn;
+	*type = op.u.read_memtype.type;
+}
+
+static int __init xen_num_var_ranges(void)
+{
+	int ranges;
+	struct xen_platform_op op;
+
+	op.cmd = XENPF_read_memtype;
+
+	for (ranges = 0; ; ranges++) {
+		op.u.read_memtype.reg = ranges;
+		if (HYPERVISOR_dom0_op(&op) != 0)
+			break;
+	}
+	return ranges;
+}
+
+/*
+ * DOM0 TODO: Need to fill in the remaining mtrr methods to have full
+ * working userland mtrr support.
+ */
+static struct mtrr_ops xen_mtrr_ops = {
+	.vendor            = X86_VENDOR_UNKNOWN,
+	.get_free_region   = generic_get_free_region,
+	.set               = xen_set_mtrr,
+	.get               = xen_get_mtrr,
+	.have_wrcomb       = positive_have_wrcomb,
+	.validate_add_page = generic_validate_add_page,
+	.use_intel_if	   = 0,
+	.num_var_ranges	   = xen_num_var_ranges,
+};
+
+void __init xen_init_mtrr(void)
+{
+	/* 
+	 * Check that we're running under Xen, and privileged enough
+	 * to play with MTRRs.
+	 */
+	if (!xen_initial_domain())
+		return;
+
+	/* 
+	 * Check that the CPU has an MTRR implementation we can
+	 * support.
+	 */
+	if (cpu_has_mtrr ||
+	    cpu_has_k6_mtrr ||
+	    cpu_has_cyrix_arr ||
+	    cpu_has_centaur_mcr) {
+		mtrr_if = &xen_mtrr_ops;
+		pat_init();
+	}
+}
-- 
1.5.6.5

Re: [PATCH 12/12] xen/mtrr: Add mtrr_if support for Xen mtrr

[Ingo Molnar]

* stefano.stabellini@eu.citrix.com <stefano.stabellini@eu.citrix.com> wrote:

> From: Stephen Tweedie <sct@redhat.com>
> 
> Add a Xen mtrr type, and reorganise mtrr initialisation slightly to
> allow the mtrr driver to set up num_var_ranges (Xen needs to do this by
> querying the hypervisor itself.)
> 
> [ Impact: add basic MTRR support ]
> 
> Signed-off-by: Stephen Tweedie <sct@redhat.com>
> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> ---
>  arch/x86/kernel/cpu/mtrr/Makefile |    2 +-
>  arch/x86/kernel/cpu/mtrr/main.c   |    3 +
>  arch/x86/kernel/cpu/mtrr/mtrr.h   |    7 ++
>  arch/x86/kernel/cpu/mtrr/xen.c    |  110 +++++++++++++++++++++++++++++++++++++
>  4 files changed, 121 insertions(+), 1 deletions(-)
>  create mode 100644 arch/x86/kernel/cpu/mtrr/xen.c

Still NAK, for the very same reasons as we NAK-ed it the previous time: 
/proc/mtrr is a problematic and complicated legacy interface that should 
die. Any modern X server will do the right thing via PAT.

Also, please get the Ack of at least one x86 maintainer for x86 patches.

Thanks,

	Ingo

Re: [PATCH 12/12] xen/mtrr: Add mtrr_if support for Xen mtrr

[Sander Eikelenboom]

>* stefano.stabellini@xxxxxxxxxxxxx <stefano.stabellini@xxxxxxxxxxxxx> wrote:
>
>> From: Stephen Tweedie <sct@xxxxxxxxxx>
>> 
>> Add a Xen mtrr type, and reorganise mtrr initialisation slightly to
>> allow the mtrr driver to set up num_var_ranges (Xen needs to do this by
>> querying the hypervisor itself.)
>> 
>> [ Impact: add basic MTRR support ]
>> 
>> Signed-off-by: Stephen Tweedie <sct@xxxxxxxxxx>
>> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@xxxxxxxxxx>
>> Signed-off-by: Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>
>> ---
>> arch/x86/kernel/cpu/mtrr/Makefile | 2 +-
>> arch/x86/kernel/cpu/mtrr/main.c | 3 +
>> arch/x86/kernel/cpu/mtrr/mtrr.h | 7 ++
>> arch/x86/kernel/cpu/mtrr/xen.c | 110 +++++++++++++++++++++++++++++++++++++
> 4 files changed, 121 insertions(+), 1 deletions(-)
> create mode 100644 arch/x86/kernel/cpu/mtrr/xen.c
>
>Still NAK, for the very same reasons as we NAK-ed it the previous time: 
>/proc/mtrr is a problematic and complicated legacy interface that should 
>die. Any modern X server will do the right thing via PAT.
>
>Also, please get the Ack of at least one x86 maintainer for x86 patches.
>
>Thanks,
>
>Ingo
>--
>

I can't find the MTRR API to be officially deprecated with any schedule for removal (at least i couldn't find it in feature-removal-schedule.txt).
KVM has had patches for MTRR as well, if it's so deprecated .. why has it ? .. legacy support for a not officially deprecated API perhaps ? To support CPU's that support MTRR but not PAT ?

If you think the MTRR interface should die, and should die fast, why hasn't it been added to feature-removal-schedule.txt for a complete removal in a not so distant future, since this is the only way to force all users to switch to PAT.

Perhaps it would be fair to ask for a working PAT implementation besides MTRR, but since it is not officially deprecated completely objecting a MTRR one just for Xen seems a bit unfair to me.

--
Sander

Re: [PATCH 12/12] xen/mtrr: Add mtrr_if support for Xen mtrr

[Stefano Stabellini]

On Tue, 28 Sep 2010, Ingo Molnar wrote:
> 
> * stefano.stabellini@eu.citrix.com <stefano.stabellini@eu.citrix.com> wrote:
> 
> > From: Stephen Tweedie <sct@redhat.com>
> > 
> > Add a Xen mtrr type, and reorganise mtrr initialisation slightly to
> > allow the mtrr driver to set up num_var_ranges (Xen needs to do this by
> > querying the hypervisor itself.)
> > 
> > [ Impact: add basic MTRR support ]
> > 
> > Signed-off-by: Stephen Tweedie <sct@redhat.com>
> > Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > ---
> >  arch/x86/kernel/cpu/mtrr/Makefile |    2 +-
> >  arch/x86/kernel/cpu/mtrr/main.c   |    3 +
> >  arch/x86/kernel/cpu/mtrr/mtrr.h   |    7 ++
> >  arch/x86/kernel/cpu/mtrr/xen.c    |  110 +++++++++++++++++++++++++++++++++++++
> >  4 files changed, 121 insertions(+), 1 deletions(-)
> >  create mode 100644 arch/x86/kernel/cpu/mtrr/xen.c
> 
> Still NAK, for the very same reasons as we NAK-ed it the previous time: 
> /proc/mtrr is a problematic and complicated legacy interface that should 
> die. Any modern X server will do the right thing via PAT.
> 

Sorry I should have read the original thread more carefully: I didn't
realize this patch had been NAK-ed.

However it is not a problem because we can easily disable MTRRs from Xen
and with no cpu_has_mtrr the kernel would still boot fine on Xen.
Also I think we do have PAT support nowadays but I'll let Jeremy comment
on that.


> Also, please get the Ack of at least one x86 maintainer for x86 patches.
> 
 
I'll repost the series without the last two patches, so there won't be
any x86 changes at all :)

Many thanks for your quick feedback,

Stefano

Re: [PATCH 12/12] xen/mtrr: Add mtrr_if support for Xen mtrr

[Jeremy Fitzhardinge]

 On 09/28/2010 07:00 AM, Stefano Stabellini wrote:
> On Tue, 28 Sep 2010, Ingo Molnar wrote:
>> * stefano.stabellini@eu.citrix.com <stefano.stabellini@eu.citrix.com> wrote:
>>
>>> From: Stephen Tweedie <sct@redhat.com>
>>>
>>> Add a Xen mtrr type, and reorganise mtrr initialisation slightly to
>>> allow the mtrr driver to set up num_var_ranges (Xen needs to do this by
>>> querying the hypervisor itself.)
>>>
>>> [ Impact: add basic MTRR support ]
>>>
>>> Signed-off-by: Stephen Tweedie <sct@redhat.com>
>>> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
>>> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>>> ---
>>>  arch/x86/kernel/cpu/mtrr/Makefile |    2 +-
>>>  arch/x86/kernel/cpu/mtrr/main.c   |    3 +
>>>  arch/x86/kernel/cpu/mtrr/mtrr.h   |    7 ++
>>>  arch/x86/kernel/cpu/mtrr/xen.c    |  110 +++++++++++++++++++++++++++++++++++++
>>>  4 files changed, 121 insertions(+), 1 deletions(-)
>>>  create mode 100644 arch/x86/kernel/cpu/mtrr/xen.c
>> Still NAK, for the very same reasons as we NAK-ed it the previous time: 
>> /proc/mtrr is a problematic and complicated legacy interface that should 
>> die. Any modern X server will do the right thing via PAT.
>>
> Sorry I should have read the original thread more carefully: I didn't
> realize this patch had been NAK-ed.
>
> However it is not a problem because we can easily disable MTRRs from Xen
> and with no cpu_has_mtrr the kernel would still boot fine on Xen.
> Also I think we do have PAT support nowadays but I'll let Jeremy comment
> on that.

Yes, we could just mask out the MTRR CPU feature and rely entirely on PAT.

The alternative would be to use the wrmsr hooks to emulate the Intel
MTRR registers by mapping them to hypercalls, but that seems needlessly
complex.

    J

Re: [PATCH 12/12] xen/mtrr: Add mtrr_if support for Xen mtrr

[Jeremy Fitzhardinge]

 On 09/28/2010 05:39 AM, Ingo Molnar wrote:
> * stefano.stabellini@eu.citrix.com <stefano.stabellini@eu.citrix.com> wrote:
>
>> From: Stephen Tweedie <sct@redhat.com>
>>
>> Add a Xen mtrr type, and reorganise mtrr initialisation slightly to
>> allow the mtrr driver to set up num_var_ranges (Xen needs to do this by
>> querying the hypervisor itself.)
>>
>> [ Impact: add basic MTRR support ]
>>
>> Signed-off-by: Stephen Tweedie <sct@redhat.com>
>> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
>> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>> ---
>>  arch/x86/kernel/cpu/mtrr/Makefile |    2 +-
>>  arch/x86/kernel/cpu/mtrr/main.c   |    3 +
>>  arch/x86/kernel/cpu/mtrr/mtrr.h   |    7 ++
>>  arch/x86/kernel/cpu/mtrr/xen.c    |  110 +++++++++++++++++++++++++++++++++++++
>>  4 files changed, 121 insertions(+), 1 deletions(-)
>>  create mode 100644 arch/x86/kernel/cpu/mtrr/xen.c
> Still NAK, for the very same reasons as we NAK-ed it the previous time: 
> /proc/mtrr is a problematic and complicated legacy interface that should 
> die.

Is your objection because we're adding a Xen mtrr implementation at all,
or because we're (slightly) extending the mtrr interface to do it?

Thanks,
    J

Re: [PATCH 12/12] xen/mtrr: Add mtrr_if support for Xen mtrr

[Stefano Stabellini]

On Tue, 28 Sep 2010, Jeremy Fitzhardinge wrote:
>  On 09/28/2010 07:00 AM, Stefano Stabellini wrote:
> > On Tue, 28 Sep 2010, Ingo Molnar wrote:
> >> * stefano.stabellini@eu.citrix.com <stefano.stabellini@eu.citrix.com> wrote:
> >>
> >>> From: Stephen Tweedie <sct@redhat.com>
> >>>
> >>> Add a Xen mtrr type, and reorganise mtrr initialisation slightly to
> >>> allow the mtrr driver to set up num_var_ranges (Xen needs to do this by
> >>> querying the hypervisor itself.)
> >>>
> >>> [ Impact: add basic MTRR support ]
> >>>
> >>> Signed-off-by: Stephen Tweedie <sct@redhat.com>
> >>> Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
> >>> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> >>> ---
> >>>  arch/x86/kernel/cpu/mtrr/Makefile |    2 +-
> >>>  arch/x86/kernel/cpu/mtrr/main.c   |    3 +
> >>>  arch/x86/kernel/cpu/mtrr/mtrr.h   |    7 ++
> >>>  arch/x86/kernel/cpu/mtrr/xen.c    |  110 +++++++++++++++++++++++++++++++++++++
> >>>  4 files changed, 121 insertions(+), 1 deletions(-)
> >>>  create mode 100644 arch/x86/kernel/cpu/mtrr/xen.c
> >> Still NAK, for the very same reasons as we NAK-ed it the previous time: 
> >> /proc/mtrr is a problematic and complicated legacy interface that should 
> >> die. Any modern X server will do the right thing via PAT.
> >>
> > Sorry I should have read the original thread more carefully: I didn't
> > realize this patch had been NAK-ed.
> >
> > However it is not a problem because we can easily disable MTRRs from Xen
> > and with no cpu_has_mtrr the kernel would still boot fine on Xen.
> > Also I think we do have PAT support nowadays but I'll let Jeremy comment
> > on that.
> 
> Yes, we could just mask out the MTRR CPU feature and rely entirely on PAT.
> 
> The alternative would be to use the wrmsr hooks to emulate the Intel
> MTRR registers by mapping them to hypercalls, but that seems needlessly
> complex.
 
Yeah, I have already tested a prototype patch to xen to mask the MTRR
cpu features and everything seems to run fine on my testbox.

Re: [PATCH 12/12] xen/mtrr: Add mtrr_if support for Xen mtrr

[H. Peter Anvin]

On 09/28/2010 10:13 AM, Jeremy Fitzhardinge wrote:
> 
> Yes, we could just mask out the MTRR CPU feature and rely entirely on PAT.
> 
> The alternative would be to use the wrmsr hooks to emulate the Intel
> MTRR registers by mapping them to hypercalls, but that seems needlessly
> complex.
> 

Indeed.  Relying on pure PAT is the Right Thing[TM].

	-hpa

Re: [PATCH 12/12] xen/mtrr: Add mtrr_if support for Xen mtrr

[Jeremy Fitzhardinge]

 On 09/28/2010 10:56 AM, H. Peter Anvin wrote:
> On 09/28/2010 10:13 AM, Jeremy Fitzhardinge wrote:
>> Yes, we could just mask out the MTRR CPU feature and rely entirely on PAT.
>>
>> The alternative would be to use the wrmsr hooks to emulate the Intel
>> MTRR registers by mapping them to hypercalls, but that seems needlessly
>> complex.
>>
> Indeed.  Relying on pure PAT is the Right Thing[TM].

Is there a plan to formally deprecate /proc/mtrr and the kernel
infrastructure behind it?

    J

Re: [PATCH 12/12] xen/mtrr: Add mtrr_if support for Xen mtrr

[H. Peter Anvin]

On 09/28/2010 11:13 AM, Jeremy Fitzhardinge wrote:
>  On 09/28/2010 10:56 AM, H. Peter Anvin wrote:
>> On 09/28/2010 10:13 AM, Jeremy Fitzhardinge wrote:
>>> Yes, we could just mask out the MTRR CPU feature and rely entirely on PAT.
>>>
>>> The alternative would be to use the wrmsr hooks to emulate the Intel
>>> MTRR registers by mapping them to hypercalls, but that seems needlessly
>>> complex.
>>>
>> Indeed.  Relying on pure PAT is the Right Thing[TM].
> 
> Is there a plan to formally deprecate /proc/mtrr and the kernel
> infrastructure behind it?
> 

No, and we really can't do it for a couple of reasons:

a) Pre-PAT hardware;
b) MTRRs and PAT interact on hardware;
c) MTRRs, but not PAT, interact with SMM.

However, since a virtual machine like Xen doesn't have these issues, it
doesn't apply.

	-hpa

Re: [PATCH 12/12] xen/mtrr: Add mtrr_if support for Xen mtrr

[Jeremy Fitzhardinge]

 On 09/28/2010 11:19 AM, H. Peter Anvin wrote:
> On 09/28/2010 11:13 AM, Jeremy Fitzhardinge wrote:
>>  On 09/28/2010 10:56 AM, H. Peter Anvin wrote:
>>> On 09/28/2010 10:13 AM, Jeremy Fitzhardinge wrote:
>>>> Yes, we could just mask out the MTRR CPU feature and rely entirely on PAT.
>>>>
>>>> The alternative would be to use the wrmsr hooks to emulate the Intel
>>>> MTRR registers by mapping them to hypercalls, but that seems needlessly
>>>> complex.
>>>>
>>> Indeed.  Relying on pure PAT is the Right Thing[TM].
>> Is there a plan to formally deprecate /proc/mtrr and the kernel
>> infrastructure behind it?
>>
> No, and we really can't do it for a couple of reasons:
>
> a) Pre-PAT hardware;
> b) MTRRs and PAT interact on hardware;
> c) MTRRs, but not PAT, interact with SMM.

What about pre-PAT software (ie, X servers which still use /proc/mtrr)?

> However, since a virtual machine like Xen doesn't have these issues, it
> doesn't apply

Well, we're specifically talking about a virtual machine which has
direct access to hardware, so it is concerned about the real physical
memory properties of real physical pages.  If we can assume that
BIOS/Xen will always set up MTRR correctly then there shouldn't be any
need for the kernel to modify the MTRR itself.  How true is that in
general?  I don't know, but if we could rely on BIOS then there'd never
be a need to touch MTRR, would there?

    J

Re: [PATCH 12/12] xen/mtrr: Add mtrr_if support for Xen mtrr

[H. Peter Anvin]

On 09/28/2010 11:24 AM, Jeremy Fitzhardinge wrote:
>>>
>> No, and we really can't do it for a couple of reasons:
>>
>> a) Pre-PAT hardware;
>> b) MTRRs and PAT interact on hardware;
>> c) MTRRs, but not PAT, interact with SMM.
> 
> What about pre-PAT software (ie, X servers which still use /proc/mtrr)?
> 

Fortunately going away... we have talked in the past about doing
"virtual MTRRs" in terms of PAT to deal with this kind of legacy
software, but the demand for it seems to be low enough to not be worth
bothering with.

>> However, since a virtual machine like Xen doesn't have these issues, it
>> doesn't apply
> 
> Well, we're specifically talking about a virtual machine which has
> direct access to hardware, so it is concerned about the real physical
> memory properties of real physical pages.  If we can assume that
> BIOS/Xen will always set up MTRR correctly then there shouldn't be any
> need for the kernel to modify the MTRR itself.  How true is that in
> general?  I don't know, but if we could rely on BIOS then there'd never
> be a need to touch MTRR, would there?

Well, in the past MTRRs were abused for device properties mainly by the
X server, but other than that, no, not really.  The other thing we do is
the MTRR cleanup (which doesn't involve /proc/mtrr) to deal with
brokenness in the BIOS setup, but that really belongs in the hypervisor
in your case since it fundamentally affects how memory is handled.

	-hpa

Re: [PATCH 12/12] xen/mtrr: Add mtrr_if support for Xen mtrr

[Jeremy Fitzhardinge]

 On 09/28/2010 11:46 AM, H. Peter Anvin wrote:
> Well, we're specifically talking about a virtual machine which has
> direct access to hardware, so it is concerned about the real physical
> memory properties of real physical pages.  If we can assume that
> BIOS/Xen will always set up MTRR correctly then there shouldn't be any
> need for the kernel to modify the MTRR itself.  How true is that in
> general?  I don't know, but if we could rely on BIOS then there'd never
> be a need to touch MTRR, would there?
> Well, in the past MTRRs were abused for device properties mainly by the
> X server, but other than that, no, not really.  The other thing we do is
> the MTRR cleanup (which doesn't involve /proc/mtrr) to deal with
> brokenness in the BIOS setup, but that really belongs in the hypervisor
> in your case since it fundamentally affects how memory is handled.

Yeah, the hypervisor should definitely deal with that.  I have no
problem in principle with leaving MTRRs entirely to Xen, but I was just
concerned about possible repercussions.  Certainly when I first did this
work, I was using Fedora 8 whose X server did depend on /proc/mtrr for
good performance.

    J

Re: [PATCH 12/12] xen/mtrr: Add mtrr_if support for Xen mtrr

[H. Peter Anvin]

On 09/28/2010 11:58 AM, Jeremy Fitzhardinge wrote:
>  On 09/28/2010 11:46 AM, H. Peter Anvin wrote:
>> Well, we're specifically talking about a virtual machine which has
>> direct access to hardware, so it is concerned about the real physical
>> memory properties of real physical pages.  If we can assume that
>> BIOS/Xen will always set up MTRR correctly then there shouldn't be any
>> need for the kernel to modify the MTRR itself.  How true is that in
>> general?  I don't know, but if we could rely on BIOS then there'd never
>> be a need to touch MTRR, would there?
>> Well, in the past MTRRs were abused for device properties mainly by the
>> X server, but other than that, no, not really.  The other thing we do is
>> the MTRR cleanup (which doesn't involve /proc/mtrr) to deal with
>> brokenness in the BIOS setup, but that really belongs in the hypervisor
>> in your case since it fundamentally affects how memory is handled.
> 
> Yeah, the hypervisor should definitely deal with that.  I have no
> problem in principle with leaving MTRRs entirely to Xen, but I was just
> concerned about possible repercussions.  Certainly when I first did this
> work, I was using Fedora 8 whose X server did depend on /proc/mtrr for
> good performance.
> 

Yeah, that should all be fixed now.

	-hpa