Re: IOMMU Interrupt Remapping query

[Andrew Cooper <andrew.cooper3@citrix.com>]

[-- Attachment #1: Type: text/plain, Size: 2783 bytes --]



On 06/06/11 16:21, Keir Fraser wrote:
> On 06/06/2011 15:32, "Andrew Cooper" <andrew.cooper3@citrix.com> wrote:
>
>> I am attempting to fix the kexec interactions with x2apic and iommu
>> functionality.  Part of this involves ensuring that all IOMMU
>> functionality is disabled, as the kdump kernels are not happy at having
>> their interrupts remapped without their knowledge.
>>
>> I have introduced iommu_disable_x2apic_IR() onto the kexec path, but it
>> does not seem to actually disable interrupt remapping on Intel boxes
>> (Specifically the two Intel Nehalem boxes I am testing on).
>>
>> Specifying iommu=no-intremap on the commandline causes everything to
>> work correctly, but leaving it out causes the kdump kernel to hang and
>> eventually reboot, as can be seen on the attached serial log.
>>
>> The lines starting DBG: are extra debugging I have put in which shows
>> that the disable_IR() function is being called and writing to the registers.
> Should have attached your patch as well. Noone else can know with certainty
> where you put your debugging, and noone else is going to want to help debug
> your code if they can't even see it. :-)
>
> Also a good idea to Cc a likely person who can help (i.e., someone who wrote
> the code that you are querying). 'hg annotate' is useful for this -- in this
> case I am adding Weidong Han to the cc list.
>
> On the bright side, this must have been got working for S3 suspend/resume to
> work properly (indeed that's what the disable code was originally added
> for). So it can't be an insurmountable problem.
>
>  -- Keir
>
>> This problem occurs with the XenServer version of 4.1.0 as well as on
>> xen-unstable at the moment.
>>
>> Is there any hardware state which is not taken down by the disable
>> function, any subtle interactions which I have not taken account of?  I
>> have looked through the source and nothing pops out, but I am out of ideas.
>>
>> Thanks in advance,
>
Attached are the two relevant patches, and two which I don't think are
relevant but might be if I am wrong.  crash_shutdown was an attempt to
make an iommu_ops which shut down all iommu functionality without saving
state.  debug-wip shows where I have put in debug statements.

kdump-fix-x2apic and apic-record-boot-mode are also in the source, but I
believe them to be unrelated to the current problem.

I have done some further debugging on the assumption that the order of
shutting down interupt remapping matters with shutting down the lapics
and ioapics, but disable_qinval causes a panic (qinval.c:222 - "queue
invalidate wait descriptor was not executed\n") if it is run before both
the lapics and ioapics are shut down.

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com


[-- Attachment #2: crash_shutdown_iommu_ops.patch --]
[-- Type: text/x-patch, Size: 4409 bytes --]

IOMMU: add crash_shutdown iommu_op

The kdump kernel has problems booting with interrupt/dma
remapping enabled, so we need a new iommu_ops called
crash_shutdown which is basically suspend but doesn't
need to bother saving state.

We make sure that crash_shutdown is called on the kexec
path.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

diff -r 09344e4f7cde xen/arch/x86/crash.c
--- a/xen/arch/x86/crash.c	Thu May 26 16:51:35 2011 +0100
+++ b/xen/arch/x86/crash.c	Thu May 26 17:00:40 2011 +0100
@@ -89,6 +89,10 @@ void machine_crash_shutdown(void)
 
     nmi_shootdown_cpus();
 
+    /* Crash shutdown any IOMMU functionality as the crashdump kernel is not
+     * happy when booting if interrupt/dma remapping is still enabled */
+    iommu_crash_shutdown();
+
     info = kexec_crash_save_info();
     info->xen_phys_start = xen_phys_start;
     info->dom0_pfn_to_mfn_frame_list_list =
diff -r 09344e4f7cde xen/drivers/passthrough/amd/iommu_init.c
--- a/xen/drivers/passthrough/amd/iommu_init.c	Thu May 26 16:51:35 2011 +0100
+++ b/xen/drivers/passthrough/amd/iommu_init.c	Thu May 26 17:00:40 2011 +0100
@@ -923,6 +923,14 @@ void amd_iommu_suspend(void)
         disable_iommu(iommu);
 }
 
+void amd_iommu_crash_shutdown(void)
+{
+    struct amd_iommu *iommu;
+
+    for_each_amd_iommu ( iommu )
+        disable_iommu(iommu);
+}
+
 void amd_iommu_resume(void)
 {
     struct amd_iommu *iommu;
diff -r 09344e4f7cde xen/drivers/passthrough/amd/pci_amd_iommu.c
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c	Thu May 26 16:51:35 2011 +0100
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c	Thu May 26 17:00:40 2011 +0100
@@ -455,4 +455,5 @@ const struct iommu_ops amd_iommu_ops = {
     .read_msi_from_ire = amd_iommu_read_msi_from_ire,
     .suspend = amd_iommu_suspend,
     .resume = amd_iommu_resume,
+    .crash_shutdown = amd_iommu_crash_shutdown,
 };
diff -r 09344e4f7cde xen/drivers/passthrough/iommu.c
--- a/xen/drivers/passthrough/iommu.c	Thu May 26 16:51:35 2011 +0100
+++ b/xen/drivers/passthrough/iommu.c	Thu May 26 17:00:40 2011 +0100
@@ -419,6 +419,15 @@ void iommu_suspend(void)
         ops->suspend();
 }
 
+void iommu_crash_shutdown(void)
+{
+    const struct iommu_ops *ops = iommu_get_ops();
+    if ( ops && ops->crash_shutdown )
+        ops->crash_shutdown();
+}
+
+
+
 /*
  * Local variables:
  * mode: C
diff -r 09344e4f7cde xen/drivers/passthrough/vtd/iommu.c
--- a/xen/drivers/passthrough/vtd/iommu.c	Thu May 26 16:51:35 2011 +0100
+++ b/xen/drivers/passthrough/vtd/iommu.c	Thu May 26 17:00:40 2011 +0100
@@ -2261,6 +2261,25 @@ static void vtd_suspend(void)
     }
 }
 
+static void vtd_crash_shutdown(void)
+{
+    struct acpi_drhd_unit *drhd;
+    struct iommu *iommu;
+
+    if ( !iommu_enabled )
+        return;
+
+    iommu_flush_all();
+
+    for_each_drhd_unit ( drhd )
+    {
+        iommu = drhd->iommu;
+        iommu_disable_translation(iommu);
+    }
+
+    iommu_disable_IR();
+}
+
 static void vtd_resume(void)
 {
     struct acpi_drhd_unit *drhd;
@@ -2311,6 +2330,7 @@ const struct iommu_ops intel_iommu_ops =
     .read_msi_from_ire = msi_msg_read_remap_rte,
     .suspend = vtd_suspend,
     .resume = vtd_resume,
+    .crash_shutdown = vtd_crash_shutdown,
 };
 
 /*
diff -r 09344e4f7cde xen/include/asm-x86/hvm/svm/amd-iommu-proto.h
--- a/xen/include/asm-x86/hvm/svm/amd-iommu-proto.h	Thu May 26 16:51:35 2011 +0100
+++ b/xen/include/asm-x86/hvm/svm/amd-iommu-proto.h	Thu May 26 17:00:40 2011 +0100
@@ -91,6 +91,7 @@ unsigned int amd_iommu_read_ioapic_from_
 /* power management support */
 void amd_iommu_resume(void);
 void amd_iommu_suspend(void);
+void amd_iommu_crash_shutdown(void);
 
 static inline u32 get_field_from_reg_u32(u32 reg_value, u32 mask, u32 shift)
 {
diff -r 09344e4f7cde xen/include/xen/iommu.h
--- a/xen/include/xen/iommu.h	Thu May 26 16:51:35 2011 +0100
+++ b/xen/include/xen/iommu.h	Thu May 26 17:00:40 2011 +0100
@@ -133,6 +133,7 @@ struct iommu_ops {
     unsigned int (*read_apic_from_ire)(unsigned int apic, unsigned int reg);
     void (*suspend)(void);
     void (*resume)(void);
+    void (*crash_shutdown)(void);
 };
 
 void iommu_update_ire_from_apic(unsigned int apic, unsigned int reg, unsigned int value);
@@ -142,6 +143,7 @@ unsigned int iommu_read_apic_from_ire(un
 
 void iommu_suspend(void);
 void iommu_resume(void);
+void iommu_crash_shutdown(void);
 
 void iommu_set_dom0_mapping(struct domain *d);
 

[-- Attachment #3: debug-wip --]
[-- Type: text/plain, Size: 4505 bytes --]

# HG changeset patch
# Parent 80448ec3e90ba0a66a495cd6b42f6dec348b5e26

diff -r 80448ec3e90b xen/arch/x86/apic.c
--- a/xen/arch/x86/apic.c	Mon Jun 06 10:09:16 2011 +0100
+++ b/xen/arch/x86/apic.c	Mon Jun 06 14:06:31 2011 +0100
@@ -367,12 +367,15 @@ void disable_local_APIC(void)
         switch(apic_boot_mode)
         {
         case APIC_MODE_DISABLED:
+            printk("DBG: Returning APIC to disabled\n");
             break; /* Nothing to do - we did this above */
         case APIC_MODE_XAPIC:
+            printk("DBG: Returning APIC to xapic\n");
             msr_content |= MSR_IA32_APICBASE_ENABLE;
             wrmsrl(MSR_IA32_APICBASE, msr_content);
             break;
         case APIC_MODE_X2APIC:
+            printk("DBG: Returning APIC to x2apic\n");
             msr_content |= ( MSR_IA32_APICBASE_ENABLE | MSR_IA32_APICBASE_EXTD );
             wrmsrl(MSR_IA32_APICBASE, msr_content);
             break;
diff -r 80448ec3e90b xen/arch/x86/crash.c
--- a/xen/arch/x86/crash.c	Mon Jun 06 10:09:16 2011 +0100
+++ b/xen/arch/x86/crash.c	Mon Jun 06 14:06:31 2011 +0100
@@ -101,7 +101,15 @@ void machine_crash_shutdown(void)
 
     /* Crash shutdown any IOMMU functionality as the crashdump kernel is not
      * happy when booting if interrupt/dma remapping is still enabled */
-    iommu_crash_shutdown();
+    /* printk("DBG: About to attempt an iommu_crash_shutdown\n"); */
+    /* iommu_crash_shutdown(); */
+    /* printk("DBG: Supposedly done an iommu_crash_shutdown\n"); */
+
+    printk("DBG: About to suspend and disable IR\n");
+    iommu_suspend();
+    iommu_disable_IR();
+    printk("DBG: Done suspending and disabling IR\n");
+
 
     info = kexec_crash_save_info();
     info->xen_phys_start = xen_phys_start;
diff -r 80448ec3e90b xen/drivers/passthrough/amd/iommu_init.c
--- a/xen/drivers/passthrough/amd/iommu_init.c	Mon Jun 06 10:09:16 2011 +0100
+++ b/xen/drivers/passthrough/amd/iommu_init.c	Mon Jun 06 14:06:31 2011 +0100
@@ -927,6 +927,8 @@ void amd_iommu_crash_shutdown(void)
 {
     struct amd_iommu *iommu;
 
+    printk("DBG: In amd_iommu_crash_shutdown\n");
+
     for_each_amd_iommu ( iommu )
         disable_iommu(iommu);
 }
diff -r 80448ec3e90b xen/drivers/passthrough/iommu.c
--- a/xen/drivers/passthrough/iommu.c	Mon Jun 06 10:09:16 2011 +0100
+++ b/xen/drivers/passthrough/iommu.c	Mon Jun 06 14:06:31 2011 +0100
@@ -422,6 +422,7 @@ void iommu_suspend(void)
 void iommu_crash_shutdown(void)
 {
     const struct iommu_ops *ops = iommu_get_ops();
+    printk("DBG: In iommu_crash_shutdown\n");
     if ( ops && ops->crash_shutdown )
         ops->crash_shutdown();
 }
diff -r 80448ec3e90b xen/drivers/passthrough/vtd/intremap.c
--- a/xen/drivers/passthrough/vtd/intremap.c	Mon Jun 06 10:09:16 2011 +0100
+++ b/xen/drivers/passthrough/vtd/intremap.c	Mon Jun 06 14:06:31 2011 +0100
@@ -804,8 +804,13 @@ void disable_intremap(struct iommu *iomm
     u32 sts;
     unsigned long flags;
 
+    printk("DBG: in disable_intremap\n");
+n
     if ( !ecap_intr_remap(iommu->ecap) )
+    {
+        printk("DBG: BAIL - failed ecap\n");
         return;
+    }
 
     spin_lock_irqsave(&iommu->register_lock, flags);
     sts = dmar_readl(iommu->reg, DMAR_GSTS_REG);
@@ -813,7 +818,7 @@ void disable_intremap(struct iommu *iomm
         goto out;
 
     dmar_writel(iommu->reg, DMAR_GCMD_REG, sts & (~DMA_GCMD_IRE));
-
+    printk("DBG: written to DMAR\n");
     IOMMU_WAIT_OP(iommu, DMAR_GSTS_REG, dmar_readl,
                   !(sts & DMA_GSTS_IRES), sts);
 out:
@@ -885,8 +890,14 @@ void iommu_disable_IR(void)
 {
     struct acpi_drhd_unit *drhd;
 
+    printk("DBG: in iommu_disable_IR\n");
+
     if ( !iommu_supports_eim() )
+    {
+        printk("DBG: BAIL - not supporting EIM\n");
         return;
+    }
+
 
     for_each_drhd_unit ( drhd )
         disable_intremap(drhd->iommu);
diff -r 80448ec3e90b xen/drivers/passthrough/vtd/iommu.c
--- a/xen/drivers/passthrough/vtd/iommu.c	Mon Jun 06 10:09:16 2011 +0100
+++ b/xen/drivers/passthrough/vtd/iommu.c	Mon Jun 06 14:06:31 2011 +0100
@@ -2266,13 +2266,19 @@ static void vtd_crash_shutdown(void)
     struct acpi_drhd_unit *drhd;
     struct iommu *iommu;
 
+    printk("DBG: Starting vtd_crash_shutdown\n");
+
     if ( !iommu_enabled )
         return;
 
+    printk("DBG: iommu_enabled was true\n");
+
     iommu_flush_all();
 
     for_each_drhd_unit ( drhd )
     {
+        printk("DBG: in for_each_drhd_unit\n");
+
         iommu = drhd->iommu;
         iommu_disable_translation(iommu);
     }

[-- Attachment #4: apic-record-boot-mode.patch --]
[-- Type: text/x-patch, Size: 4264 bytes --]

APIC: record local APIC state on boot

Xen does not store the boot local APIC state which leads to problems
when shutting down for a kexec jump.  This patch records the boot
state so we can return to the boot state when kexecing.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

diff -r b0efa9e0087a xen/arch/x86/apic.c
--- a/xen/arch/x86/apic.c	Fri Jun 03 13:22:33 2011 +0100
+++ b/xen/arch/x86/apic.c	Mon Jun 06 10:09:12 2011 +0100
@@ -74,6 +74,8 @@ int apic_verbosity;
 static bool_t __initdata opt_x2apic = 1;
 boolean_param("x2apic", opt_x2apic);
 
+enum apic_mode apic_boot_mode = APIC_MODE_INVALID;
+
 bool_t __read_mostly x2apic_enabled = 0;
 bool_t __read_mostly directed_eoi_enabled = 0;
 
@@ -1482,6 +1484,61 @@ int __init APIC_init_uniprocessor (void)
     return 0;
 }
 
+/* Needs to be called during startup.  It records the state the BIOS
+ * leaves the local APIC so we can undo upon kexec.
+ */
+void __init record_boot_APIC_mode(void)
+{
+    /* Sanity check - we should only ever run once, but could possibly be called
+     * several times */
+    if ( APIC_MODE_INVALID != apic_boot_mode )
+        return;
+
+    apic_boot_mode = current_local_apic_mode();
+
+    apic_printk(APIC_DEBUG, "APIC boot state is '%s'\n", apic_mode_to_str(apic_boot_mode));
+}
+
+/* Look at the bits in MSR_IA32_APICBASE and work out which APIC mode we are in */
+enum apic_mode current_local_apic_mode(void)
+{
+    u64 msr_contents;
+
+    rdmsrl(MSR_IA32_APICBASE, msr_contents);
+
+    /* Reading EXTD bit from the MSR is only valid if CPUID says so, else reserved */
+    if ( cpu_has(&current_cpu_data, X86_FEATURE_X2APIC)
+         && (msr_contents & MSR_IA32_APICBASE_EXTD) )
+        return APIC_MODE_X2APIC;
+    else
+        {
+            /* EN bit should always be valid as long as we can read the MSR
+             */
+            if ( msr_contents & MSR_IA32_APICBASE_ENABLE )
+                return APIC_MODE_XAPIC;
+            else
+                return APIC_MODE_DISABLED;
+        }
+}
+
+
+const char * apic_mode_to_str(const enum apic_mode mode)
+{
+    switch(mode)
+        {
+        case APIC_MODE_INVALID:
+            return "invalid";
+        case APIC_MODE_DISABLED:
+            return "disabled";
+        case APIC_MODE_XAPIC:
+            return "xapic";
+        case APIC_MODE_X2APIC:
+            return "x2apic";
+        default:
+            return "unrecognised";
+        }
+}
+
 void check_for_unexpected_msi(unsigned int vector)
 {
     unsigned long v = apic_read(APIC_ISR + ((vector & ~0x1f) >> 1));
diff -r b0efa9e0087a xen/arch/x86/genapic/probe.c
--- a/xen/arch/x86/genapic/probe.c	Fri Jun 03 13:22:33 2011 +0100
+++ b/xen/arch/x86/genapic/probe.c	Mon Jun 06 10:09:12 2011 +0100
@@ -61,6 +61,8 @@ void __init generic_apic_probe(void)
 { 
 	int i, changed;
 
+	record_boot_APIC_mode();
+
 	check_x2apic_preenabled();
 	cmdline_apic = changed = (genapic != NULL);
 
diff -r b0efa9e0087a xen/include/asm-x86/apic.h
--- a/xen/include/asm-x86/apic.h	Fri Jun 03 13:22:33 2011 +0100
+++ b/xen/include/asm-x86/apic.h	Mon Jun 06 10:09:12 2011 +0100
@@ -21,6 +21,21 @@
 #define IO_APIC_REDIR_DEST_LOGICAL	0x00800
 #define IO_APIC_REDIR_DEST_PHYSICAL	0x00000
 
+/* Possible APIC states */
+enum apic_mode { APIC_MODE_INVALID,  /* Not set yet */
+                 APIC_MODE_DISABLED, /* If uniprocessor, or smp in uniprocessor mode */
+                 APIC_MODE_XAPIC,    /* xAPIC mode - default upon chipset reset */
+                 APIC_MODE_X2APIC    /* x2APIC mode - common for large smp machines */
+};
+
+/* Bootstrap processor local APIC boot mode - so we can undo our changes to the APIC state */
+extern enum apic_mode apic_boot_mode;
+
+/* enum apic_mode -> str function for logging/debug */
+const char * apic_mode_to_str(const enum apic_mode);
+
+
+
 extern int apic_verbosity;
 extern bool_t x2apic_enabled;
 extern bool_t directed_eoi_enabled;
@@ -206,6 +221,8 @@ extern void disable_APIC_timer(void);
 extern void enable_APIC_timer(void);
 extern int lapic_suspend(void);
 extern int lapic_resume(void);
+extern void record_boot_APIC_mode(void);
+extern enum apic_mode current_local_apic_mode(void);
 
 extern int check_nmi_watchdog (void);
 extern void enable_NMI_through_LVT0 (void * dummy);

[-- Attachment #5: kexec-fix-x2apic.patch --]
[-- Type: text/x-patch, Size: 5347 bytes --]

KEXEC: correctly revert x2apic state when kexecing

Introduce the boolean variable 'kexecing' which indicates to
the more general functions whether we are on the kexec path
or not.  This is used by disable_local_APIC() to try and revert
the APIC mode back to how it was found on boot, and used by
hpet_disable_legacy_broadcast() to prevent it IPI'ing the
other processors after they have already been shot down.

We also need some fudging of the x2apic_enabled variable
to prevent a protection fault when disabling the IOAPICs.

Finally, make sure we dont jump into the purgatory code with
interupts enabled.  We dont want to be servicing stray
interupts once we are out of Xen.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

diff -r c656613b1e73 xen/arch/x86/apic.c
--- a/xen/arch/x86/apic.c	Fri Jun 03 13:22:33 2011 +0100
+++ b/xen/arch/x86/apic.c	Fri Jun 03 16:19:00 2011 +0100
@@ -37,6 +37,7 @@
 #include <asm/asm_defns.h> /* for BUILD_SMP_INTERRUPT */
 #include <mach_apic.h>
 #include <io_ports.h>
+#include <xen/kexec.h>
 
 static bool_t tdt_enabled __read_mostly;
 static bool_t tdt_enable __initdata = 1;
@@ -356,6 +357,32 @@ void disable_local_APIC(void)
         wrmsrl(MSR_IA32_APICBASE, msr_content &
                ~(MSR_IA32_APICBASE_ENABLE|MSR_IA32_APICBASE_EXTD));
     }
+
+    if (kexecing){
+        uint64_t msr_content;
+        rdmsrl(MSR_IA32_APICBASE, msr_content);
+        msr_content &= ~ ( MSR_IA32_APICBASE_ENABLE | MSR_IA32_APICBASE_EXTD );
+        wrmsrl(MSR_IA32_APICBASE, msr_content);
+
+        switch(apic_boot_mode)
+        {
+        case APIC_MODE_DISABLED:
+            break; /* Nothing to do - we did this above */
+        case APIC_MODE_XAPIC:
+            msr_content |= MSR_IA32_APICBASE_ENABLE;
+            wrmsrl(MSR_IA32_APICBASE, msr_content);
+            break;
+        case APIC_MODE_X2APIC:
+            msr_content |= ( MSR_IA32_APICBASE_ENABLE | MSR_IA32_APICBASE_EXTD );
+            wrmsrl(MSR_IA32_APICBASE, msr_content);
+            break;
+        default:
+            printk("Hit default case when reverting lapic to boot state on core #%d\n",
+                   smp_processor_id());
+            break;
+        }
+    }
+
 }
 
 extern int ioapic_ack_new;
diff -r c656613b1e73 xen/arch/x86/crash.c
--- a/xen/arch/x86/crash.c	Fri Jun 03 13:22:33 2011 +0100
+++ b/xen/arch/x86/crash.c	Fri Jun 03 16:19:00 2011 +0100
@@ -27,6 +27,7 @@
 #include <asm/hvm/support.h>
 #include <asm/apic.h>
 #include <asm/io_apic.h>
+#include <xen/iommu.h>
 
 static atomic_t waiting_for_crash_ipi;
 static unsigned int crashing_cpu;
@@ -78,6 +79,15 @@ static void nmi_shootdown_cpus(void)
     }
 
     __stop_this_cpu();
+
+    /* This is a bit of a hack due to the problems with the x2apic_enabled
+     * variable, but we can't do any better without a significant refactoring
+     * of the APIC code */
+    if ( current_local_apic_mode() == APIC_MODE_X2APIC )
+        x2apic_enabled = 1;
+    else
+        x2apic_enabled = 0;
+
     disable_IO_APIC();
 
     local_irq_restore(flags);
diff -r c656613b1e73 xen/arch/x86/hpet.c
--- a/xen/arch/x86/hpet.c	Fri Jun 03 13:22:33 2011 +0100
+++ b/xen/arch/x86/hpet.c	Fri Jun 03 16:19:00 2011 +0100
@@ -663,7 +663,8 @@ void hpet_disable_legacy_broadcast(void)
 
     spin_unlock_irqrestore(&legacy_hpet_event.lock, flags);
 
-    smp_send_event_check_mask(&cpu_online_map);
+    if (! kexecing)
+        smp_send_event_check_mask(&cpu_online_map);
 }
 
 void hpet_broadcast_enter(void)
diff -r c656613b1e73 xen/arch/x86/machine_kexec.c
--- a/xen/arch/x86/machine_kexec.c	Fri Jun 03 13:22:33 2011 +0100
+++ b/xen/arch/x86/machine_kexec.c	Fri Jun 03 16:19:00 2011 +0100
@@ -99,6 +99,11 @@ void machine_kexec(xen_kexec_image_t *im
     if ( hpet_broadcast_is_available() )
         hpet_disable_legacy_broadcast();
 
+    /* We are about to permenantly jump out of the Xen context into the kexec
+     * purgatory code.  We really dont want to be still servicing interupts.
+     */
+    local_irq_disable();
+
     /*
      * compat_machine_kexec() returns to idle pagetables, which requires us
      * to be running on a static GDT mapping (idle pagetables have no GDT
diff -r c656613b1e73 xen/common/kexec.c
--- a/xen/common/kexec.c	Fri Jun 03 13:22:33 2011 +0100
+++ b/xen/common/kexec.c	Fri Jun 03 16:19:00 2011 +0100
@@ -29,6 +29,8 @@
 #include <compat/kexec.h>
 #endif
 
+bool_t kexecing = FALSE;
+
 static DEFINE_PER_CPU_READ_MOSTLY(void *, crash_notes);
 
 static Elf_Note *xen_crash_note;
@@ -238,6 +240,8 @@ void kexec_crash(void)
     if ( !test_bit(KEXEC_IMAGE_CRASH_BASE + pos, &kexec_flags) )
         return;
 
+    kexecing = TRUE;
+
     kexec_common_shutdown();
     kexec_crash_save_cpu();
     machine_crash_shutdown();
@@ -250,6 +254,8 @@ static long kexec_reboot(void *_image)
 {
     xen_kexec_image_t *image = _image;
 
+    kexecing = TRUE;
+
     kexec_common_shutdown();
     machine_reboot_kexec(image);
 
diff -r c656613b1e73 xen/include/xen/kexec.h
--- a/xen/include/xen/kexec.h	Fri Jun 03 13:22:33 2011 +0100
+++ b/xen/include/xen/kexec.h	Fri Jun 03 16:19:00 2011 +0100
@@ -12,6 +12,8 @@ typedef struct xen_kexec_reserve {
 
 extern xen_kexec_reserve_t kexec_crash_area;
 
+extern bool_t kexecing;
+
 void set_kexec_crash_area_size(u64 system_ram);
 
 /* We have space for 4 images to support atomic update

[-- Attachment #6: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel