[PATCH] KEXEC: disable hpet legacy broadcasts earlier

[PATCH] KEXEC: disable hpet legacy broadcasts earlier

[Andrew Cooper]

On x2apic machines which booted in xapic mode,
hpet_disable_legacy_broadcast() sends an event check IPI to all online
processors.  This leads to a protection fault as the genapic blindly
pokes x2apic MSRs while the local apic is in xapic mode.

One option is to change genapic when we shut down the local apic, but
there are still problems with trying to IPI processors in the online
processor map which are actually sitting in NMI loops

Another option is to have each CPU take itself out of the online CPU
map during the NMI shootdown.

Realistically however, disabling hpet legacy broadcasts earlier in the
kexec path is the easiest fix to the problem.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

diff -r fc2be6cb89ad -r dcb3ed7b7dbd xen/arch/x86/crash.c
--- a/xen/arch/x86/crash.c	Tue Aug 16 15:05:55 2011 +0100
+++ b/xen/arch/x86/crash.c	Thu Aug 18 14:30:32 2011 +0100
@@ -29,6 +29,7 @@
 #include <asm/io_apic.h>
 #include <xen/iommu.h>
 #include <xen/pci.h>
+#include <asm/hpet.h>
 
 static atomic_t waiting_for_crash_ipi;
 static unsigned int crashing_cpu;
@@ -61,6 +62,9 @@ static void nmi_shootdown_cpus(void)
 
     local_irq_disable();
 
+    if ( hpet_broadcast_is_available() )
+        hpet_disable_legacy_broadcast();
+
     crashing_cpu = smp_processor_id();
     local_irq_count(crashing_cpu) = 0;
 
diff -r fc2be6cb89ad -r dcb3ed7b7dbd xen/arch/x86/machine_kexec.c
--- a/xen/arch/x86/machine_kexec.c	Tue Aug 16 15:05:55 2011 +0100
+++ b/xen/arch/x86/machine_kexec.c	Thu Aug 18 14:30:32 2011 +0100
@@ -85,9 +85,6 @@ void machine_kexec(xen_kexec_image_t *im
         .limit = LAST_RESERVED_GDT_BYTE
     };
 
-    if ( hpet_broadcast_is_available() )
-        hpet_disable_legacy_broadcast();
-
     /* We are about to permenantly jump out of the Xen context into the kexec
      * purgatory code.  We really dont want to be still servicing interupts.
      */

Re: [PATCH] KEXEC: disable hpet legacy broadcasts earlier

[Keir Fraser]

On 18/08/2011 14:31, "Andrew Cooper" <andrew.cooper3@citrix.com> wrote:

> On x2apic machines which booted in xapic mode,
> hpet_disable_legacy_broadcast() sends an event check IPI to all online
> processors.  This leads to a protection fault as the genapic blindly
> pokes x2apic MSRs while the local apic is in xapic mode.
> 
> One option is to change genapic when we shut down the local apic, but
> there are still problems with trying to IPI processors in the online
> processor map which are actually sitting in NMI loops
> 
> Another option is to have each CPU take itself out of the online CPU
> map during the NMI shootdown.
> 
> Realistically however, disabling hpet legacy broadcasts earlier in the
> kexec path is the easiest fix to the problem.

This for 4.1.2 as well I guess?

> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> diff -r fc2be6cb89ad -r dcb3ed7b7dbd xen/arch/x86/crash.c
> --- a/xen/arch/x86/crash.c Tue Aug 16 15:05:55 2011 +0100
> +++ b/xen/arch/x86/crash.c Thu Aug 18 14:30:32 2011 +0100
> @@ -29,6 +29,7 @@
>  #include <asm/io_apic.h>
>  #include <xen/iommu.h>
>  #include <xen/pci.h>
> +#include <asm/hpet.h>
>  
>  static atomic_t waiting_for_crash_ipi;
>  static unsigned int crashing_cpu;
> @@ -61,6 +62,9 @@ static void nmi_shootdown_cpus(void)
>  
>      local_irq_disable();
>  
> +    if ( hpet_broadcast_is_available() )
> +        hpet_disable_legacy_broadcast();
> +
>      crashing_cpu = smp_processor_id();
>      local_irq_count(crashing_cpu) = 0;
>  
> diff -r fc2be6cb89ad -r dcb3ed7b7dbd xen/arch/x86/machine_kexec.c
> --- a/xen/arch/x86/machine_kexec.c Tue Aug 16 15:05:55 2011 +0100
> +++ b/xen/arch/x86/machine_kexec.c Thu Aug 18 14:30:32 2011 +0100
> @@ -85,9 +85,6 @@ void machine_kexec(xen_kexec_image_t *im
>          .limit = LAST_RESERVED_GDT_BYTE
>      };
>  
> -    if ( hpet_broadcast_is_available() )
> -        hpet_disable_legacy_broadcast();
> -
>      /* We are about to permenantly jump out of the Xen context into the kexec
>       * purgatory code.  We really dont want to be still servicing interupts.
>       */
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: [PATCH] KEXEC: disable hpet legacy broadcasts earlier

[Andrew Cooper]

On 18/08/11 14:47, Keir Fraser wrote:
> On 18/08/2011 14:31, "Andrew Cooper" <andrew.cooper3@citrix.com> wrote:
>
>> On x2apic machines which booted in xapic mode,
>> hpet_disable_legacy_broadcast() sends an event check IPI to all online
>> processors.  This leads to a protection fault as the genapic blindly
>> pokes x2apic MSRs while the local apic is in xapic mode.
>>
>> One option is to change genapic when we shut down the local apic, but
>> there are still problems with trying to IPI processors in the online
>> processor map which are actually sitting in NMI loops
>>
>> Another option is to have each CPU take itself out of the online CPU
>> map during the NMI shootdown.
>>
>> Realistically however, disabling hpet legacy broadcasts earlier in the
>> kexec path is the easiest fix to the problem.
> This for 4.1.2 as well I guess?

Yes please

~Andrew

>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>
>> diff -r fc2be6cb89ad -r dcb3ed7b7dbd xen/arch/x86/crash.c
>> --- a/xen/arch/x86/crash.c Tue Aug 16 15:05:55 2011 +0100
>> +++ b/xen/arch/x86/crash.c Thu Aug 18 14:30:32 2011 +0100
>> @@ -29,6 +29,7 @@
>>  #include <asm/io_apic.h>
>>  #include <xen/iommu.h>
>>  #include <xen/pci.h>
>> +#include <asm/hpet.h>
>>  
>>  static atomic_t waiting_for_crash_ipi;
>>  static unsigned int crashing_cpu;
>> @@ -61,6 +62,9 @@ static void nmi_shootdown_cpus(void)
>>  
>>      local_irq_disable();
>>  
>> +    if ( hpet_broadcast_is_available() )
>> +        hpet_disable_legacy_broadcast();
>> +
>>      crashing_cpu = smp_processor_id();
>>      local_irq_count(crashing_cpu) = 0;
>>  
>> diff -r fc2be6cb89ad -r dcb3ed7b7dbd xen/arch/x86/machine_kexec.c
>> --- a/xen/arch/x86/machine_kexec.c Tue Aug 16 15:05:55 2011 +0100
>> +++ b/xen/arch/x86/machine_kexec.c Thu Aug 18 14:30:32 2011 +0100
>> @@ -85,9 +85,6 @@ void machine_kexec(xen_kexec_image_t *im
>>          .limit = LAST_RESERVED_GDT_BYTE
>>      };
>>  
>> -    if ( hpet_broadcast_is_available() )
>> -        hpet_disable_legacy_broadcast();
>> -
>>      /* We are about to permenantly jump out of the Xen context into the kexec
>>       * purgatory code.  We really dont want to be still servicing interupts.
>>       */
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>

-- 
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com