* [PATCH 1/3] x86/apic: Set up through LAPIC on boot CPU's LINT0 if ioapic is disabled to kexec/kdump kernel
2018-01-05 2:55 [PATCH 0/3] x86/apic/kexec: Enable legacy irq mode before jump to kexec/kdump kernel Baoquan He
@ 2018-01-05 3:22 ` Baoquan He
2018-01-05 3:26 ` Baoquan He
2018-01-05 3:23 ` [PATCH 2/3] x86/apic/kexec: Enable legacy irq mode before jump " Baoquan He
2018-01-05 3:24 ` [PATCH 3/3] x86/apic: Clean up the names of legacy irq mode setting related functions Baoquan He
2 siblings, 1 reply; 5+ messages in thread
From: Baoquan He @ 2018-01-05 3:22 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, tglx, hpa, x86, douly.fnst, rostedt, jgross, peterz,
uobergfe, joro
Kdump kernel will become very slow if 'noapic' is specified in kernel
command line. Normal kernel doesn't have this issue.
This is because the legacy irq mode is disabled in crashed kernel before
jump jump to kdump kernel since commit 522e66464467 ("x86/apic: Disable I/O
APIC before shutdown of the local APIC") is merged. While in normal kernel,
the legacy irq mode has been set in BIOS.
So we need set the delivery mode AS ExtINT for LVT0 of boot CPU's LAPIC
explicitly if IO-APIC is disabled, to set up through-local-APIC.
Signed-off-by: Baoquan He <bhe@redhat.com>
---
arch/x86/kernel/apic/apic.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index 880441f24146..7e613fb90630 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -1521,7 +1521,7 @@ void setup_local_APIC(void)
* TODO: set up through-local-APIC from through-I/O-APIC? --macro
*/
value = apic_read(APIC_LVT0) & APIC_LVT_MASKED;
- if (!cpu && (pic_mode || !value)) {
+ if (!cpu && (pic_mode || !value || skip_ioapic_setup)) {
value = APIC_DM_EXTINT;
apic_printk(APIC_VERBOSE, "enabled ExtINT on CPU#%d\n", cpu);
} else {
--
2.5.5
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH 1/3] x86/apic: Set up through LAPIC on boot CPU's LINT0 if ioapic is disabled to kexec/kdump kernel
2018-01-05 3:22 ` [PATCH 1/3] x86/apic: Set up through LAPIC on boot CPU's LINT0 if ioapic is disabled " Baoquan He
@ 2018-01-05 3:26 ` Baoquan He
0 siblings, 0 replies; 5+ messages in thread
From: Baoquan He @ 2018-01-05 3:26 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, tglx, hpa, x86, douly.fnst, rostedt, jgross, peterz,
uobergfe, joro
Sorry, my git send-email config seems incorrect. The patch subject is
not right. So NACK this series, will repost v2.
On 01/05/18 at 11:22am, Baoquan He wrote:
> Kdump kernel will become very slow if 'noapic' is specified in kernel
> command line. Normal kernel doesn't have this issue.
>
> This is because the legacy irq mode is disabled in crashed kernel before
> jump jump to kdump kernel since commit 522e66464467 ("x86/apic: Disable I/O
> APIC before shutdown of the local APIC") is merged. While in normal kernel,
> the legacy irq mode has been set in BIOS.
>
> So we need set the delivery mode AS ExtINT for LVT0 of boot CPU's LAPIC
> explicitly if IO-APIC is disabled, to set up through-local-APIC.
>
> Signed-off-by: Baoquan He <bhe@redhat.com>
> ---
> arch/x86/kernel/apic/apic.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
> index 880441f24146..7e613fb90630 100644
> --- a/arch/x86/kernel/apic/apic.c
> +++ b/arch/x86/kernel/apic/apic.c
> @@ -1521,7 +1521,7 @@ void setup_local_APIC(void)
> * TODO: set up through-local-APIC from through-I/O-APIC? --macro
> */
> value = apic_read(APIC_LVT0) & APIC_LVT_MASKED;
> - if (!cpu && (pic_mode || !value)) {
> + if (!cpu && (pic_mode || !value || skip_ioapic_setup)) {
> value = APIC_DM_EXTINT;
> apic_printk(APIC_VERBOSE, "enabled ExtINT on CPU#%d\n", cpu);
> } else {
> --
> 2.5.5
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 2/3] x86/apic/kexec: Enable legacy irq mode before jump to kexec/kdump kernel
2018-01-05 2:55 [PATCH 0/3] x86/apic/kexec: Enable legacy irq mode before jump to kexec/kdump kernel Baoquan He
2018-01-05 3:22 ` [PATCH 1/3] x86/apic: Set up through LAPIC on boot CPU's LINT0 if ioapic is disabled " Baoquan He
@ 2018-01-05 3:23 ` Baoquan He
2018-01-05 3:24 ` [PATCH 3/3] x86/apic: Clean up the names of legacy irq mode setting related functions Baoquan He
2 siblings, 0 replies; 5+ messages in thread
From: Baoquan He @ 2018-01-05 3:23 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, tglx, hpa, x86, douly.fnst, rostedt, jgross, peterz,
uobergfe, joro
In commit
commit 522e66464467 ("x86/apic: Disable I/O APIC before shutdown of the local APIC").
lapic_shutdown() invocation is moved after disable_IO_APIC(). In fact
in disable_IO_APIC(), it not only calls clear_IO_APIC() to disable
IO-APIC, also sets LAPIC and IO-APIC to make system be PIC or
Virtual wire mode. While the above commit putting disable_IO_APIC earlier
causes local APIC completely disabled. So the legacy irq mode is
disabled too before jump to kexec/kdump kernel.
In normal kernel it defaults to be PIC mode or Virtual Wire mode during
system initialization before APIC mode is enabled and this is done by
BIOS initialization. But kexec/kdump kernel won't go through BIOS, so
we should set system as PIC or Virtual Wire mode before jump to kdump
kernel code directly.
Let's take clear_IO_APIC out from disable_IO_APIC and rename
disable_IO_APIC as switch_to_legacy_irq_mode. Then only call clear_IO_APIC
when IO-APIC need be disabled. And call switch_to_legacy_irq_mode before
kexec/kdump jumping.
Signed-off-by: Baoquan He <bhe@redhat.com>
---
arch/x86/include/asm/io_apic.h | 3 ++-
arch/x86/kernel/apic/io_apic.c | 12 ++++--------
arch/x86/kernel/crash.c | 2 +-
arch/x86/kernel/machine_kexec_32.c | 15 +++++----------
arch/x86/kernel/machine_kexec_64.c | 15 +++++----------
arch/x86/kernel/reboot.c | 2 +-
6 files changed, 18 insertions(+), 31 deletions(-)
diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index a8834dd546cd..e38ad3863a2c 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -192,7 +192,8 @@ static inline unsigned int io_apic_read(unsigned int apic, unsigned int reg)
extern void setup_IO_APIC(void);
extern void enable_IO_APIC(void);
-extern void disable_IO_APIC(void);
+extern void clear_IO_APIC(void);
+extern void switch_to_legacy_irq_mode(void);
extern int IO_APIC_get_PCI_irq_vector(int bus, int devfn, int pin);
extern void print_IO_APICs(void);
#else /* !CONFIG_X86_IO_APIC */
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index 8a7963421460..a47aa915d18c 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -587,7 +587,7 @@ static void clear_IO_APIC_pin(unsigned int apic, unsigned int pin)
mpc_ioapic_id(apic), pin);
}
-static void clear_IO_APIC(void)
+void clear_IO_APIC(void)
{
int apic, pin;
@@ -1439,15 +1439,11 @@ void native_disable_io_apic(void)
}
/*
- * Not an __init, needed by the reboot code
+ * Not an __init, needed by kexec/kdump code.
+ * For safety IO-APIC and Local APIC need be cleared before this.
*/
-void disable_IO_APIC(void)
+void switch_to_legacy_irq_mode(void)
{
- /*
- * Clear the IO-APIC before rebooting:
- */
- clear_IO_APIC();
-
if (!nr_legacy_irqs())
return;
diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c
index 10e74d4778a1..318ffeaaf55a 100644
--- a/arch/x86/kernel/crash.c
+++ b/arch/x86/kernel/crash.c
@@ -199,7 +199,7 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
#ifdef CONFIG_X86_IO_APIC
/* Prevent crash_kexec() from deadlocking on ioapic_lock. */
ioapic_zap_locks();
- disable_IO_APIC();
+ clear_IO_APIC();
#endif
lapic_shutdown();
#ifdef CONFIG_HPET_TIMER
diff --git a/arch/x86/kernel/machine_kexec_32.c b/arch/x86/kernel/machine_kexec_32.c
index edfede768688..7ab10d930cc6 100644
--- a/arch/x86/kernel/machine_kexec_32.c
+++ b/arch/x86/kernel/machine_kexec_32.c
@@ -190,18 +190,13 @@ void machine_kexec(struct kimage *image)
local_irq_disable();
hw_breakpoint_disable();
- if (image->preserve_context) {
#ifdef CONFIG_X86_IO_APIC
- /*
- * We need to put APICs in legacy mode so that we can
- * get timer interrupts in second kernel. kexec/kdump
- * paths already have calls to disable_IO_APIC() in
- * one form or other. kexec jump path also need
- * one.
- */
- disable_IO_APIC();
+ /*
+ * We need to put APICs in legacy mode so that we can
+ * get timer interrupts in second kernel.
+ */
+ switch_to_legacy_irq_mode();
#endif
- }
control_page = page_address(image->control_code_page);
memcpy(control_page, relocate_kernel, KEXEC_CONTROL_CODE_MAX_SIZE);
diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
index 1f790cf9d38f..b5c0cbed6791 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -288,18 +288,13 @@ void machine_kexec(struct kimage *image)
local_irq_disable();
hw_breakpoint_disable();
- if (image->preserve_context) {
#ifdef CONFIG_X86_IO_APIC
- /*
- * We need to put APICs in legacy mode so that we can
- * get timer interrupts in second kernel. kexec/kdump
- * paths already have calls to disable_IO_APIC() in
- * one form or other. kexec jump path also need
- * one.
- */
- disable_IO_APIC();
+ /*
+ * We need to put APICs in legacy mode so that we can
+ * get timer interrupts in second kernel.
+ */
+ switch_to_legacy_irq_mode();
#endif
- }
control_page = page_address(image->control_code_page) + PAGE_SIZE;
memcpy(control_page, relocate_kernel, KEXEC_CONTROL_CODE_MAX_SIZE);
diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c
index 2126b9d27c34..b70cc0f38a29 100644
--- a/arch/x86/kernel/reboot.c
+++ b/arch/x86/kernel/reboot.c
@@ -666,7 +666,7 @@ void native_machine_shutdown(void)
* Even without the erratum, it still makes sense to quiet IO APIC
* before disabling Local APIC.
*/
- disable_IO_APIC();
+ clear_IO_APIC();
#endif
#ifdef CONFIG_SMP
--
2.5.5
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PATCH 3/3] x86/apic: Clean up the names of legacy irq mode setting related functions
2018-01-05 2:55 [PATCH 0/3] x86/apic/kexec: Enable legacy irq mode before jump to kexec/kdump kernel Baoquan He
2018-01-05 3:22 ` [PATCH 1/3] x86/apic: Set up through LAPIC on boot CPU's LINT0 if ioapic is disabled " Baoquan He
2018-01-05 3:23 ` [PATCH 2/3] x86/apic/kexec: Enable legacy irq mode before jump " Baoquan He
@ 2018-01-05 3:24 ` Baoquan He
2 siblings, 0 replies; 5+ messages in thread
From: Baoquan He @ 2018-01-05 3:24 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, tglx, hpa, x86, douly.fnst, rostedt, jgross, peterz,
uobergfe, joro
X86 MP spec defines 3 different interrupt modes:
1) PIC Mode—bypasses all APIC components and forces the system to
operate in single-processor mode.
2) Virtual Wire Mode—uses an APIC as a virtual wire, but otherwise
operates the same as PIC Mode.
3) Symmetric I/O Mode—enables the system to operate with more than
one processor.
The current disconnect_bsp_APIC includes two parts: one is to set system
as PIC mode if it's available, the other is to change system back to
Virtual Wire mode. Only PIC mode will detach the APIC from the interrupt
system, Virtual Wire mode doesn't.
Besides Virutal Wire mode has two kinds: one is only setting Local APIC
as Virtual Wire mode and interrupts are delivered from the PIC to the
CPU which Local APIC connected to, the other is both Loca APIC and IO-APIC
need be set as Virtual Wire mode.
So based on above knowledge, take IO-APIC Virtual Wire mode setting code
out and wrap it inot a new function ioapic_set_virtual_wire_mode. Meanwhile
change the name of disconnect_bsp_APIC as lapic_set_legacy_irq_mode. These
makes the legacy irq mode setting more understandable.
Signed-off-by: Baoquan He <bhe@redhat.com>
---
arch/x86/include/asm/apic.h | 2 +-
arch/x86/include/asm/io_apic.h | 5 ++---
arch/x86/kernel/apic/apic.c | 11 ++++++-----
arch/x86/kernel/apic/io_apic.c | 17 ++++++++++-------
arch/x86/kernel/x86_init.c | 2 +-
drivers/iommu/irq_remapping.c | 2 +-
6 files changed, 21 insertions(+), 18 deletions(-)
diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index a9e57f08bfa6..004c48bc8bc8 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -132,7 +132,7 @@ extern int get_physical_broadcast(void);
extern int lapic_get_maxlvt(void);
extern void clear_local_APIC(void);
-extern void disconnect_bsp_APIC(int virt_wire_setup);
+extern void lapic_set_legacy_irq_mode(int virt_wire_setup);
extern void disable_local_APIC(void);
extern void lapic_shutdown(void);
extern void sync_Arb_IDs(void);
diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h
index e38ad3863a2c..6800dcea1d21 100644
--- a/arch/x86/include/asm/io_apic.h
+++ b/arch/x86/include/asm/io_apic.h
@@ -183,7 +183,7 @@ extern void disable_ioapic_support(void);
extern void __init io_apic_init_mappings(void);
extern unsigned int native_io_apic_read(unsigned int apic, unsigned int reg);
-extern void native_disable_io_apic(void);
+extern void switch_to_legacy_irq_mode(void);
static inline unsigned int io_apic_read(unsigned int apic, unsigned int reg)
{
@@ -193,7 +193,6 @@ static inline unsigned int io_apic_read(unsigned int apic, unsigned int reg)
extern void setup_IO_APIC(void);
extern void enable_IO_APIC(void);
extern void clear_IO_APIC (void);
-extern void switch_to_legacy_irq_mode(void);
extern int IO_APIC_get_PCI_irq_vector(int bus, int devfn, int pin);
extern void print_IO_APICs(void);
#else /* !CONFIG_X86_IO_APIC */
@@ -229,7 +228,7 @@ static inline void mp_save_irq(struct mpc_intsrc *m) { }
static inline void disable_ioapic_support(void) { }
static inline void io_apic_init_mappings(void) { }
#define native_io_apic_read NULL
-#define native_disable_io_apic NULL
+#define switch_to_legacy_irq_mode NULL
static inline void setup_IO_APIC(void) { }
static inline void enable_IO_APIC(void) { }
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index 7e613fb90630..301d90d4a0c3 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -2050,13 +2050,14 @@ static void __init connect_bsp_APIC(void)
}
/**
- * disconnect_bsp_APIC - detach the APIC from the interrupt system
- * @virt_wire_setup: indicates, whether virtual wire mode is selected
+ * lapic_set_legacy_irq_mode - switch Local APIC back to be legacy irq mode.
+ * @virt_wire_setup: indicates, whether virtual wire mode is selected
*
- * Virtual wire mode is necessary to deliver legacy interrupts even when the
- * APIC is disabled.
+ * If PIC mode is available, LAPIC need be disconnected with CPU. Otherwise
+ * enable LAPIC and set it to be virtual wire mode. However if IO-APIC has
+ * been virtual wire mode, LVT0 of LAPIC need be masked.
*/
-void disconnect_bsp_APIC(int virt_wire_setup)
+void lapic_set_legacy_irq_mode(int virt_wire_setup)
{
unsigned int value;
diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c
index a47aa915d18c..b7fd4236b0e5 100644
--- a/arch/x86/kernel/apic/io_apic.c
+++ b/arch/x86/kernel/apic/io_apic.c
@@ -1410,7 +1410,7 @@ void __init enable_IO_APIC(void)
clear_IO_APIC();
}
-void native_disable_io_apic(void)
+static void ioapic_set_virtual_wire_mode(void)
{
/*
* If the i8259 is routed through an IOAPIC
@@ -1433,21 +1433,24 @@ void native_disable_io_apic(void)
*/
ioapic_write_entry(ioapic_i8259.apic, ioapic_i8259.pin, entry);
}
-
- if (boot_cpu_has(X86_FEATURE_APIC) || apic_from_smp_config())
- disconnect_bsp_APIC(ioapic_i8259.pin != -1);
}
/*
- * Not an __init, needed by kexec/kdump code.
- * For safety IO-APIC and Local APIC need be cleared before this.
+ * In legacy irq mode, full DOS compatibility with the uniprocessor PC/AT is
+ * provided by using the APICs in conjunction with standard 8259A-equivalent
+ * programmable interrupt controllers (PICs). It's necessary to deliver legacy
+ * interrupts even when APIC mode is not enabled. This is required by kexec/
+ * kdump before enter into the 2nd kernel.
*/
void switch_to_legacy_irq_mode(void)
{
if (!nr_legacy_irqs())
return;
- x86_io_apic_ops.disable();
+ ioapic_set_virtual_wire_mode();
+
+ if (boot_cpu_has(X86_FEATURE_APIC) || apic_from_smp_config())
+ lapic_set_legacy_irq_mode(ioapic_i8259.pin != -1);
}
#ifdef CONFIG_X86_32
diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c
index 1151ccd72ce9..c30f0f273dbd 100644
--- a/arch/x86/kernel/x86_init.c
+++ b/arch/x86/kernel/x86_init.c
@@ -148,5 +148,5 @@ void arch_restore_msi_irqs(struct pci_dev *dev)
struct x86_io_apic_ops x86_io_apic_ops __ro_after_init = {
.read = native_io_apic_read,
- .disable = native_disable_io_apic,
+ .disable = switch_to_legacy_irq_mode,
};
diff --git a/drivers/iommu/irq_remapping.c b/drivers/iommu/irq_remapping.c
index 49721b4e1975..751472ddf536 100644
--- a/drivers/iommu/irq_remapping.c
+++ b/drivers/iommu/irq_remapping.c
@@ -37,7 +37,7 @@ static void irq_remapping_disable_io_apic(void)
* now.
*/
if (boot_cpu_has(X86_FEATURE_APIC) || apic_from_smp_config())
- disconnect_bsp_APIC(0);
+ lapic_set_legacy_irq_mode(0);
}
static void __init irq_remapping_modify_x86_ops(void)
--
2.5.5
^ permalink raw reply related [flat|nested] 5+ messages in thread