* [PATCH v6 1/3] x86, apic: add bios_bsp_physical_apicid
2013-11-21 2:00 [PATCH v6 0/3] x86, apic, kexec: Add disable_cpu_apic kernel parameter HATAYAMA Daisuke
@ 2013-11-21 2:00 ` HATAYAMA Daisuke
2013-11-21 2:00 ` [PATCH v6 2/3] x86, apic: Add disable_cpu_apicid kernel parameter HATAYAMA Daisuke
2013-11-21 2:00 ` [PATCH v6 3/3] Documentation, x86, apic, kexec: " HATAYAMA Daisuke
2 siblings, 0 replies; 7+ messages in thread
From: HATAYAMA Daisuke @ 2013-11-21 2:00 UTC (permalink / raw)
To: hpa, ebiederm, vgoyal
Cc: kexec, linux-kernel, bp, akpm, fengguang.wu, jingbai.ma
Add bios_bsp_physical_apicid variable. This variable is initialized
with the value reported by BIOS tables such as ACPI MADT or MP table.
Without this variable, boot_cpu_physical_apicid temporarilly has the
value around MP table related codes such as kernel/mpparse.c,
mm/amdtopology.c and platform/visws/visws_quirks.c until it is
rewritten back in init_apic_mappings() to the initial APIC ID for the
processor that is doing boot up.
This change is also required by the next commit introducing new
disable_cpu_apicid kernel parameter, where the value specified via
disable_cpu_apicid is compared with boot_cpu_physical_apicid in
generic_processor_info(), but in case of using MP table, the
boot_cpu_physical_apicid is rewiriten by MP_processor_info() to the
initial APIC ID of the BSP reported by BIOS, which is problematic if
some AP is specified in disable_cpu_apic.
There's no functional change intended on kernel/mpparse.c,
mm/amdtopology.c and platform/visws/visws_quirks.c by the introduction
of bios_bsp_physical_apicid.
The reason why we don't use cpuid is that we cannot use IA32_APIC_BASE
MSR to determine if the booting up cpu is BSP since rdmsr() to the MSR
could not work well even if it exists on some system, e.g. some
cluster system.
Signed-off-by: HATAYAMA Daisuke <d.hatayama@jp.fujitsu.com>
---
arch/x86/include/asm/mpspec.h | 1 +
arch/x86/kernel/acpi/boot.c | 8 ++++++++
arch/x86/kernel/apic/apic.c | 9 +++++++++
arch/x86/kernel/mpparse.c | 2 +-
arch/x86/mm/amdtopology.c | 6 +++---
arch/x86/platform/visws/visws_quirks.c | 2 +-
6 files changed, 23 insertions(+), 5 deletions(-)
diff --git a/arch/x86/include/asm/mpspec.h b/arch/x86/include/asm/mpspec.h
index 626cf70..3f9f94b 100644
--- a/arch/x86/include/asm/mpspec.h
+++ b/arch/x86/include/asm/mpspec.h
@@ -47,6 +47,7 @@ extern int mp_bus_id_to_type[MAX_MP_BUSSES];
extern DECLARE_BITMAP(mp_bus_not_pci, MAX_MP_BUSSES);
extern unsigned int boot_cpu_physical_apicid;
+extern unsigned int bios_bsp_physical_apicid;
extern unsigned int max_physical_apicid;
extern int mpc_default_type;
extern unsigned long mp_lapic_addr;
diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
index 40c7660..f62226a 100644
--- a/arch/x86/kernel/acpi/boot.c
+++ b/arch/x86/kernel/acpi/boot.c
@@ -206,6 +206,14 @@ static void acpi_register_lapic(int id, u8 enabled)
if (boot_cpu_physical_apicid != -1U)
ver = apic_version[boot_cpu_physical_apicid];
+ /*
+ * ACPI specification describes that platform firmware should
+ * list the BSP processor as the first LAPIC entry in the
+ * MADT.
+ */
+ if (!num_processors && !disabled_cpus)
+ bios_bsp_physical_apicid = id;
+
generic_processor_info(id, ver);
}
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index a7eb82d..cff8d71 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -63,6 +63,15 @@ unsigned disabled_cpus;
/* Processor that is doing the boot up */
unsigned int boot_cpu_physical_apicid = -1U;
+/* Processor with BP flag on IA32_APIC_BASE MSR to be initialized with
+ * the value reported by BIOS tables such as ACPI MADT or MP table,
+ * not with the value obtained by rdmsr(IA32_APIC_MSR) since it
+ * possibly doesn't work well on some clustered system.
+ *
+ * In case of kexec, this can differ from the processor that is doing
+ * the boot up. */
+unsigned int bios_bsp_physical_apicid = BAD_APICID;
+
/*
* The highest APIC ID seen during enumeration.
*/
diff --git a/arch/x86/kernel/mpparse.c b/arch/x86/kernel/mpparse.c
index d2b5648..5bb5220 100644
--- a/arch/x86/kernel/mpparse.c
+++ b/arch/x86/kernel/mpparse.c
@@ -64,7 +64,7 @@ static void __init MP_processor_info(struct mpc_cpu *m)
if (m->cpuflag & CPU_BOOTPROCESSOR) {
bootup_cpu = " (Bootup-CPU)";
- boot_cpu_physical_apicid = m->apicid;
+ bios_bsp_physical_apicid = m->apicid;
}
printk(KERN_INFO "Processor #%d%s\n", m->apicid, bootup_cpu);
diff --git a/arch/x86/mm/amdtopology.c b/arch/x86/mm/amdtopology.c
index 2ca15b5..788012e 100644
--- a/arch/x86/mm/amdtopology.c
+++ b/arch/x86/mm/amdtopology.c
@@ -183,9 +183,9 @@ int __init amd_numa_init(void)
/* get the APIC ID of the BSP early for systems with apicid lifting */
early_get_boot_cpu_id();
- if (boot_cpu_physical_apicid > 0) {
- pr_info("BSP APIC ID: %02x\n", boot_cpu_physical_apicid);
- apicid_base = boot_cpu_physical_apicid;
+ if (bios_bsp_physical_apicid > 0) {
+ pr_info("BSP APIC ID: %02x\n", bios_bsp_physical_apicid);
+ apicid_base = bios_bsp_physical_apicid;
}
for_each_node_mask(i, numa_nodes_parsed)
diff --git a/arch/x86/platform/visws/visws_quirks.c b/arch/x86/platform/visws/visws_quirks.c
index 94d8a39..5a91e50 100644
--- a/arch/x86/platform/visws/visws_quirks.c
+++ b/arch/x86/platform/visws/visws_quirks.c
@@ -166,7 +166,7 @@ static void __init MP_processor_info(struct mpc_cpu *m)
(m->cpufeature & CPU_MODEL_MASK) >> 4, m->apicver);
if (m->cpuflag & CPU_BOOTPROCESSOR)
- boot_cpu_physical_apicid = m->apicid;
+ bios_bsp_physical_apicid = m->apicid;
ver = m->apicver;
if ((ver >= 0x14 && m->apicid >= 0xff) || m->apicid >= 0xf) {
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH v6 2/3] x86, apic: Add disable_cpu_apicid kernel parameter
2013-11-21 2:00 [PATCH v6 0/3] x86, apic, kexec: Add disable_cpu_apic kernel parameter HATAYAMA Daisuke
2013-11-21 2:00 ` [PATCH v6 1/3] x86, apic: add bios_bsp_physical_apicid HATAYAMA Daisuke
@ 2013-11-21 2:00 ` HATAYAMA Daisuke
2013-11-21 21:33 ` Vivek Goyal
2013-11-21 2:00 ` [PATCH v6 3/3] Documentation, x86, apic, kexec: " HATAYAMA Daisuke
2 siblings, 1 reply; 7+ messages in thread
From: HATAYAMA Daisuke @ 2013-11-21 2:00 UTC (permalink / raw)
To: hpa, ebiederm, vgoyal
Cc: kexec, linux-kernel, bp, akpm, fengguang.wu, jingbai.ma
Add disable_cpu_apicid kernel parameter. To use this kernel parameter,
specify an initial APIC ID of the corresponding CPU you want to
disable.
This is mostly used for the kdump 2nd kernel to disable BSP to wake up
multiple CPUs without causing system reset or hang due to sending INIT
from AP to BSP.
Kdump users first figure out initial APIC ID of the BSP, CPU0 in the
1st kernel, for example from /proc/cpuinfo and then set up this kernel
parameter for the 2nd kernel using the obtained APIC ID.
However, doing this procedure at each boot time manually is awkward,
which should be automatically done by user-land service scripts, for
example, kexec-tools on fedora/RHEL distributions.
This design is more flexible than disabling BSP in kernel boot time
automatically in that in kernel boot time we have no choice but
referring to ACPI/MP table to obtain initial APIC ID for BSP, meaning
that the method is not applicable to the systems without such BIOS
tables.
One assumption behind this design is that users get initial APIC ID of
the BSP in still healthy state and so BSP is uniquely kept in
CPU0. Thus, through the kernel parameter, only one initial APIC ID can
be specified.
Signed-off-by: HATAYAMA Daisuke <d.hatayama@jp.fujitsu.com>
---
arch/x86/kernel/apic/apic.c | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index cff8d71..77fd68d 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -83,6 +83,13 @@ unsigned int max_physical_apicid;
physid_mask_t phys_cpu_present_map;
/*
+ * Processor to be disabled specified by kernel parameter
+ * disable_cpu_apicid=<int>, mostly used for the kdump 2nd kernel to
+ * avoid undefined behaviour caused by sending INIT from AP to BSP.
+ */
+unsigned int disabled_cpu_apicid = BAD_APICID;
+
+/*
* Map cpu index to physical APIC ID
*/
DEFINE_EARLY_PER_CPU_READ_MOSTLY(u16, x86_cpu_to_apicid, BAD_APICID);
@@ -2122,6 +2129,19 @@ void generic_processor_info(int apicid, int version)
bool boot_cpu_detected = physid_isset(boot_cpu_physical_apicid,
phys_cpu_present_map);
+ if (disabled_cpu_apicid != BAD_APICID &&
+ disabled_cpu_apicid != boot_cpu_physical_apicid &&
+ disabled_cpu_apicid == apicid) {
+ int thiscpu = num_processors + disabled_cpus;
+
+ pr_warning("ACPI: Disabling requested cpu."
+ " Processor %d/0x%x ignored.\n",
+ thiscpu, apicid);
+
+ disabled_cpus++;
+ return;
+ }
+
/*
* If boot cpu has not been detected yet, then only allow upto
* nr_cpu_ids - 1 processors and keep one slot free for boot cpu
@@ -2598,3 +2618,12 @@ static int __init lapic_insert_resource(void)
* that is using request_resource
*/
late_initcall(lapic_insert_resource);
+
+static int __init apic_set_disabled_cpu_apicid(char *arg)
+{
+ if (!arg || !get_option(&arg, &disabled_cpu_apicid))
+ return -EINVAL;
+
+ return 0;
+}
+early_param("disable_cpu_apicid", apic_set_disabled_cpu_apicid);
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [PATCH v6 2/3] x86, apic: Add disable_cpu_apicid kernel parameter
2013-11-21 2:00 ` [PATCH v6 2/3] x86, apic: Add disable_cpu_apicid kernel parameter HATAYAMA Daisuke
@ 2013-11-21 21:33 ` Vivek Goyal
2013-11-22 0:42 ` HATAYAMA Daisuke
0 siblings, 1 reply; 7+ messages in thread
From: Vivek Goyal @ 2013-11-21 21:33 UTC (permalink / raw)
To: HATAYAMA Daisuke
Cc: fengguang.wu, kexec, linux-kernel, bp, ebiederm, akpm, hpa,
jingbai.ma
On Thu, Nov 21, 2013 at 11:00:44AM +0900, HATAYAMA Daisuke wrote:
[..]
> @@ -2122,6 +2129,19 @@ void generic_processor_info(int apicid, int version)
> bool boot_cpu_detected = physid_isset(boot_cpu_physical_apicid,
> phys_cpu_present_map);
>
> + if (disabled_cpu_apicid != BAD_APICID &&
> + disabled_cpu_apicid != boot_cpu_physical_apicid &&
Hi Hatayama,
So we are comparing disabled_cpu_apicid with boot_cpu_physical_apicid
to make sure that one can not disable the cpu we are booting on. Can
we just read the apic id of booting cpu in local variable and compare
against that?
Something like as follows.
if (disabled_cpu_apicid != BAD_APICID &&
disabled_cpu_apicid == apicid &&
disabled_cpu_apicid != read_apic_id()) {
/* Disable cpu */
}
If above works, you will not need first patch in the series?
Thanks
Vivek
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v6 2/3] x86, apic: Add disable_cpu_apicid kernel parameter
2013-11-21 21:33 ` Vivek Goyal
@ 2013-11-22 0:42 ` HATAYAMA Daisuke
2013-11-22 1:59 ` Vivek Goyal
0 siblings, 1 reply; 7+ messages in thread
From: HATAYAMA Daisuke @ 2013-11-22 0:42 UTC (permalink / raw)
To: Vivek Goyal
Cc: fengguang.wu, kexec, linux-kernel, bp, ebiederm, akpm, hpa,
jingbai.ma
(2013/11/22 6:33), Vivek Goyal wrote:
> On Thu, Nov 21, 2013 at 11:00:44AM +0900, HATAYAMA Daisuke wrote:
>
> [..]
>> @@ -2122,6 +2129,19 @@ void generic_processor_info(int apicid, int version)
>> bool boot_cpu_detected = physid_isset(boot_cpu_physical_apicid,
>> phys_cpu_present_map);
>>
>> + if (disabled_cpu_apicid != BAD_APICID &&
>> + disabled_cpu_apicid != boot_cpu_physical_apicid &&
>
> Hi Hatayama,
>
> So we are comparing disabled_cpu_apicid with boot_cpu_physical_apicid
> to make sure that one can not disable the cpu we are booting on. Can
> we just read the apic id of booting cpu in local variable and compare
> against that?
>
> Something like as follows.
>
> if (disabled_cpu_apicid != BAD_APICID &&
> disabled_cpu_apicid == apicid &&
> disabled_cpu_apicid != read_apic_id()) {
> /* Disable cpu */
> }
>
> If above works, you will not need first patch in the series?
>
Yes, I came up with the idea, too. But doing this means we leave two different
ways boot_cpu_physical_apicid is used at boot, which seemed incomplete as a
patch. Also, then we could even lost the reason why boot_cpu_physical_apicid
exists.
But, it's true that the 1st patch causes one more reviewing point. I'll
remove it and fix the 2nd patch just as you suggested here.
--
Thanks.
HATAYAMA, Daisuke
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v6 2/3] x86, apic: Add disable_cpu_apicid kernel parameter
2013-11-22 0:42 ` HATAYAMA Daisuke
@ 2013-11-22 1:59 ` Vivek Goyal
0 siblings, 0 replies; 7+ messages in thread
From: Vivek Goyal @ 2013-11-22 1:59 UTC (permalink / raw)
To: HATAYAMA Daisuke
Cc: fengguang.wu, kexec, linux-kernel, bp, ebiederm, akpm, hpa,
jingbai.ma
On Fri, Nov 22, 2013 at 09:42:28AM +0900, HATAYAMA Daisuke wrote:
> (2013/11/22 6:33), Vivek Goyal wrote:
> >On Thu, Nov 21, 2013 at 11:00:44AM +0900, HATAYAMA Daisuke wrote:
> >
> >[..]
> >>@@ -2122,6 +2129,19 @@ void generic_processor_info(int apicid, int version)
> >> bool boot_cpu_detected = physid_isset(boot_cpu_physical_apicid,
> >> phys_cpu_present_map);
> >>
> >>+ if (disabled_cpu_apicid != BAD_APICID &&
> >>+ disabled_cpu_apicid != boot_cpu_physical_apicid &&
> >
> >Hi Hatayama,
> >
> >So we are comparing disabled_cpu_apicid with boot_cpu_physical_apicid
> >to make sure that one can not disable the cpu we are booting on. Can
> >we just read the apic id of booting cpu in local variable and compare
> >against that?
> >
> >Something like as follows.
> >
> > if (disabled_cpu_apicid != BAD_APICID &&
> > disabled_cpu_apicid == apicid &&
> > disabled_cpu_apicid != read_apic_id()) {
> > /* Disable cpu */
> > }
> >
> >If above works, you will not need first patch in the series?
> >
>
> Yes, I came up with the idea, too. But doing this means we leave two different
> ways boot_cpu_physical_apicid is used at boot, which seemed incomplete as a
> patch. Also, then we could even lost the reason why boot_cpu_physical_apicid
> exists.
>
> But, it's true that the 1st patch causes one more reviewing point. I'll
> remove it and fix the 2nd patch just as you suggested here.
I would say that do the boot_cpu_physical_id related cleanup in a separate
series. If we keep this patch really simple without fear of breaking
anything else, acking and committing the patch becomes easier.
Feel free to post boot cpu related cleanup in a separate series.
Thanks
Vivek
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v6 3/3] Documentation, x86, apic, kexec: Add disable_cpu_apicid kernel parameter
2013-11-21 2:00 [PATCH v6 0/3] x86, apic, kexec: Add disable_cpu_apic kernel parameter HATAYAMA Daisuke
2013-11-21 2:00 ` [PATCH v6 1/3] x86, apic: add bios_bsp_physical_apicid HATAYAMA Daisuke
2013-11-21 2:00 ` [PATCH v6 2/3] x86, apic: Add disable_cpu_apicid kernel parameter HATAYAMA Daisuke
@ 2013-11-21 2:00 ` HATAYAMA Daisuke
2 siblings, 0 replies; 7+ messages in thread
From: HATAYAMA Daisuke @ 2013-11-21 2:00 UTC (permalink / raw)
To: hpa, ebiederm, vgoyal
Cc: kexec, linux-kernel, bp, akpm, fengguang.wu, jingbai.ma
Add disable_cpu_apicid kernel parameter to disable the CPU with the
specified number of initial APIC ID, mostly used for the kdump 2nd
kernel to disable BSP to wake up multiple CPUs without causing system
reset or hang due to sending INIT from AP to BSP.
Signed-off-by: HATAYAMA Daisuke <d.hatayama@jp.fujitsu.com>
---
Documentation/kernel-parameters.txt | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index fcbb736..0ca0902 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -774,6 +774,15 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
disable= [IPV6]
See Documentation/networking/ipv6.txt.
+ disable_cpu_apicid= [X86,APIC,KEXEC,SMP]
+ Format: <int>
+ The number of initial APIC ID for the
+ corresponding CPU to be disabled at boot,
+ mostly used for the kdump 2nd kernel to
+ disable BSP to wake up multiple CPUs without
+ causing system reset or hang due to sending
+ INIT from AP to BSP.
+
disable_ddw [PPC/PSERIES]
Disable Dynamic DMA Window support. Use this if
to workaround buggy firmware.
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply related [flat|nested] 7+ messages in thread