[PATCH RFC 0/3] Support CPU hotplug for ARM64

[PATCH RFC 0/3] Support CPU hotplug for ARM64

[Xiongfeng Wang]

This patchset mark all the GICC node in MADT as possible CPUs even though it
is disabled. But only those enabled GICC node are marked as present CPUs.
So that kernel will initialize some CPU related data structure in advance before
the CPU is actually hot added into the system. This patchset also implement 
'acpi_(un)map_cpu()' and 'arch_(un)register_cpu()' for ARM64. These functions are
needed to enable CPU hotplug.

To support CPU hotplug, we need to add all the possible GICC node in MADT
including those CPUs that are not present but may be hot added later. Those
CPUs are marked as disabled in GICC nodes.

Xiongfeng Wang (3):
  ACPI / scan: evaluate _STA for processors declared via ASL Device
    statement
  arm64: mark all the GICC nodes in MADT as possible cpu
  arm64: Add CPU hotplug support

 arch/arm64/kernel/acpi.c  | 22 ++++++++++++++++++++++
 arch/arm64/kernel/setup.c | 19 ++++++++++++++++++-
 arch/arm64/kernel/smp.c   | 11 +++++------
 drivers/acpi/scan.c       | 12 ++++++++++++
 4 files changed, 57 insertions(+), 7 deletions(-)

-- 
1.7.12.4


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH RFC 1/3] ACPI / scan: evaluate _STA for processors declared via ASL Device statement

[Xiongfeng Wang]

When we scan all the acpi namespace node in
acpi_scan_init()->acpi_bus_scan(), we evaluate '_STA' method for processor
type node to determine whether the device is present. But processors can
also be declared via ASL Device statement. ACPI 6.3 spec specifically
says that the Processor statement is deprecated and a Device statement
should be used for processors. In that case, acpi_object_type is
ACPI_TYPE_DEVICE rather than ACPI_TYPE_PROCESSOR.

Current code doesn't evaluate '_STA' for nodes with ACPI_TYPE_DEVICE, and
the device status is set to 'present' as default. This patch get the
device status from '_STA' method for processors declared via ASL Device
statement if it does have a '_STA' method.

Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>

---
I am not sure if I should set 'type' as ACPI_BUS_TYPE_PROCESSOR rather than
ACPI_BUS_TYPE_DEVICE for processors declared via ASL Device statement.
---
 drivers/acpi/scan.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/acpi/scan.c b/drivers/acpi/scan.c
index e1b6231..ad50904 100644
--- a/drivers/acpi/scan.c
+++ b/drivers/acpi/scan.c
@@ -15,6 +15,7 @@
 #include <linux/dma-mapping.h>
 #include <linux/platform_data/x86/apple.h>
 
+#include <acpi/processor.h>
 #include <asm/pgtable.h>
 
 #include "internal.h"
@@ -1691,6 +1692,7 @@ static int acpi_bus_type_and_status(acpi_handle handle, int *type,
 {
 	acpi_status status;
 	acpi_object_type acpi_type;
+	struct acpi_device_info *info;
 
 	status = acpi_get_type(handle, &acpi_type);
 	if (ACPI_FAILURE(status))
@@ -1703,6 +1705,16 @@ static int acpi_bus_type_and_status(acpi_handle handle, int *type,
 			return -ENODEV;
 
 		*type = ACPI_BUS_TYPE_DEVICE;
+
+		status = acpi_get_object_info(handle, &info);
+		if (ACPI_SUCCESS(status) && info->valid & ACPI_VALID_HID &&
+		    !strcmp(info->hardware_id.string,
+					ACPI_PROCESSOR_DEVICE_HID)) {
+			status = acpi_bus_get_status_handle(handle, sta);
+			if (ACPI_SUCCESS(status))
+				break;
+		}
+
 		/*
 		 * acpi_add_single_object updates this once we've an acpi_device
 		 * so that acpi_bus_get_status' quirk handling can be used.
-- 
1.7.12.4


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH RFC 2/3] arm64: mark all the GICC nodes in MADT as possible cpu

[Xiongfeng Wang]

We set 'cpu_possible_mask' based on the enabled GICC node in MADT. If
the GICC node is disabled, we will skip initializing the kernel data
structure for that CPU.

To support CPU hotplug, we need to initialize some CPU related data
structure in advance. This patch mark all the GICC nodes as possible CPU
and only these enabled GICC nodes as present CPU.

Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
---
 arch/arm64/kernel/setup.c |  2 +-
 arch/arm64/kernel/smp.c   | 11 +++++------
 2 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index ec75d20..a82d0c2 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -391,7 +391,7 @@ static int __init topology_init(void)
 	for_each_online_node(i)
 		register_one_node(i);
 
-	for_each_possible_cpu(i) {
+	for_each_online_cpu(i) {
 		struct cpu *cpu = &per_cpu(cpu_data.cpu, i);
 		cpu->hotpluggable = 1;
 		register_cpu(cpu, i);
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 7aa9471..854d32c 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -534,16 +534,14 @@ struct acpi_madt_generic_interrupt *acpi_cpu_get_madt_gicc(int cpu)
 {
 	u64 hwid = processor->arm_mpidr;
 
-	if (!(processor->flags & ACPI_MADT_ENABLED)) {
-		pr_debug("skipping disabled CPU entry with 0x%llx MPIDR\n", hwid);
-		return;
-	}
-
 	if (hwid & ~MPIDR_HWID_BITMASK || hwid == INVALID_HWID) {
 		pr_err("skipping CPU entry with invalid MPIDR 0x%llx\n", hwid);
 		return;
 	}
 
+	if (!(processor->flags & ACPI_MADT_ENABLED))
+		pr_debug("disabled CPU entry with 0x%llx MPIDR\n", hwid);
+
 	if (is_mpidr_duplicate(cpu_count, hwid)) {
 		pr_err("duplicate CPU MPIDR 0x%llx in MADT\n", hwid);
 		return;
@@ -764,7 +762,8 @@ void __init smp_prepare_cpus(unsigned int max_cpus)
 		if (err)
 			continue;
 
-		set_cpu_present(cpu, true);
+		if ((cpu_madt_gicc[cpu].flags & ACPI_MADT_ENABLED))
+			set_cpu_present(cpu, true);
 		numa_store_cpu_info(cpu);
 	}
 }
-- 
1.7.12.4


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH RFC 3/3] arm64: Add CPU hotplug support

[Xiongfeng Wang]

To support CPU hotplug, we need to implement 'acpi_(un)map_cpu()' and
'arch_(un)register_cpu()' for ARM64. These functions are called in
'acpi_processor_hotadd_init()/acpi_processor_remove()' when the CPU is hot
added into or hot removed from the system.

Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
---
 arch/arm64/kernel/acpi.c  | 22 ++++++++++++++++++++++
 arch/arm64/kernel/setup.c | 17 +++++++++++++++++
 2 files changed, 39 insertions(+)

diff --git a/arch/arm64/kernel/acpi.c b/arch/arm64/kernel/acpi.c
index ed46dc1..0e7b4f5 100644
--- a/arch/arm64/kernel/acpi.c
+++ b/arch/arm64/kernel/acpi.c
@@ -28,6 +28,7 @@
 #include <linux/smp.h>
 #include <linux/serial_core.h>
 
+#include <acpi/processor.h>
 #include <asm/cputype.h>
 #include <asm/cpu_ops.h>
 #include <asm/pgtable.h>
@@ -257,3 +258,24 @@ pgprot_t __acpi_get_mem_attribute(phys_addr_t addr)
 		return __pgprot(PROT_NORMAL_NC);
 	return __pgprot(PROT_DEVICE_nGnRnE);
 }
+
+int acpi_map_cpu(acpi_handle handle, phys_cpuid_t physid, u32 acpi_id,
+		 int *pcpu)
+{
+	int cpu;
+
+	cpu = acpi_map_cpuid(physid, acpi_id);
+	*pcpu = cpu;
+	set_cpu_present(cpu, true);
+
+	return 0;
+}
+EXPORT_SYMBOL(acpi_map_cpu);
+
+int acpi_unmap_cpu(int cpu)
+{
+	set_cpu_present(cpu, false);
+
+	return 0;
+}
+EXPORT_SYMBOL(acpi_unmap_cpu);
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index a82d0c2..7b1a675 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -429,3 +429,20 @@ static int __init register_kernel_offset_dumper(void)
 	return 0;
 }
 __initcall(register_kernel_offset_dumper);
+
+int arch_register_cpu(int num)
+{
+	struct cpu *cpu = &per_cpu(cpu_data.cpu, num);
+
+	cpu->hotpluggable = 1;
+	return register_cpu(cpu, num);
+}
+EXPORT_SYMBOL(arch_register_cpu);
+
+void arch_unregister_cpu(int num)
+{
+	struct cpu *cpu = &per_cpu(cpu_data.cpu, num);
+
+	unregister_cpu(cpu);
+}
+EXPORT_SYMBOL(arch_unregister_cpu);
-- 
1.7.12.4


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/3] Support CPU hotplug for ARM64

[Xiongfeng Wang]

Sorry, the third patch can't be applied to the lastest kernel. I will send another
version and attach the method to test this patchset.

On 2019/6/28 19:13, Xiongfeng Wang wrote:
> This patchset mark all the GICC node in MADT as possible CPUs even though it
> is disabled. But only those enabled GICC node are marked as present CPUs.
> So that kernel will initialize some CPU related data structure in advance before
> the CPU is actually hot added into the system. This patchset also implement 
> 'acpi_(un)map_cpu()' and 'arch_(un)register_cpu()' for ARM64. These functions are
> needed to enable CPU hotplug.
> 
> To support CPU hotplug, we need to add all the possible GICC node in MADT
> including those CPUs that are not present but may be hot added later. Those
> CPUs are marked as disabled in GICC nodes.
> 
> Xiongfeng Wang (3):
>   ACPI / scan: evaluate _STA for processors declared via ASL Device
>     statement
>   arm64: mark all the GICC nodes in MADT as possible cpu
>   arm64: Add CPU hotplug support
> 
>  arch/arm64/kernel/acpi.c  | 22 ++++++++++++++++++++++
>  arch/arm64/kernel/setup.c | 19 ++++++++++++++++++-
>  arch/arm64/kernel/smp.c   | 11 +++++------
>  drivers/acpi/scan.c       | 12 ++++++++++++
>  4 files changed, 57 insertions(+), 7 deletions(-)
> 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/3] Support CPU hotplug for ARM64

[Jia He]

Hi Xiongfeng

It is a little bit awkful that I am also  investigating acpi based cpu hotplug 
issue silimar with

your idea. My question is your purpose to implement the vcpu hotplug in arm64 qemu?

Thanks for the ellaboration

---
Cheers,
Justin (Jia He)

On 2019/6/28 19:13, Xiongfeng Wang wrote:
> This patchset mark all the GICC node in MADT as possible CPUs even though it
> is disabled. But only those enabled GICC node are marked as present CPUs.
> So that kernel will initialize some CPU related data structure in advance before
> the CPU is actually hot added into the system. This patchset also implement
> 'acpi_(un)map_cpu()' and 'arch_(un)register_cpu()' for ARM64. These functions are
> needed to enable CPU hotplug.
>
> To support CPU hotplug, we need to add all the possible GICC node in MADT
> including those CPUs that are not present but may be hot added later. Those
> CPUs are marked as disabled in GICC nodes.
>
> Xiongfeng Wang (3):
>    ACPI / scan: evaluate _STA for processors declared via ASL Device
>      statement
>    arm64: mark all the GICC nodes in MADT as possible cpu
>    arm64: Add CPU hotplug support
>
>   arch/arm64/kernel/acpi.c  | 22 ++++++++++++++++++++++
>   arch/arm64/kernel/setup.c | 19 ++++++++++++++++++-
>   arch/arm64/kernel/smp.c   | 11 +++++------
>   drivers/acpi/scan.c       | 12 ++++++++++++
>   4 files changed, 57 insertions(+), 7 deletions(-)
>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/3] Support CPU hotplug for ARM64

[Xiongfeng Wang]

Hi Justin,

On 2019/7/4 11:00, Jia He wrote:
> Hi Xiongfeng
> 
> It is a little bit awkful that I am also  investigating acpi based cpu hotplug issue silimar with
> 
> your idea. My question is your purpose to implement the vcpu hotplug in arm64 qemu?

Yes, my purpose is to implement the vcpu hotplug in arm64 qemu. So that I can add or remove vcpu
without shutting down the Guest OS.

Thanks,
Xiongfeng

> 
> Thanks for the ellaboration
> 
> ---
> Cheers,
> Justin (Jia He)
> 
> On 2019/6/28 19:13, Xiongfeng Wang wrote:
>> This patchset mark all the GICC node in MADT as possible CPUs even though it
>> is disabled. But only those enabled GICC node are marked as present CPUs.
>> So that kernel will initialize some CPU related data structure in advance before
>> the CPU is actually hot added into the system. This patchset also implement
>> 'acpi_(un)map_cpu()' and 'arch_(un)register_cpu()' for ARM64. These functions are
>> needed to enable CPU hotplug.
>>
>> To support CPU hotplug, we need to add all the possible GICC node in MADT
>> including those CPUs that are not present but may be hot added later. Those
>> CPUs are marked as disabled in GICC nodes.
>>
>> Xiongfeng Wang (3):
>>    ACPI / scan: evaluate _STA for processors declared via ASL Device
>>      statement
>>    arm64: mark all the GICC nodes in MADT as possible cpu
>>    arm64: Add CPU hotplug support
>>
>>   arch/arm64/kernel/acpi.c  | 22 ++++++++++++++++++++++
>>   arch/arm64/kernel/setup.c | 19 ++++++++++++++++++-
>>   arch/arm64/kernel/smp.c   | 11 +++++------
>>   drivers/acpi/scan.c       | 12 ++++++++++++
>>   4 files changed, 57 insertions(+), 7 deletions(-)
>>
> 
> .
> 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/3] Support CPU hotplug for ARM64

[Jia He]

Hi Xiongfeng

On 2019/7/4 11:26, Xiongfeng Wang wrote:
> Hi Justin,
>
> On 2019/7/4 11:00, Jia He wrote:
>> Hi Xiongfeng
>>
>> It is a little bit awkful that I am also  investigating acpi based cpu hotplug issue silimar with
>>
>> your idea. My question is your purpose to implement the vcpu hotplug in arm64 qemu?
> Yes, my purpose is to implement the vcpu hotplug in arm64 qemu. So that I can add or remove vcpu
> without shutting down the Guest OS.

Thanks for the infor, I guess you used GED device in qemu ;-)?

---
Cheers,
Justin (Jia He)


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/3] Support CPU hotplug for ARM64

[Jia He]

Hi Xiongfeng

Sorry, I missed your latter mail, you used a emulated SCI interrupt

---
Cheers,
Justin (Jia He)

On 2019/7/4 11:26, Xiongfeng Wang wrote:
> Hi Justin,
>
> On 2019/7/4 11:00, Jia He wrote:
>> Hi Xiongfeng
>>
>> It is a little bit awkful that I am also  investigating acpi based cpu hotplug issue silimar with
>>
>> your idea. My question is your purpose to implement the vcpu hotplug in arm64 qemu?
> Yes, my purpose is to implement the vcpu hotplug in arm64 qemu. So that I can add or remove vcpu
> without shutting down the Guest OS.
>
> Thanks,
> Xiongfeng
>
>> Thanks for the ellaboration
>>
>> ---
>> Cheers,
>> Justin (Jia He)
>>
>> On 2019/6/28 19:13, Xiongfeng Wang wrote:
>>> This patchset mark all the GICC node in MADT as possible CPUs even though it
>>> is disabled. But only those enabled GICC node are marked as present CPUs.
>>> So that kernel will initialize some CPU related data structure in advance before
>>> the CPU is actually hot added into the system. This patchset also implement
>>> 'acpi_(un)map_cpu()' and 'arch_(un)register_cpu()' for ARM64. These functions are
>>> needed to enable CPU hotplug.
>>>
>>> To support CPU hotplug, we need to add all the possible GICC node in MADT
>>> including those CPUs that are not present but may be hot added later. Those
>>> CPUs are marked as disabled in GICC nodes.
>>>
>>> Xiongfeng Wang (3):
>>>     ACPI / scan: evaluate _STA for processors declared via ASL Device
>>>       statement
>>>     arm64: mark all the GICC nodes in MADT as possible cpu
>>>     arm64: Add CPU hotplug support
>>>
>>>    arch/arm64/kernel/acpi.c  | 22 ++++++++++++++++++++++
>>>    arch/arm64/kernel/setup.c | 19 ++++++++++++++++++-
>>>    arch/arm64/kernel/smp.c   | 11 +++++------
>>>    drivers/acpi/scan.c       | 12 ++++++++++++
>>>    4 files changed, 57 insertions(+), 7 deletions(-)
>>>
>> .
>>
-- 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH RFC 0/3] Support CPU hotplug for ARM64

[Xiongfeng Wang]

On 2019/7/4 14:04, Jia He wrote:
> Hi Xiongfeng
> 
> Sorry, I missed your latter mail, you used a emulated SCI interrupt

Yes, I only used a emulated SCI interrupt. My colleague is working on the qemu part.
He used the GED device in qemu. But there is still some other issues with the qemu
and he is working on it.

> 
> ---
> Cheers,
> Justin (Jia He)
> 
> On 2019/7/4 11:26, Xiongfeng Wang wrote:
>> Hi Justin,
>>
>> On 2019/7/4 11:00, Jia He wrote:
>>> Hi Xiongfeng
>>>
>>> It is a little bit awkful that I am also  investigating acpi based cpu hotplug issue silimar with
>>>
>>> your idea. My question is your purpose to implement the vcpu hotplug in arm64 qemu?
>> Yes, my purpose is to implement the vcpu hotplug in arm64 qemu. So that I can add or remove vcpu
>> without shutting down the Guest OS.
>>
>> Thanks,
>> Xiongfeng
>>
>>> Thanks for the ellaboration
>>>
>>> ---
>>> Cheers,
>>> Justin (Jia He)
>>>
>>> On 2019/6/28 19:13, Xiongfeng Wang wrote:
>>>> This patchset mark all the GICC node in MADT as possible CPUs even though it
>>>> is disabled. But only those enabled GICC node are marked as present CPUs.
>>>> So that kernel will initialize some CPU related data structure in advance before
>>>> the CPU is actually hot added into the system. This patchset also implement
>>>> 'acpi_(un)map_cpu()' and 'arch_(un)register_cpu()' for ARM64. These functions are
>>>> needed to enable CPU hotplug.
>>>>
>>>> To support CPU hotplug, we need to add all the possible GICC node in MADT
>>>> including those CPUs that are not present but may be hot added later. Those
>>>> CPUs are marked as disabled in GICC nodes.
>>>>
>>>> Xiongfeng Wang (3):
>>>>     ACPI / scan: evaluate _STA for processors declared via ASL Device
>>>>       statement
>>>>     arm64: mark all the GICC nodes in MADT as possible cpu
>>>>     arm64: Add CPU hotplug support
>>>>
>>>>    arch/arm64/kernel/acpi.c  | 22 ++++++++++++++++++++++
>>>>    arch/arm64/kernel/setup.c | 19 ++++++++++++++++++-
>>>>    arch/arm64/kernel/smp.c   | 11 +++++------
>>>>    drivers/acpi/scan.c       | 12 ++++++++++++
>>>>    4 files changed, 57 insertions(+), 7 deletions(-)
>>>>
>>> .
>>>


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel