* [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
@ 2024-07-13 18:25 Salil Mehta via
2024-07-13 18:25 ` [PATCH V15 1/7] accel/kvm: Extract common KVM vCPU {creation, parking} code Salil Mehta via
` (11 more replies)
0 siblings, 12 replies; 35+ messages in thread
From: Salil Mehta via @ 2024-07-13 18:25 UTC (permalink / raw)
To: qemu-devel, qemu-arm
Cc: salil.mehta, maz, jean-philippe, jonathan.cameron, lpieralisi,
peter.maydell, richard.henderson, imammedo, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm
[Note: References are present at the last after the revision history]
Virtual CPU hotplug support is being added across various architectures [1][3].
This series adds various code bits common across all architectures:
1. vCPU creation and Parking code refactor [Patch 1]
2. Update ACPI GED framework to support vCPU Hotplug [Patch 2,3]
3. ACPI CPUs AML code change [Patch 4,5]
4. Helper functions to support unrealization of CPU objects [Patch 6,7]
Repository:
[*] Architecture *Agnostic* Patch-set (This series)
V14: https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v3.arch.agnostic.v15
NOTE: This series is meant to work in conjunction with the architecture-specific
patch-set. For ARM, a combined patch-set (architecture agnostic + specific) was
earlier pushed as RFC V2 [1]. Later, RFC V2 was split into the ARM Architecture
specific patch-set RFC V3 [4] (a subset of RFC V2) and the architecture agnostic
patch-set. Patch-set V14 is the latest version in that series. This series
works in conjunction with RFC V4-rc2, present at the following link.
[*] ARM Architecture *Specific* Patch-set
RFC V3 [4]: https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v3
RFC V4-rc2: https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v4-rc2 (combined)
Revision History:
Patch-set V14 -> V15
1. Addressed commnet from Igor Mammedov's on [PATCH V14 4/7]
- Removed ACPI_CPU_SCAN_METHOD
- Introduced AML_GED_EVT_CPU_SCAN_METHOD ("\\_SB.GED.CPSCN") macro
2. Fix the stray change of "assert (" in "PATCH V14 3/7"
Link: https://lore.kernel.org/qemu-devel/20240712134201.214699-4-salil.mehta@huawei.com/
Patch-set V13 -> V14
1. Addressed Igor Mammedov's following review comments
- Mentioned abput new external APIs in the header note of [PATCH 1/7]
- Merged Doc [PATCH V13 8/8] with [PATCH V14 3/7]
- Introduced GED realize function for various CPU Hotplug regions initializations
- Added back event handler method to indirectly expose \\_SB.CPUS.CSCN to GED
_EVT. Like for ARM, it would be through \\_SB.GED.CSCN event handler method
- Collected the Ack given for [Patch V13 6/8]
- Added back the gfree'ing of GDB regs in common finalize and made it conditional
- Updated the header notes of [PATCH V13 3/8,4/8,5/8] to reflect the changes
Patch-set V12 -> V13
1. Added Reviewed-by Tag of Harsh Prateek Bora's (IBM) [PATCH V12 1/8]
2. Moved the kvm_{create,park,unpark}_vcpu prototypes from accel/kvm/kvm-cpus.h
to include/sysemu/kvm.h. These can later be exported through AccelOps.
Link: https://lore.kernel.org/qemu-devel/62f55169-1796-4d8e-a35d-7f003a172750@linux.ibm.com/
Patch-set V11 -> V12
1. Addressed Harsh Prateek Bora's (IBM) comment
- Changed @cpu to @vcpu_id in the kvm_unpark_vcpu protoype header/
2. Added Zhao Liu's (Intel) Tested-by for whole series
- Qtest does not breaks on Intel platforms now.
3. Added Zhao Liu's (Intel) Reviewed-by for [PATCH V11 {1/8 - 3/8}]
Link: https://lore.kernel.org/qemu-devel/ZlRSPuJGBgyEUW6w@intel.com/
Link: https://lore.kernel.org/qemu-devel/a5f3d78e-cfed-441f-9c56-e3e78fa5edee@linux.ibm.com/
Patch-set V10 -> V11
1. Addressed Nicholas Piggin's (IBM) comment
- moved the traces in kvm_unpark_vcpu and kvm_create_vcpu at the end
- Added the Reviewed-by Tag for [PATCH V10 1/8]
2. Addressed Alex Bennée's (Linaro) comments
- Added a note explaining dependency of the [PATCH V10 7/8] on Arch specific patch-set
Link: https://lore.kernel.org/qemu-devel/D1FS5GOOFWWK.2PNRIVL0V6DBL@gmail.com/
Link: https://lore.kernel.org/qemu-devel/87frubi402.fsf@draig.linaro.org/
Patch-set V9 -> V10
1. Addressed Nicholas Piggin's (IBM) & Philippe Mathieu-Daudé (Linaro) comments
- carved out kvm_unpark_vcpu and added its trace
- Widened the scope of the kvm_unpark_vcpu so that it can be used by generic framework
being thought out
Link: https://lore.kernel.org/qemu-devel/20240519210620.228342-1-salil.mehta@huawei.com/
Link: https://lore.kernel.org/qemu-devel/e94b0e14-efee-4050-9c9f-08382a36b63a@linaro.org/
Patch-set V8 -> V9
1. Addressed Vishnu Pajjuri's (Ampere) comments
- Added kvm_fd to trace in kvm_create_vcpu
- Some clean ups: arch vcpu-id and sbd variable
- Added the missed initialization of cpu->gdb_num_regs
2. Addressed the commnet from Zhao Liu (Intel)
- Make initialization of CPU Hotplug state conditional (possible_cpu_arch_ids!=NULL)
Link: https://lore.kernel.org/qemu-devel/20240312020000.12992-1-salil.mehta@huawei.com/
Patch-set V7 -> V8
1. Rebased and Fixed the conflicts
Patch-set V6 -> V7
1. Addressed Alex Bennée's comments
- Updated the docs
2. Addressed Igor Mammedov's comments
- Merged patches [Patch V6 3/9] & [Patch V6 7/9] with [Patch V6 4/9]
- Updated commit-log of [Patch V6 1/9] and [Patch V6 5/9]
3. Added Shaoqin Huang's Reviewed-by tags for whole series.
Link: https://lore.kernel.org/qemu-devel/20231013105129.25648-1-salil.mehta@huawei.com/
Patch-set V5 -> V6
1. Addressed Gavin Shan's comments
- Fixed the assert() ranges of address spaces
- Rebased the patch-set to latest changes in the qemu.git
- Added Reviewed-by tags for patches {8,9}
2. Addressed Jonathan Cameron's comments
- Updated commit-log for [Patch V5 1/9] with mention of trace events
- Added Reviewed-by tags for patches {1,5}
3. Added Tested-by tags from Xianglai Li
4. Fixed checkpatch.pl error "Qemu -> QEMU" in [Patch V5 1/9]
Link: https://lore.kernel.org/qemu-devel/20231011194355.15628-1-salil.mehta@huawei.com/
Patch-set V4 -> V5
1. Addressed Gavin Shan's comments
- Fixed the trace events print string for kvm_{create,get,park,destroy}_vcpu
- Added Reviewed-by tag for patch {1}
2. Added Shaoqin Huang's Reviewed-by tags for Patches {2,3}
3. Added Tested-by Tag from Vishnu Pajjuri to the patch-set
4. Dropped the ARM specific [Patch V4 10/10]
Link: https://lore.kernel.org/qemu-devel/20231009203601.17584-1-salil.mehta@huawei.com/
Patch-set V3 -> V4
1. Addressed David Hilderbrand's comments
- Fixed the wrong doc comment of kvm_park_vcpu API prototype
- Added Reviewed-by tags for patches {2,4}
Link: https://lore.kernel.org/qemu-devel/20231009112812.10612-1-salil.mehta@huawei.com/
Patch-set V2 -> V3
1. Addressed Jonathan Cameron's comments
- Fixed 'vcpu-id' type wrongly changed from 'unsigned long' to 'integer'
- Removed unnecessary use of variable 'vcpu_id' in kvm_park_vcpu
- Updated [Patch V2 3/10] commit-log with details of ACPI_CPU_SCAN_METHOD macro
- Updated [Patch V2 5/10] commit-log with details of conditional event handler method
- Added Reviewed-by tags for patches {2,3,4,6,7}
2. Addressed Gavin Shan's comments
- Remove unnecessary use of variable 'vcpu_id' in kvm_par_vcpu
- Fixed return value in kvm_get_vcpu from -1 to -ENOENT
- Reset the value of 'gdb_num_g_regs' in gdb_unregister_coprocessor_all
- Fixed the kvm_{create,park}_vcpu prototypes docs
- Added Reviewed-by tags for patches {2,3,4,5,6,7,9,10}
3. Addressed one earlier missed comment by Alex Bennée in RFC V1
- Added traces instead of DPRINTF in the newly added and some existing functions
Link: https://lore.kernel.org/qemu-devel/20230930001933.2660-1-salil.mehta@huawei.com/
Patch-set V1 -> V2
1. Addressed Alex Bennée's comments
- Refactored the kvm_create_vcpu logic to get rid of goto
- Added the docs for kvm_{create,park}_vcpu prototypes
- Splitted the gdbstub and AddressSpace destruction change into separate patches
- Added Reviewed-by tags for patches {2,10}
Link: https://lore.kernel.org/qemu-devel/20230929124304.13672-1-salil.mehta@huawei.com/
References:
[1] https://lore.kernel.org/qemu-devel/20230926100436.28284-1-salil.mehta@huawei.com/
[2] https://lore.kernel.org/all/20230913163823.7880-1-james.morse@arm.com/
[3] https://lore.kernel.org/qemu-devel/cover.1695697701.git.lixianglai@loongson.cn/
[4] https://lore.kernel.org/qemu-devel/20240613233639.202896-2-salil.mehta@huawei.com/
Salil Mehta (7):
accel/kvm: Extract common KVM vCPU {creation,parking} code
hw/acpi: Move CPU ctrl-dev MMIO region len macro to common header file
hw/acpi: Update ACPI GED framework to support vCPU Hotplug
hw/acpi: Update GED _EVT method AML with CPU scan
hw/acpi: Update CPUs AML with cpu-(ctrl)dev change
physmem: Add helper function to destroy CPU AddressSpace
gdbstub: Add helper function to unregister GDB register space
accel/kvm/kvm-all.c | 95 +++++++++++++++++---------
accel/kvm/kvm-cpus.h | 1 -
accel/kvm/trace-events | 5 +-
docs/specs/acpi_hw_reduced_hotplug.rst | 3 +-
gdbstub/gdbstub.c | 13 ++++
hw/acpi/acpi-cpu-hotplug-stub.c | 6 ++
hw/acpi/cpu.c | 18 +++--
hw/acpi/generic_event_device.c | 27 ++++++++
hw/core/cpu-common.c | 4 +-
hw/i386/acpi-build.c | 3 +-
include/exec/cpu-common.h | 8 +++
include/exec/gdbstub.h | 6 ++
include/hw/acpi/cpu.h | 7 +-
include/hw/acpi/generic_event_device.h | 5 ++
include/hw/core/cpu.h | 1 +
include/sysemu/kvm.h | 25 +++++++
system/physmem.c | 29 ++++++++
17 files changed, 212 insertions(+), 44 deletions(-)
--
2.34.1
^ permalink raw reply [flat|nested] 35+ messages in thread
* [PATCH V15 1/7] accel/kvm: Extract common KVM vCPU {creation, parking} code
2024-07-13 18:25 [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Salil Mehta via
@ 2024-07-13 18:25 ` Salil Mehta via
2024-07-15 12:49 ` [PATCH V15 1/7] accel/kvm: Extract common KVM vCPU {creation,parking} code Igor Mammedov
2024-07-13 18:25 ` [PATCH V15 2/7] hw/acpi: Move CPU ctrl-dev MMIO region len macro to common header file Salil Mehta via
` (10 subsequent siblings)
11 siblings, 1 reply; 35+ messages in thread
From: Salil Mehta via @ 2024-07-13 18:25 UTC (permalink / raw)
To: qemu-devel, qemu-arm
Cc: salil.mehta, maz, jean-philippe, jonathan.cameron, lpieralisi,
peter.maydell, richard.henderson, imammedo, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm,
Jonathan Cameron, Shaoqin Huang, Zhao Liu
KVM vCPU creation is done once during the vCPU realization when Qemu vCPU thread
is spawned. This is common to all the architectures as of now.
Hot-unplug of vCPU results in destruction of the vCPU object in QOM but the
corresponding KVM vCPU object in the Host KVM is not destroyed as KVM doesn't
support vCPU removal. Therefore, its representative KVM vCPU object/context in
Qemu is parked.
Refactor architecture common logic so that some APIs could be reused by vCPU
Hotplug code of some architectures likes ARM, Loongson etc. Update new/old APIs
with trace events. New APIs qemu_{create,park,unpark}_vcpu() can be externally
called. No functional change is intended here.
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Tested-by: Xianglai Li <lixianglai@loongson.cn>
Tested-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
Reviewed-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
Tested-by: Zhao Liu <zhao1.liu@intel.com>
Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
Reviewed-by: Harsh Prateek Bora <harshpb@linux.ibm.com>
---
accel/kvm/kvm-all.c | 95 ++++++++++++++++++++++++++++--------------
accel/kvm/kvm-cpus.h | 1 -
accel/kvm/trace-events | 5 ++-
include/sysemu/kvm.h | 25 +++++++++++
4 files changed, 92 insertions(+), 34 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 2b4ab89679..e446d18944 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -340,14 +340,71 @@ err:
return ret;
}
+void kvm_park_vcpu(CPUState *cpu)
+{
+ struct KVMParkedVcpu *vcpu;
+
+ trace_kvm_park_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu));
+
+ vcpu = g_malloc0(sizeof(*vcpu));
+ vcpu->vcpu_id = kvm_arch_vcpu_id(cpu);
+ vcpu->kvm_fd = cpu->kvm_fd;
+ QLIST_INSERT_HEAD(&kvm_state->kvm_parked_vcpus, vcpu, node);
+}
+
+int kvm_unpark_vcpu(KVMState *s, unsigned long vcpu_id)
+{
+ struct KVMParkedVcpu *cpu;
+ int kvm_fd = -ENOENT;
+
+ QLIST_FOREACH(cpu, &s->kvm_parked_vcpus, node) {
+ if (cpu->vcpu_id == vcpu_id) {
+ QLIST_REMOVE(cpu, node);
+ kvm_fd = cpu->kvm_fd;
+ g_free(cpu);
+ }
+ }
+
+ trace_kvm_unpark_vcpu(vcpu_id, kvm_fd > 0 ? "unparked" : "not found parked");
+
+ return kvm_fd;
+}
+
+int kvm_create_vcpu(CPUState *cpu)
+{
+ unsigned long vcpu_id = kvm_arch_vcpu_id(cpu);
+ KVMState *s = kvm_state;
+ int kvm_fd;
+
+ /* check if the KVM vCPU already exist but is parked */
+ kvm_fd = kvm_unpark_vcpu(s, vcpu_id);
+ if (kvm_fd < 0) {
+ /* vCPU not parked: create a new KVM vCPU */
+ kvm_fd = kvm_vm_ioctl(s, KVM_CREATE_VCPU, vcpu_id);
+ if (kvm_fd < 0) {
+ error_report("KVM_CREATE_VCPU IOCTL failed for vCPU %lu", vcpu_id);
+ return kvm_fd;
+ }
+ }
+
+ cpu->kvm_fd = kvm_fd;
+ cpu->kvm_state = s;
+ cpu->vcpu_dirty = true;
+ cpu->dirty_pages = 0;
+ cpu->throttle_us_per_full = 0;
+
+ trace_kvm_create_vcpu(cpu->cpu_index, vcpu_id, kvm_fd);
+
+ return 0;
+}
+
static int do_kvm_destroy_vcpu(CPUState *cpu)
{
KVMState *s = kvm_state;
long mmap_size;
- struct KVMParkedVcpu *vcpu = NULL;
int ret = 0;
- trace_kvm_destroy_vcpu();
+ trace_kvm_destroy_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu));
ret = kvm_arch_destroy_vcpu(cpu);
if (ret < 0) {
@@ -373,10 +430,7 @@ static int do_kvm_destroy_vcpu(CPUState *cpu)
}
}
- vcpu = g_malloc0(sizeof(*vcpu));
- vcpu->vcpu_id = kvm_arch_vcpu_id(cpu);
- vcpu->kvm_fd = cpu->kvm_fd;
- QLIST_INSERT_HEAD(&kvm_state->kvm_parked_vcpus, vcpu, node);
+ kvm_park_vcpu(cpu);
err:
return ret;
}
@@ -389,24 +443,6 @@ void kvm_destroy_vcpu(CPUState *cpu)
}
}
-static int kvm_get_vcpu(KVMState *s, unsigned long vcpu_id)
-{
- struct KVMParkedVcpu *cpu;
-
- QLIST_FOREACH(cpu, &s->kvm_parked_vcpus, node) {
- if (cpu->vcpu_id == vcpu_id) {
- int kvm_fd;
-
- QLIST_REMOVE(cpu, node);
- kvm_fd = cpu->kvm_fd;
- g_free(cpu);
- return kvm_fd;
- }
- }
-
- return kvm_vm_ioctl(s, KVM_CREATE_VCPU, (void *)vcpu_id);
-}
-
int kvm_init_vcpu(CPUState *cpu, Error **errp)
{
KVMState *s = kvm_state;
@@ -415,19 +451,14 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
trace_kvm_init_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu));
- ret = kvm_get_vcpu(s, kvm_arch_vcpu_id(cpu));
+ ret = kvm_create_vcpu(cpu);
if (ret < 0) {
- error_setg_errno(errp, -ret, "kvm_init_vcpu: kvm_get_vcpu failed (%lu)",
+ error_setg_errno(errp, -ret,
+ "kvm_init_vcpu: kvm_create_vcpu failed (%lu)",
kvm_arch_vcpu_id(cpu));
goto err;
}
- cpu->kvm_fd = ret;
- cpu->kvm_state = s;
- cpu->vcpu_dirty = true;
- cpu->dirty_pages = 0;
- cpu->throttle_us_per_full = 0;
-
mmap_size = kvm_ioctl(s, KVM_GET_VCPU_MMAP_SIZE, 0);
if (mmap_size < 0) {
ret = mmap_size;
diff --git a/accel/kvm/kvm-cpus.h b/accel/kvm/kvm-cpus.h
index ca40add32c..171b22fd29 100644
--- a/accel/kvm/kvm-cpus.h
+++ b/accel/kvm/kvm-cpus.h
@@ -22,5 +22,4 @@ bool kvm_supports_guest_debug(void);
int kvm_insert_breakpoint(CPUState *cpu, int type, vaddr addr, vaddr len);
int kvm_remove_breakpoint(CPUState *cpu, int type, vaddr addr, vaddr len);
void kvm_remove_all_breakpoints(CPUState *cpu);
-
#endif /* KVM_CPUS_H */
diff --git a/accel/kvm/trace-events b/accel/kvm/trace-events
index 681ccb667d..37626c1ac5 100644
--- a/accel/kvm/trace-events
+++ b/accel/kvm/trace-events
@@ -9,6 +9,10 @@ kvm_device_ioctl(int fd, int type, void *arg) "dev fd %d, type 0x%x, arg %p"
kvm_failed_reg_get(uint64_t id, const char *msg) "Warning: Unable to retrieve ONEREG %" PRIu64 " from KVM: %s"
kvm_failed_reg_set(uint64_t id, const char *msg) "Warning: Unable to set ONEREG %" PRIu64 " to KVM: %s"
kvm_init_vcpu(int cpu_index, unsigned long arch_cpu_id) "index: %d id: %lu"
+kvm_create_vcpu(int cpu_index, unsigned long arch_cpu_id, int kvm_fd) "index: %d, id: %lu, kvm fd: %d"
+kvm_destroy_vcpu(int cpu_index, unsigned long arch_cpu_id) "index: %d id: %lu"
+kvm_park_vcpu(int cpu_index, unsigned long arch_cpu_id) "index: %d id: %lu"
+kvm_unpark_vcpu(unsigned long arch_cpu_id, const char *msg) "id: %lu %s"
kvm_irqchip_commit_routes(void) ""
kvm_irqchip_add_msi_route(char *name, int vector, int virq) "dev %s vector %d virq %d"
kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d"
@@ -25,7 +29,6 @@ kvm_dirty_ring_reaper(const char *s) "%s"
kvm_dirty_ring_reap(uint64_t count, int64_t t) "reaped %"PRIu64" pages (took %"PRIi64" us)"
kvm_dirty_ring_reaper_kick(const char *reason) "%s"
kvm_dirty_ring_flush(int finished) "%d"
-kvm_destroy_vcpu(void) ""
kvm_failed_get_vcpu_mmap_size(void) ""
kvm_cpu_exec(void) ""
kvm_interrupt_exit_request(void) ""
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index c31d9c7356..c4a914b3d8 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -313,6 +313,31 @@ int kvm_create_device(KVMState *s, uint64_t type, bool test);
*/
bool kvm_device_supported(int vmfd, uint64_t type);
+/**
+ * kvm_create_vcpu - Gets a parked KVM vCPU or creates a KVM vCPU
+ * @cpu: QOM CPUState object for which KVM vCPU has to be fetched/created.
+ *
+ * @returns: 0 when success, errno (<0) when failed.
+ */
+int kvm_create_vcpu(CPUState *cpu);
+
+/**
+ * kvm_park_vcpu - Park QEMU KVM vCPU context
+ * @cpu: QOM CPUState object for which QEMU KVM vCPU context has to be parked.
+ *
+ * @returns: none
+ */
+void kvm_park_vcpu(CPUState *cpu);
+
+/**
+ * kvm_unpark_vcpu - unpark QEMU KVM vCPU context
+ * @s: KVM State
+ * @vcpu_id: Architecture vCPU ID of the parked vCPU
+ *
+ * @returns: KVM fd
+ */
+int kvm_unpark_vcpu(KVMState *s, unsigned long vcpu_id);
+
/* Arch specific hooks */
extern const KVMCapabilityInfo kvm_arch_required_capabilities[];
--
2.34.1
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH V15 2/7] hw/acpi: Move CPU ctrl-dev MMIO region len macro to common header file
2024-07-13 18:25 [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Salil Mehta via
2024-07-13 18:25 ` [PATCH V15 1/7] accel/kvm: Extract common KVM vCPU {creation, parking} code Salil Mehta via
@ 2024-07-13 18:25 ` Salil Mehta via
2024-07-15 12:50 ` Igor Mammedov
2024-07-13 18:25 ` [PATCH V15 3/7] hw/acpi: Update ACPI GED framework to support vCPU Hotplug Salil Mehta via
` (9 subsequent siblings)
11 siblings, 1 reply; 35+ messages in thread
From: Salil Mehta via @ 2024-07-13 18:25 UTC (permalink / raw)
To: qemu-devel, qemu-arm
Cc: salil.mehta, maz, jean-philippe, jonathan.cameron, lpieralisi,
peter.maydell, richard.henderson, imammedo, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm,
Jonathan Cameron, Shaoqin Huang, Zhao Liu
CPU ctrl-dev MMIO region length could be used in ACPI GED and various other
architecture specific places. Move ACPI_CPU_HOTPLUG_REG_LEN macro to more
appropriate common header file.
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
Tested-by: Xianglai Li <lixianglai@loongson.cn>
Tested-by: Miguel Luis <miguel.luis@oracle.com>
Tested-by: Zhao Liu <zhao1.liu@intel.com>
Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
---
hw/acpi/cpu.c | 1 -
include/hw/acpi/cpu.h | 2 ++
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/hw/acpi/cpu.c b/hw/acpi/cpu.c
index 2d81c1e790..cf5e9183e4 100644
--- a/hw/acpi/cpu.c
+++ b/hw/acpi/cpu.c
@@ -7,7 +7,6 @@
#include "trace.h"
#include "sysemu/numa.h"
-#define ACPI_CPU_HOTPLUG_REG_LEN 12
#define ACPI_CPU_SELECTOR_OFFSET_WR 0
#define ACPI_CPU_FLAGS_OFFSET_RW 4
#define ACPI_CPU_CMD_OFFSET_WR 5
diff --git a/include/hw/acpi/cpu.h b/include/hw/acpi/cpu.h
index e6e1a9ef59..df87b15997 100644
--- a/include/hw/acpi/cpu.h
+++ b/include/hw/acpi/cpu.h
@@ -19,6 +19,8 @@
#include "hw/boards.h"
#include "hw/hotplug.h"
+#define ACPI_CPU_HOTPLUG_REG_LEN 12
+
typedef struct AcpiCpuStatus {
CPUState *cpu;
uint64_t arch_id;
--
2.34.1
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH V15 3/7] hw/acpi: Update ACPI GED framework to support vCPU Hotplug
2024-07-13 18:25 [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Salil Mehta via
2024-07-13 18:25 ` [PATCH V15 1/7] accel/kvm: Extract common KVM vCPU {creation, parking} code Salil Mehta via
2024-07-13 18:25 ` [PATCH V15 2/7] hw/acpi: Move CPU ctrl-dev MMIO region len macro to common header file Salil Mehta via
@ 2024-07-13 18:25 ` Salil Mehta via
2024-07-15 13:12 ` Igor Mammedov
2024-07-13 18:25 ` [PATCH V15 4/7] hw/acpi: Update GED _EVT method AML with CPU scan Salil Mehta via
` (8 subsequent siblings)
11 siblings, 1 reply; 35+ messages in thread
From: Salil Mehta via @ 2024-07-13 18:25 UTC (permalink / raw)
To: qemu-devel, qemu-arm
Cc: salil.mehta, maz, jean-philippe, jonathan.cameron, lpieralisi,
peter.maydell, richard.henderson, imammedo, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm,
Jonathan Cameron, Shaoqin Huang, Zhao Liu
ACPI GED (as described in the ACPI 6.4 spec) uses an interrupt listed in the
_CRS object of GED to intimate OSPM about an event. Later then demultiplexes the
notified event by evaluating ACPI _EVT method to know the type of event. Use
ACPI GED to also notify the guest kernel about any CPU hot(un)plug events.
Note, GED interface is used by many hotplug events like memory hotplug, NVDIMM
hotplug and non-hotplug events like system power down event. Each of these can
be selected using a bit in the 32 bit GED IO interface. A bit has been reserved
for the CPU hotplug event.
ACPI CPU hotplug related initialization should only happen if ACPI_CPU_HOTPLUG
support has been enabled for particular architecture. Add cpu_hotplug_hw_init()
stub to avoid compilation break.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
Tested-by: Xianglai Li <lixianglai@loongson.cn>
Tested-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
Tested-by: Zhao Liu <zhao1.liu@intel.com>
Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
---
docs/specs/acpi_hw_reduced_hotplug.rst | 3 ++-
hw/acpi/acpi-cpu-hotplug-stub.c | 6 ++++++
hw/acpi/generic_event_device.c | 24 ++++++++++++++++++++++++
include/hw/acpi/generic_event_device.h | 4 ++++
4 files changed, 36 insertions(+), 1 deletion(-)
diff --git a/docs/specs/acpi_hw_reduced_hotplug.rst b/docs/specs/acpi_hw_reduced_hotplug.rst
index 0bd3f9399f..3acd6fcd8b 100644
--- a/docs/specs/acpi_hw_reduced_hotplug.rst
+++ b/docs/specs/acpi_hw_reduced_hotplug.rst
@@ -64,7 +64,8 @@ GED IO interface (4 byte access)
0: Memory hotplug event
1: System power down event
2: NVDIMM hotplug event
- 3-31: Reserved
+ 3: CPU hotplug event
+ 4-31: Reserved
**write_access:**
diff --git a/hw/acpi/acpi-cpu-hotplug-stub.c b/hw/acpi/acpi-cpu-hotplug-stub.c
index 3fc4b14c26..c6c61bb9cd 100644
--- a/hw/acpi/acpi-cpu-hotplug-stub.c
+++ b/hw/acpi/acpi-cpu-hotplug-stub.c
@@ -19,6 +19,12 @@ void legacy_acpi_cpu_hotplug_init(MemoryRegion *parent, Object *owner,
return;
}
+void cpu_hotplug_hw_init(MemoryRegion *as, Object *owner,
+ CPUHotplugState *state, hwaddr base_addr)
+{
+ return;
+}
+
void acpi_cpu_ospm_status(CPUHotplugState *cpu_st, ACPIOSTInfoList ***list)
{
return;
diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c
index 2d6e91b124..1b31d633ba 100644
--- a/hw/acpi/generic_event_device.c
+++ b/hw/acpi/generic_event_device.c
@@ -25,6 +25,7 @@ static const uint32_t ged_supported_events[] = {
ACPI_GED_MEM_HOTPLUG_EVT,
ACPI_GED_PWR_DOWN_EVT,
ACPI_GED_NVDIMM_HOTPLUG_EVT,
+ ACPI_GED_CPU_HOTPLUG_EVT,
};
/*
@@ -234,6 +235,8 @@ static void acpi_ged_device_plug_cb(HotplugHandler *hotplug_dev,
} else {
acpi_memory_plug_cb(hotplug_dev, &s->memhp_state, dev, errp);
}
+ } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+ acpi_cpu_plug_cb(hotplug_dev, &s->cpuhp_state, dev, errp);
} else {
error_setg(errp, "virt: device plug request for unsupported device"
" type: %s", object_get_typename(OBJECT(dev)));
@@ -248,6 +251,8 @@ static void acpi_ged_unplug_request_cb(HotplugHandler *hotplug_dev,
if ((object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) &&
!(object_dynamic_cast(OBJECT(dev), TYPE_NVDIMM)))) {
acpi_memory_unplug_request_cb(hotplug_dev, &s->memhp_state, dev, errp);
+ } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+ acpi_cpu_unplug_request_cb(hotplug_dev, &s->cpuhp_state, dev, errp);
} else {
error_setg(errp, "acpi: device unplug request for unsupported device"
" type: %s", object_get_typename(OBJECT(dev)));
@@ -261,6 +266,8 @@ static void acpi_ged_unplug_cb(HotplugHandler *hotplug_dev,
if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
acpi_memory_unplug_cb(&s->memhp_state, dev, errp);
+ } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
+ acpi_cpu_unplug_cb(&s->cpuhp_state, dev, errp);
} else {
error_setg(errp, "acpi: device unplug for unsupported device"
" type: %s", object_get_typename(OBJECT(dev)));
@@ -272,6 +279,7 @@ static void acpi_ged_ospm_status(AcpiDeviceIf *adev, ACPIOSTInfoList ***list)
AcpiGedState *s = ACPI_GED(adev);
acpi_memory_ospm_status(&s->memhp_state, list);
+ acpi_cpu_ospm_status(&s->cpuhp_state, list);
}
static void acpi_ged_send_event(AcpiDeviceIf *adev, AcpiEventStatusBits ev)
@@ -286,6 +294,8 @@ static void acpi_ged_send_event(AcpiDeviceIf *adev, AcpiEventStatusBits ev)
sel = ACPI_GED_PWR_DOWN_EVT;
} else if (ev & ACPI_NVDIMM_HOTPLUG_STATUS) {
sel = ACPI_GED_NVDIMM_HOTPLUG_EVT;
+ } else if (ev & ACPI_CPU_HOTPLUG_STATUS) {
+ sel = ACPI_GED_CPU_HOTPLUG_EVT;
} else {
/* Unknown event. Return without generating interrupt. */
warn_report("GED: Unsupported event %d. No irq injected", ev);
@@ -371,6 +381,19 @@ static const VMStateDescription vmstate_acpi_ged = {
}
};
+static void acpi_ged_realize(DeviceState *dev, Error **errp)
+{
+ AcpiGedState *s = ACPI_GED(dev);
+ SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
+
+ /* initialize CPU Hotplug related regions */
+ memory_region_init(&s->container_cpuhp, OBJECT(dev), "cpuhp container",
+ ACPI_CPU_HOTPLUG_REG_LEN);
+ sysbus_init_mmio(sbd, &s->container_cpuhp);
+ cpu_hotplug_hw_init(&s->container_cpuhp, OBJECT(dev),
+ &s->cpuhp_state, 0);
+}
+
static void acpi_ged_initfn(Object *obj)
{
DeviceState *dev = DEVICE(obj);
@@ -411,6 +434,7 @@ static void acpi_ged_class_init(ObjectClass *class, void *data)
dc->desc = "ACPI Generic Event Device";
device_class_set_props(dc, acpi_ged_properties);
dc->vmsd = &vmstate_acpi_ged;
+ dc->realize = acpi_ged_realize;
hc->plug = acpi_ged_device_plug_cb;
hc->unplug_request = acpi_ged_unplug_request_cb;
diff --git a/include/hw/acpi/generic_event_device.h b/include/hw/acpi/generic_event_device.h
index ba84ce0214..e091ac2108 100644
--- a/include/hw/acpi/generic_event_device.h
+++ b/include/hw/acpi/generic_event_device.h
@@ -62,6 +62,7 @@
#include "hw/sysbus.h"
#include "hw/acpi/memory_hotplug.h"
#include "hw/acpi/ghes.h"
+#include "hw/acpi/cpu.h"
#include "qom/object.h"
#define ACPI_POWER_BUTTON_DEVICE "PWRB"
@@ -95,6 +96,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(AcpiGedState, ACPI_GED)
#define ACPI_GED_MEM_HOTPLUG_EVT 0x1
#define ACPI_GED_PWR_DOWN_EVT 0x2
#define ACPI_GED_NVDIMM_HOTPLUG_EVT 0x4
+#define ACPI_GED_CPU_HOTPLUG_EVT 0x8
typedef struct GEDState {
MemoryRegion evt;
@@ -106,6 +108,8 @@ struct AcpiGedState {
SysBusDevice parent_obj;
MemHotplugState memhp_state;
MemoryRegion container_memhp;
+ CPUHotplugState cpuhp_state;
+ MemoryRegion container_cpuhp;
GEDState ged_state;
uint32_t ged_event_bitmap;
qemu_irq irq;
--
2.34.1
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH V15 4/7] hw/acpi: Update GED _EVT method AML with CPU scan
2024-07-13 18:25 [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Salil Mehta via
` (2 preceding siblings ...)
2024-07-13 18:25 ` [PATCH V15 3/7] hw/acpi: Update ACPI GED framework to support vCPU Hotplug Salil Mehta via
@ 2024-07-13 18:25 ` Salil Mehta via
2024-07-15 12:55 ` Igor Mammedov
2024-07-13 18:25 ` [PATCH V15 5/7] hw/acpi: Update CPUs AML with cpu-(ctrl)dev change Salil Mehta via
` (7 subsequent siblings)
11 siblings, 1 reply; 35+ messages in thread
From: Salil Mehta via @ 2024-07-13 18:25 UTC (permalink / raw)
To: qemu-devel, qemu-arm
Cc: salil.mehta, maz, jean-philippe, jonathan.cameron, lpieralisi,
peter.maydell, richard.henderson, imammedo, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm,
Jonathan Cameron, Shaoqin Huang, Zhao Liu
OSPM evaluates _EVT method to map the event. The CPU hotplug event eventually
results in start of the CPU scan. Scan figures out the CPU and the kind of
event(plug/unplug) and notifies it back to the guest. Update the GED AML _EVT
method with the call to method \\_SB.CPUS.CSCN (via \\_SB.GED.CSCN)
Architecture specific code [1] might initialize its CPUs AML code by calling
common function build_cpus_aml() like below for ARM:
build_cpus_aml(scope, ms, opts, xx_madt_cpu_entry, memmap[VIRT_CPUHP_ACPI].base,
"\\_SB", "\\_SB.GED.CSCN", AML_SYSTEM_MEMORY);
[1] https://lore.kernel.org/qemu-devel/20240613233639.202896-13-salil.mehta@huawei.com/
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
Tested-by: Xianglai Li <lixianglai@loongson.cn>
Tested-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
Tested-by: Zhao Liu <zhao1.liu@intel.com>
---
hw/acpi/generic_event_device.c | 3 +++
include/hw/acpi/generic_event_device.h | 1 +
2 files changed, 4 insertions(+)
diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c
index 1b31d633ba..15ffa12cb2 100644
--- a/hw/acpi/generic_event_device.c
+++ b/hw/acpi/generic_event_device.c
@@ -108,6 +108,9 @@ void build_ged_aml(Aml *table, const char *name, HotplugHandler *hotplug_dev,
aml_append(if_ctx, aml_call0(MEMORY_DEVICES_CONTAINER "."
MEMORY_SLOT_SCAN_METHOD));
break;
+ case ACPI_GED_CPU_HOTPLUG_EVT:
+ aml_append(if_ctx, aml_call0(AML_GED_EVT_CPU_SCAN_METHOD));
+ break;
case ACPI_GED_PWR_DOWN_EVT:
aml_append(if_ctx,
aml_notify(aml_name(ACPI_POWER_BUTTON_DEVICE),
diff --git a/include/hw/acpi/generic_event_device.h b/include/hw/acpi/generic_event_device.h
index e091ac2108..40af3550b5 100644
--- a/include/hw/acpi/generic_event_device.h
+++ b/include/hw/acpi/generic_event_device.h
@@ -87,6 +87,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(AcpiGedState, ACPI_GED)
#define GED_DEVICE "GED"
#define AML_GED_EVT_REG "EREG"
#define AML_GED_EVT_SEL "ESEL"
+#define AML_GED_EVT_CPU_SCAN_METHOD "\\_SB.GED.CSCN"
/*
* Platforms need to specify the GED event bitmap
--
2.34.1
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH V15 5/7] hw/acpi: Update CPUs AML with cpu-(ctrl)dev change
2024-07-13 18:25 [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Salil Mehta via
` (3 preceding siblings ...)
2024-07-13 18:25 ` [PATCH V15 4/7] hw/acpi: Update GED _EVT method AML with CPU scan Salil Mehta via
@ 2024-07-13 18:25 ` Salil Mehta via
2024-07-15 13:39 ` Igor Mammedov
2024-07-13 18:25 ` [PATCH V15 6/7] physmem: Add helper function to destroy CPU AddressSpace Salil Mehta via
` (6 subsequent siblings)
11 siblings, 1 reply; 35+ messages in thread
From: Salil Mehta via @ 2024-07-13 18:25 UTC (permalink / raw)
To: qemu-devel, qemu-arm
Cc: salil.mehta, maz, jean-philippe, jonathan.cameron, lpieralisi,
peter.maydell, richard.henderson, imammedo, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm,
Jonathan Cameron, Shaoqin Huang, Zhao Liu
CPUs Control device(\\_SB.PCI0) register interface for the x86 arch is IO port
based and existing CPUs AML code assumes _CRS objects would evaluate to a system
resource which describes IO Port address. But on ARM arch CPUs control
device(\\_SB.PRES) register interface is memory-mapped hence _CRS object should
evaluate to system resource which describes memory-mapped base address. Update
build CPUs AML function to accept both IO/MEMORY region spaces and accordingly
update the _CRS object.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Tested-by: Xianglai Li <lixianglai@loongson.cn>
Tested-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
Tested-by: Zhao Liu <zhao1.liu@intel.com>
---
hw/acpi/cpu.c | 17 +++++++++++++----
hw/i386/acpi-build.c | 3 ++-
include/hw/acpi/cpu.h | 5 +++--
3 files changed, 18 insertions(+), 7 deletions(-)
diff --git a/hw/acpi/cpu.c b/hw/acpi/cpu.c
index cf5e9183e4..5cb60ca8bc 100644
--- a/hw/acpi/cpu.c
+++ b/hw/acpi/cpu.c
@@ -338,9 +338,10 @@ const VMStateDescription vmstate_cpu_hotplug = {
#define CPU_FW_EJECT_EVENT "CEJF"
void build_cpus_aml(Aml *table, MachineState *machine, CPUHotplugFeatures opts,
- build_madt_cpu_fn build_madt_cpu, hwaddr io_base,
+ build_madt_cpu_fn build_madt_cpu, hwaddr base_addr,
const char *res_root,
- const char *event_handler_method)
+ const char *event_handler_method,
+ AmlRegionSpace rs)
{
Aml *ifctx;
Aml *field;
@@ -364,14 +365,22 @@ void build_cpus_aml(Aml *table, MachineState *machine, CPUHotplugFeatures opts,
aml_name_decl("_UID", aml_string("CPU Hotplug resources")));
aml_append(cpu_ctrl_dev, aml_mutex(CPU_LOCK, 0));
+ assert((rs == AML_SYSTEM_IO) || (rs == AML_SYSTEM_MEMORY));
+
crs = aml_resource_template();
- aml_append(crs, aml_io(AML_DECODE16, io_base, io_base, 1,
+ if (rs == AML_SYSTEM_IO) {
+ aml_append(crs, aml_io(AML_DECODE16, base_addr, base_addr, 1,
ACPI_CPU_HOTPLUG_REG_LEN));
+ } else if (rs == AML_SYSTEM_MEMORY) {
+ aml_append(crs, aml_memory32_fixed(base_addr,
+ ACPI_CPU_HOTPLUG_REG_LEN, AML_READ_WRITE));
+ }
+
aml_append(cpu_ctrl_dev, aml_name_decl("_CRS", crs));
/* declare CPU hotplug MMIO region with related access fields */
aml_append(cpu_ctrl_dev,
- aml_operation_region("PRST", AML_SYSTEM_IO, aml_int(io_base),
+ aml_operation_region("PRST", rs, aml_int(base_addr),
ACPI_CPU_HOTPLUG_REG_LEN));
field = aml_field("PRST", AML_BYTE_ACC, AML_NOLOCK,
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index f4e366f64f..5d4bd2b710 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1536,7 +1536,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
.fw_unplugs_cpu = pm->smi_on_cpu_unplug,
};
build_cpus_aml(dsdt, machine, opts, pc_madt_cpu_entry,
- pm->cpu_hp_io_base, "\\_SB.PCI0", "\\_GPE._E02");
+ pm->cpu_hp_io_base, "\\_SB.PCI0", "\\_GPE._E02",
+ AML_SYSTEM_IO);
}
if (pcms->memhp_io_base && nr_mem) {
diff --git a/include/hw/acpi/cpu.h b/include/hw/acpi/cpu.h
index df87b15997..32654dc274 100644
--- a/include/hw/acpi/cpu.h
+++ b/include/hw/acpi/cpu.h
@@ -63,9 +63,10 @@ typedef void (*build_madt_cpu_fn)(int uid, const CPUArchIdList *apic_ids,
GArray *entry, bool force_enabled);
void build_cpus_aml(Aml *table, MachineState *machine, CPUHotplugFeatures opts,
- build_madt_cpu_fn build_madt_cpu, hwaddr io_base,
+ build_madt_cpu_fn build_madt_cpu, hwaddr base_addr,
const char *res_root,
- const char *event_handler_method);
+ const char *event_handler_method,
+ AmlRegionSpace rs);
void acpi_cpu_ospm_status(CPUHotplugState *cpu_st, ACPIOSTInfoList ***list);
--
2.34.1
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH V15 6/7] physmem: Add helper function to destroy CPU AddressSpace
2024-07-13 18:25 [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Salil Mehta via
` (4 preceding siblings ...)
2024-07-13 18:25 ` [PATCH V15 5/7] hw/acpi: Update CPUs AML with cpu-(ctrl)dev change Salil Mehta via
@ 2024-07-13 18:25 ` Salil Mehta via
2024-07-13 18:25 ` [PATCH V15 7/7] gdbstub: Add helper function to unregister GDB register space Salil Mehta via
` (5 subsequent siblings)
11 siblings, 0 replies; 35+ messages in thread
From: Salil Mehta via @ 2024-07-13 18:25 UTC (permalink / raw)
To: qemu-devel, qemu-arm
Cc: salil.mehta, maz, jean-philippe, jonathan.cameron, lpieralisi,
peter.maydell, richard.henderson, imammedo, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm,
Shaoqin Huang, Zhao Liu
Virtual CPU Hot-unplug leads to unrealization of a CPU object. This also
involves destruction of the CPU AddressSpace. Add common function to help
destroy the CPU AddressSpace.
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Tested-by: Xianglai Li <lixianglai@loongson.cn>
Tested-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
Tested-by: Zhao Liu <zhao1.liu@intel.com>
Acked-by: Igor Mammedov <imammedo@redhat.com>
---
include/exec/cpu-common.h | 8 ++++++++
include/hw/core/cpu.h | 1 +
system/physmem.c | 29 +++++++++++++++++++++++++++++
3 files changed, 38 insertions(+)
diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
index 815342d043..240ee04369 100644
--- a/include/exec/cpu-common.h
+++ b/include/exec/cpu-common.h
@@ -129,6 +129,14 @@ size_t qemu_ram_pagesize_largest(void);
*/
void cpu_address_space_init(CPUState *cpu, int asidx,
const char *prefix, MemoryRegion *mr);
+/**
+ * cpu_address_space_destroy:
+ * @cpu: CPU for which address space needs to be destroyed
+ * @asidx: integer index of this address space
+ *
+ * Note that with KVM only one address space is supported.
+ */
+void cpu_address_space_destroy(CPUState *cpu, int asidx);
void cpu_physical_memory_rw(hwaddr addr, void *buf,
hwaddr len, bool is_write);
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index a2c8536943..c7dbfdfb11 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -496,6 +496,7 @@ struct CPUState {
QSIMPLEQ_HEAD(, qemu_work_item) work_list;
struct CPUAddressSpace *cpu_ases;
+ int cpu_ases_count;
int num_ases;
AddressSpace *as;
MemoryRegion *memory;
diff --git a/system/physmem.c b/system/physmem.c
index 14aa025d41..ddd1fc3a25 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -763,6 +763,7 @@ void cpu_address_space_init(CPUState *cpu, int asidx,
if (!cpu->cpu_ases) {
cpu->cpu_ases = g_new0(CPUAddressSpace, cpu->num_ases);
+ cpu->cpu_ases_count = cpu->num_ases;
}
newas = &cpu->cpu_ases[asidx];
@@ -776,6 +777,34 @@ void cpu_address_space_init(CPUState *cpu, int asidx,
}
}
+void cpu_address_space_destroy(CPUState *cpu, int asidx)
+{
+ CPUAddressSpace *cpuas;
+
+ assert(cpu->cpu_ases);
+ assert(asidx >= 0 && asidx < cpu->num_ases);
+ /* KVM cannot currently support multiple address spaces. */
+ assert(asidx == 0 || !kvm_enabled());
+
+ cpuas = &cpu->cpu_ases[asidx];
+ if (tcg_enabled()) {
+ memory_listener_unregister(&cpuas->tcg_as_listener);
+ }
+
+ address_space_destroy(cpuas->as);
+ g_free_rcu(cpuas->as, rcu);
+
+ if (asidx == 0) {
+ /* reset the convenience alias for address space 0 */
+ cpu->as = NULL;
+ }
+
+ if (--cpu->cpu_ases_count == 0) {
+ g_free(cpu->cpu_ases);
+ cpu->cpu_ases = NULL;
+ }
+}
+
AddressSpace *cpu_get_address_space(CPUState *cpu, int asidx)
{
/* Return the AddressSpace corresponding to the specified index */
--
2.34.1
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [PATCH V15 7/7] gdbstub: Add helper function to unregister GDB register space
2024-07-13 18:25 [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Salil Mehta via
` (5 preceding siblings ...)
2024-07-13 18:25 ` [PATCH V15 6/7] physmem: Add helper function to destroy CPU AddressSpace Salil Mehta via
@ 2024-07-13 18:25 ` Salil Mehta via
2024-07-15 13:40 ` Igor Mammedov
2024-07-15 6:11 ` [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Zhao Liu
` (4 subsequent siblings)
11 siblings, 1 reply; 35+ messages in thread
From: Salil Mehta via @ 2024-07-13 18:25 UTC (permalink / raw)
To: qemu-devel, qemu-arm
Cc: salil.mehta, maz, jean-philippe, jonathan.cameron, lpieralisi,
peter.maydell, richard.henderson, imammedo, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm,
Shaoqin Huang, Zhao Liu
Add common function to help unregister the GDB register space. This shall be
done in context to the CPU unrealization.
Note: These are common functions exported to arch specific code. For example,
for ARM this code is being referred in associated arch specific patch-set:
Link: https://lore.kernel.org/qemu-devel/20230926103654.34424-1-salil.mehta@huawei.com/
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Tested-by: Xianglai Li <lixianglai@loongson.cn>
Tested-by: Miguel Luis <miguel.luis@oracle.com>
Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
Reviewed-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
Tested-by: Zhao Liu <zhao1.liu@intel.com>
---
gdbstub/gdbstub.c | 13 +++++++++++++
hw/core/cpu-common.c | 4 +++-
include/exec/gdbstub.h | 6 ++++++
3 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/gdbstub/gdbstub.c b/gdbstub/gdbstub.c
index b9ad0a063e..5da17d6530 100644
--- a/gdbstub/gdbstub.c
+++ b/gdbstub/gdbstub.c
@@ -618,6 +618,19 @@ void gdb_register_coprocessor(CPUState *cpu,
}
}
+void gdb_unregister_coprocessor_all(CPUState *cpu)
+{
+ /*
+ * Safe to nuke everything. GDBRegisterState::xml is static const char so
+ * it won't be freed
+ */
+ g_array_free(cpu->gdb_regs, true);
+
+ cpu->gdb_regs = NULL;
+ cpu->gdb_num_regs = 0;
+ cpu->gdb_num_g_regs = 0;
+}
+
static void gdb_process_breakpoint_remove_all(GDBProcess *p)
{
CPUState *cpu = gdb_get_first_cpu_in_process(p);
diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c
index b19e1fdacf..fe5383b4f9 100644
--- a/hw/core/cpu-common.c
+++ b/hw/core/cpu-common.c
@@ -281,7 +281,9 @@ static void cpu_common_finalize(Object *obj)
g_free(cpu->plugin_state);
}
#endif
- g_array_free(cpu->gdb_regs, TRUE);
+ /* If cleanup didn't happen in context to gdb_unregister_coprocessor_all */
+ if (cpu->gdb_regs)
+ g_array_free(cpu->gdb_regs, TRUE);
qemu_lockcnt_destroy(&cpu->in_ioctl_lock);
qemu_mutex_destroy(&cpu->work_mutex);
qemu_cond_destroy(cpu->halt_cond);
diff --git a/include/exec/gdbstub.h b/include/exec/gdbstub.h
index 1bd2c4ec2a..d73f424f56 100644
--- a/include/exec/gdbstub.h
+++ b/include/exec/gdbstub.h
@@ -40,6 +40,12 @@ void gdb_register_coprocessor(CPUState *cpu,
gdb_get_reg_cb get_reg, gdb_set_reg_cb set_reg,
const GDBFeature *feature, int g_pos);
+/**
+ * gdb_unregister_coprocessor_all() - unregisters supplemental set of registers
+ * @cpu - the CPU associated with registers
+ */
+void gdb_unregister_coprocessor_all(CPUState *cpu);
+
/**
* gdbserver_start: start the gdb server
* @port_or_device: connection spec for gdb
--
2.34.1
^ permalink raw reply related [flat|nested] 35+ messages in thread
* Re: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-13 18:25 [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Salil Mehta via
` (6 preceding siblings ...)
2024-07-13 18:25 ` [PATCH V15 7/7] gdbstub: Add helper function to unregister GDB register space Salil Mehta via
@ 2024-07-15 6:11 ` Zhao Liu
2024-07-15 8:45 ` Salil Mehta via
2024-07-15 11:03 ` Vishnu Pajjuri
` (3 subsequent siblings)
11 siblings, 1 reply; 35+ messages in thread
From: Zhao Liu @ 2024-07-15 6:11 UTC (permalink / raw)
To: Salil Mehta
Cc: qemu-devel, qemu-arm, maz, jean-philippe, jonathan.cameron,
lpieralisi, peter.maydell, richard.henderson, imammedo,
andrew.jones, david, philmd, eric.auger, oliver.upton, pbonzini,
mst, will, gshan, rafael, alex.bennee, linux, darren, ilkka,
vishnu, karl.heubaum, miguel.luis, salil.mehta, zhukeqian1,
wangxiongfeng2, wangyanan55, jiakernel2, maobibo, lixianglai,
npiggin, harshpb, linuxarm
Hi Salil,
I ran the unit tests again on x86 platform, and everything looks good.
Please feel free to keep my tested-by tag.
Regards,
Zhao
On Sat, Jul 13, 2024 at 07:25:09PM +0100, Salil Mehta via wrote:
> Date: Sat, 13 Jul 2024 19:25:09 +0100
> From: Salil Mehta via <qemu-devel@nongnu.org>
> Subject: [PATCH V15 0/7] Add architecture agnostic code to support vCPU
> Hotplug
> X-Mailer: git-send-email 2.34.1
>
> [Note: References are present at the last after the revision history]
>
> Virtual CPU hotplug support is being added across various architectures [1][3].
> This series adds various code bits common across all architectures:
>
> 1. vCPU creation and Parking code refactor [Patch 1]
> 2. Update ACPI GED framework to support vCPU Hotplug [Patch 2,3]
> 3. ACPI CPUs AML code change [Patch 4,5]
> 4. Helper functions to support unrealization of CPU objects [Patch 6,7]
>
> Repository:
>
> [*] Architecture *Agnostic* Patch-set (This series)
> V14: https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v3.arch.agnostic.v15
>
> NOTE: This series is meant to work in conjunction with the architecture-specific
> patch-set. For ARM, a combined patch-set (architecture agnostic + specific) was
> earlier pushed as RFC V2 [1]. Later, RFC V2 was split into the ARM Architecture
> specific patch-set RFC V3 [4] (a subset of RFC V2) and the architecture agnostic
> patch-set. Patch-set V14 is the latest version in that series. This series
> works in conjunction with RFC V4-rc2, present at the following link.
>
> [*] ARM Architecture *Specific* Patch-set
> RFC V3 [4]: https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v3
> RFC V4-rc2: https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v4-rc2 (combined)
>
>
> Revision History:
>
> Patch-set V14 -> V15
> 1. Addressed commnet from Igor Mammedov's on [PATCH V14 4/7]
> - Removed ACPI_CPU_SCAN_METHOD
> - Introduced AML_GED_EVT_CPU_SCAN_METHOD ("\\_SB.GED.CPSCN") macro
> 2. Fix the stray change of "assert (" in "PATCH V14 3/7"
> Link: https://lore.kernel.org/qemu-devel/20240712134201.214699-4-salil.mehta@huawei.com/
>
> Patch-set V13 -> V14
> 1. Addressed Igor Mammedov's following review comments
> - Mentioned abput new external APIs in the header note of [PATCH 1/7]
> - Merged Doc [PATCH V13 8/8] with [PATCH V14 3/7]
> - Introduced GED realize function for various CPU Hotplug regions initializations
> - Added back event handler method to indirectly expose \\_SB.CPUS.CSCN to GED
> _EVT. Like for ARM, it would be through \\_SB.GED.CSCN event handler method
> - Collected the Ack given for [Patch V13 6/8]
> - Added back the gfree'ing of GDB regs in common finalize and made it conditional
> - Updated the header notes of [PATCH V13 3/8,4/8,5/8] to reflect the changes
>
> Patch-set V12 -> V13
> 1. Added Reviewed-by Tag of Harsh Prateek Bora's (IBM) [PATCH V12 1/8]
> 2. Moved the kvm_{create,park,unpark}_vcpu prototypes from accel/kvm/kvm-cpus.h
> to include/sysemu/kvm.h. These can later be exported through AccelOps.
> Link: https://lore.kernel.org/qemu-devel/62f55169-1796-4d8e-a35d-7f003a172750@linux.ibm.com/
>
> Patch-set V11 -> V12
> 1. Addressed Harsh Prateek Bora's (IBM) comment
> - Changed @cpu to @vcpu_id in the kvm_unpark_vcpu protoype header/
> 2. Added Zhao Liu's (Intel) Tested-by for whole series
> - Qtest does not breaks on Intel platforms now.
> 3. Added Zhao Liu's (Intel) Reviewed-by for [PATCH V11 {1/8 - 3/8}]
> Link: https://lore.kernel.org/qemu-devel/ZlRSPuJGBgyEUW6w@intel.com/
> Link: https://lore.kernel.org/qemu-devel/a5f3d78e-cfed-441f-9c56-e3e78fa5edee@linux.ibm.com/
>
> Patch-set V10 -> V11
> 1. Addressed Nicholas Piggin's (IBM) comment
> - moved the traces in kvm_unpark_vcpu and kvm_create_vcpu at the end
> - Added the Reviewed-by Tag for [PATCH V10 1/8]
> 2. Addressed Alex Bennée's (Linaro) comments
> - Added a note explaining dependency of the [PATCH V10 7/8] on Arch specific patch-set
> Link: https://lore.kernel.org/qemu-devel/D1FS5GOOFWWK.2PNRIVL0V6DBL@gmail.com/
> Link: https://lore.kernel.org/qemu-devel/87frubi402.fsf@draig.linaro.org/
>
> Patch-set V9 -> V10
> 1. Addressed Nicholas Piggin's (IBM) & Philippe Mathieu-Daudé (Linaro) comments
> - carved out kvm_unpark_vcpu and added its trace
> - Widened the scope of the kvm_unpark_vcpu so that it can be used by generic framework
> being thought out
> Link: https://lore.kernel.org/qemu-devel/20240519210620.228342-1-salil.mehta@huawei.com/
> Link: https://lore.kernel.org/qemu-devel/e94b0e14-efee-4050-9c9f-08382a36b63a@linaro.org/
>
> Patch-set V8 -> V9
> 1. Addressed Vishnu Pajjuri's (Ampere) comments
> - Added kvm_fd to trace in kvm_create_vcpu
> - Some clean ups: arch vcpu-id and sbd variable
> - Added the missed initialization of cpu->gdb_num_regs
> 2. Addressed the commnet from Zhao Liu (Intel)
> - Make initialization of CPU Hotplug state conditional (possible_cpu_arch_ids!=NULL)
> Link: https://lore.kernel.org/qemu-devel/20240312020000.12992-1-salil.mehta@huawei.com/
>
> Patch-set V7 -> V8
> 1. Rebased and Fixed the conflicts
>
> Patch-set V6 -> V7
> 1. Addressed Alex Bennée's comments
> - Updated the docs
> 2. Addressed Igor Mammedov's comments
> - Merged patches [Patch V6 3/9] & [Patch V6 7/9] with [Patch V6 4/9]
> - Updated commit-log of [Patch V6 1/9] and [Patch V6 5/9]
> 3. Added Shaoqin Huang's Reviewed-by tags for whole series.
> Link: https://lore.kernel.org/qemu-devel/20231013105129.25648-1-salil.mehta@huawei.com/
>
> Patch-set V5 -> V6
> 1. Addressed Gavin Shan's comments
> - Fixed the assert() ranges of address spaces
> - Rebased the patch-set to latest changes in the qemu.git
> - Added Reviewed-by tags for patches {8,9}
> 2. Addressed Jonathan Cameron's comments
> - Updated commit-log for [Patch V5 1/9] with mention of trace events
> - Added Reviewed-by tags for patches {1,5}
> 3. Added Tested-by tags from Xianglai Li
> 4. Fixed checkpatch.pl error "Qemu -> QEMU" in [Patch V5 1/9]
> Link: https://lore.kernel.org/qemu-devel/20231011194355.15628-1-salil.mehta@huawei.com/
>
> Patch-set V4 -> V5
> 1. Addressed Gavin Shan's comments
> - Fixed the trace events print string for kvm_{create,get,park,destroy}_vcpu
> - Added Reviewed-by tag for patch {1}
> 2. Added Shaoqin Huang's Reviewed-by tags for Patches {2,3}
> 3. Added Tested-by Tag from Vishnu Pajjuri to the patch-set
> 4. Dropped the ARM specific [Patch V4 10/10]
> Link: https://lore.kernel.org/qemu-devel/20231009203601.17584-1-salil.mehta@huawei.com/
>
> Patch-set V3 -> V4
> 1. Addressed David Hilderbrand's comments
> - Fixed the wrong doc comment of kvm_park_vcpu API prototype
> - Added Reviewed-by tags for patches {2,4}
> Link: https://lore.kernel.org/qemu-devel/20231009112812.10612-1-salil.mehta@huawei.com/
>
> Patch-set V2 -> V3
> 1. Addressed Jonathan Cameron's comments
> - Fixed 'vcpu-id' type wrongly changed from 'unsigned long' to 'integer'
> - Removed unnecessary use of variable 'vcpu_id' in kvm_park_vcpu
> - Updated [Patch V2 3/10] commit-log with details of ACPI_CPU_SCAN_METHOD macro
> - Updated [Patch V2 5/10] commit-log with details of conditional event handler method
> - Added Reviewed-by tags for patches {2,3,4,6,7}
> 2. Addressed Gavin Shan's comments
> - Remove unnecessary use of variable 'vcpu_id' in kvm_par_vcpu
> - Fixed return value in kvm_get_vcpu from -1 to -ENOENT
> - Reset the value of 'gdb_num_g_regs' in gdb_unregister_coprocessor_all
> - Fixed the kvm_{create,park}_vcpu prototypes docs
> - Added Reviewed-by tags for patches {2,3,4,5,6,7,9,10}
> 3. Addressed one earlier missed comment by Alex Bennée in RFC V1
> - Added traces instead of DPRINTF in the newly added and some existing functions
> Link: https://lore.kernel.org/qemu-devel/20230930001933.2660-1-salil.mehta@huawei.com/
>
> Patch-set V1 -> V2
> 1. Addressed Alex Bennée's comments
> - Refactored the kvm_create_vcpu logic to get rid of goto
> - Added the docs for kvm_{create,park}_vcpu prototypes
> - Splitted the gdbstub and AddressSpace destruction change into separate patches
> - Added Reviewed-by tags for patches {2,10}
> Link: https://lore.kernel.org/qemu-devel/20230929124304.13672-1-salil.mehta@huawei.com/
>
> References:
>
> [1] https://lore.kernel.org/qemu-devel/20230926100436.28284-1-salil.mehta@huawei.com/
> [2] https://lore.kernel.org/all/20230913163823.7880-1-james.morse@arm.com/
> [3] https://lore.kernel.org/qemu-devel/cover.1695697701.git.lixianglai@loongson.cn/
> [4] https://lore.kernel.org/qemu-devel/20240613233639.202896-2-salil.mehta@huawei.com/
>
> Salil Mehta (7):
> accel/kvm: Extract common KVM vCPU {creation,parking} code
> hw/acpi: Move CPU ctrl-dev MMIO region len macro to common header file
> hw/acpi: Update ACPI GED framework to support vCPU Hotplug
> hw/acpi: Update GED _EVT method AML with CPU scan
> hw/acpi: Update CPUs AML with cpu-(ctrl)dev change
> physmem: Add helper function to destroy CPU AddressSpace
> gdbstub: Add helper function to unregister GDB register space
>
> accel/kvm/kvm-all.c | 95 +++++++++++++++++---------
> accel/kvm/kvm-cpus.h | 1 -
> accel/kvm/trace-events | 5 +-
> docs/specs/acpi_hw_reduced_hotplug.rst | 3 +-
> gdbstub/gdbstub.c | 13 ++++
> hw/acpi/acpi-cpu-hotplug-stub.c | 6 ++
> hw/acpi/cpu.c | 18 +++--
> hw/acpi/generic_event_device.c | 27 ++++++++
> hw/core/cpu-common.c | 4 +-
> hw/i386/acpi-build.c | 3 +-
> include/exec/cpu-common.h | 8 +++
> include/exec/gdbstub.h | 6 ++
> include/hw/acpi/cpu.h | 7 +-
> include/hw/acpi/generic_event_device.h | 5 ++
> include/hw/core/cpu.h | 1 +
> include/sysemu/kvm.h | 25 +++++++
> system/physmem.c | 29 ++++++++
> 17 files changed, 212 insertions(+), 44 deletions(-)
>
> --
> 2.34.1
>
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* RE: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-15 6:11 ` [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Zhao Liu
@ 2024-07-15 8:45 ` Salil Mehta via
0 siblings, 0 replies; 35+ messages in thread
From: Salil Mehta via @ 2024-07-15 8:45 UTC (permalink / raw)
To: Zhao Liu
Cc: qemu-devel@nongnu.org, qemu-arm@nongnu.org, maz@kernel.org,
jean-philippe@linaro.org, Jonathan Cameron, lpieralisi@kernel.org,
peter.maydell@linaro.org, richard.henderson@linaro.org,
imammedo@redhat.com, andrew.jones@linux.dev, david@redhat.com,
philmd@linaro.org, eric.auger@redhat.com, oliver.upton@linux.dev,
pbonzini@redhat.com, mst@redhat.com, will@kernel.org,
gshan@redhat.com, rafael@kernel.org, alex.bennee@linaro.org,
linux@armlinux.org.uk, darren@os.amperecomputing.com,
ilkka@os.amperecomputing.com, vishnu@os.amperecomputing.com,
karl.heubaum@oracle.com, miguel.luis@oracle.com,
salil.mehta@opnsrc.net, zhukeqian, wangxiongfeng (C),
wangyanan (Y), jiakernel2@gmail.com, maobibo@loongson.cn,
lixianglai@loongson.cn, npiggin@gmail.com, harshpb@linux.ibm.com,
Linuxarm
Hi Zhao,
> From: Zhao Liu <zhao1.liu@intel.com>
> Sent: Monday, July 15, 2024 7:11 AM
> To: Salil Mehta <salil.mehta@huawei.com>
>
> Hi Salil,
>
> I ran the unit tests again on x86 platform, and everything looks good.
>
> Please feel free to keep my tested-by tag.
Many thanks for confirming this. Appreciate this.
Best Wishes
Salil.
>
> Regards,
> Zhao
>
> On Sat, Jul 13, 2024 at 07:25:09PM +0100, Salil Mehta via wrote:
> > Date: Sat, 13 Jul 2024 19:25:09 +0100
> > From: Salil Mehta via <qemu-devel@nongnu.org>
> > Subject: [PATCH V15 0/7] Add architecture agnostic code to support
> > vCPU Hotplug
> > X-Mailer: git-send-email 2.34.1
> >
> > [Note: References are present at the last after the revision history]
> >
> > Virtual CPU hotplug support is being added across various architectures
> [1][3].
> > This series adds various code bits common across all architectures:
> >
> > 1. vCPU creation and Parking code refactor [Patch 1] 2. Update ACPI
> > GED framework to support vCPU Hotplug [Patch 2,3] 3. ACPI CPUs AML
> > code change [Patch 4,5] 4. Helper functions to support unrealization
> > of CPU objects [Patch 6,7]
> >
> > Repository:
> >
> > [*] Architecture *Agnostic* Patch-set (This series)
> > V14: https://github.com/salil-mehta/qemu.git
> > virt-cpuhp-armv8/rfc-v3.arch.agnostic.v15
> >
> > NOTE: This series is meant to work in conjunction with the architecture-
> specific
> > patch-set. For ARM, a combined patch-set (architecture agnostic +
> specific) was
> > earlier pushed as RFC V2 [1]. Later, RFC V2 was split into the ARM
> Architecture
> > specific patch-set RFC V3 [4] (a subset of RFC V2) and the architecture
> agnostic
> > patch-set. Patch-set V14 is the latest version in that series. This series
> > works in conjunction with RFC V4-rc2, present at the following link.
> >
> > [*] ARM Architecture *Specific* Patch-set
> > RFC V3 [4]: https://github.com/salil-mehta/qemu.git virt-cpuhp-
> armv8/rfc-v3
> > RFC V4-rc2: https://github.com/salil-mehta/qemu.git
> > virt-cpuhp-armv8/rfc-v4-rc2 (combined)
> >
> >
> > Revision History:
> >
> > Patch-set V14 -> V15
> > 1. Addressed commnet from Igor Mammedov's on [PATCH V14 4/7]
> > - Removed ACPI_CPU_SCAN_METHOD
> > - Introduced AML_GED_EVT_CPU_SCAN_METHOD
> ("\\_SB.GED.CPSCN") macro
> > 2. Fix the stray change of "assert (" in "PATCH V14 3/7"
> > Link:
> > https://lore.kernel.org/qemu-devel/20240712134201.214699-4-salil.mehta
> > @huawei.com/
> >
> > Patch-set V13 -> V14
> > 1. Addressed Igor Mammedov's following review comments
> > - Mentioned abput new external APIs in the header note of [PATCH 1/7]
> > - Merged Doc [PATCH V13 8/8] with [PATCH V14 3/7]
> > - Introduced GED realize function for various CPU Hotplug regions
> initializations
> > - Added back event handler method to indirectly expose
> \\_SB.CPUS.CSCN to GED
> > _EVT. Like for ARM, it would be through \\_SB.GED.CSCN event handler
> method
> > - Collected the Ack given for [Patch V13 6/8]
> > - Added back the gfree'ing of GDB regs in common finalize and made it
> conditional
> > - Updated the header notes of [PATCH V13 3/8,4/8,5/8] to reflect
> > the changes
> >
> > Patch-set V12 -> V13
> > 1. Added Reviewed-by Tag of Harsh Prateek Bora's (IBM) [PATCH V12 1/8]
> > 2. Moved the kvm_{create,park,unpark}_vcpu prototypes from
> accel/kvm/kvm-cpus.h
> > to include/sysemu/kvm.h. These can later be exported through
> AccelOps.
> > Link:
> > https://lore.kernel.org/qemu-devel/62f55169-1796-4d8e-a35d-
> 7f003a17275
> > 0@linux.ibm.com/
> >
> > Patch-set V11 -> V12
> > 1. Addressed Harsh Prateek Bora's (IBM) comment
> > - Changed @cpu to @vcpu_id in the kvm_unpark_vcpu protoype
> header/
> > 2. Added Zhao Liu's (Intel) Tested-by for whole series
> > - Qtest does not breaks on Intel platforms now.
> > 3. Added Zhao Liu's (Intel) Reviewed-by for [PATCH V11 {1/8 - 3/8}]
> > Link: https://lore.kernel.org/qemu-
> devel/ZlRSPuJGBgyEUW6w@intel.com/
> > Link:
> > https://lore.kernel.org/qemu-devel/a5f3d78e-cfed-441f-9c56-
> e3e78fa5ede
> > e@linux.ibm.com/
> >
> > Patch-set V10 -> V11
> > 1. Addressed Nicholas Piggin's (IBM) comment
> > - moved the traces in kvm_unpark_vcpu and kvm_create_vcpu at the
> end
> > - Added the Reviewed-by Tag for [PATCH V10 1/8] 2. Addressed Alex
> > Bennée's (Linaro) comments
> > - Added a note explaining dependency of the [PATCH V10 7/8] on Arch
> > specific patch-set
> > Link:
> > https://lore.kernel.org/qemu-
> devel/D1FS5GOOFWWK.2PNRIVL0V6DBL@gmail.co
> > m/
> > Link:
> > https://lore.kernel.org/qemu-devel/87frubi402.fsf@draig.linaro.org/
> >
> > Patch-set V9 -> V10
> > 1. Addressed Nicholas Piggin's (IBM) & Philippe Mathieu-Daudé (Linaro)
> comments
> > - carved out kvm_unpark_vcpu and added its trace
> > - Widened the scope of the kvm_unpark_vcpu so that it can be used by
> generic framework
> > being thought out
> > Link:
> > https://lore.kernel.org/qemu-devel/20240519210620.228342-1-salil.mehta
> > @huawei.com/
> > Link:
> > https://lore.kernel.org/qemu-devel/e94b0e14-efee-4050-9c9f-
> 08382a36b63
> > a@linaro.org/
> >
> > Patch-set V8 -> V9
> > 1. Addressed Vishnu Pajjuri's (Ampere) comments
> > - Added kvm_fd to trace in kvm_create_vcpu
> > - Some clean ups: arch vcpu-id and sbd variable
> > - Added the missed initialization of cpu->gdb_num_regs 2. Addressed
> > the commnet from Zhao Liu (Intel)
> > - Make initialization of CPU Hotplug state conditional
> > (possible_cpu_arch_ids!=NULL)
> > Link:
> > https://lore.kernel.org/qemu-devel/20240312020000.12992-1-
> salil.mehta@
> > huawei.com/
> >
> > Patch-set V7 -> V8
> > 1. Rebased and Fixed the conflicts
> >
> > Patch-set V6 -> V7
> > 1. Addressed Alex Bennée's comments
> > - Updated the docs
> > 2. Addressed Igor Mammedov's comments
> > - Merged patches [Patch V6 3/9] & [Patch V6 7/9] with [Patch V6 4/9]
> > - Updated commit-log of [Patch V6 1/9] and [Patch V6 5/9]
> > 3. Added Shaoqin Huang's Reviewed-by tags for whole series.
> > Link:
> > https://lore.kernel.org/qemu-devel/20231013105129.25648-1-
> salil.mehta@
> > huawei.com/
> >
> > Patch-set V5 -> V6
> > 1. Addressed Gavin Shan's comments
> > - Fixed the assert() ranges of address spaces
> > - Rebased the patch-set to latest changes in the qemu.git
> > - Added Reviewed-by tags for patches {8,9} 2. Addressed Jonathan
> > Cameron's comments
> > - Updated commit-log for [Patch V5 1/9] with mention of trace events
> > - Added Reviewed-by tags for patches {1,5} 3. Added Tested-by tags
> > from Xianglai Li 4. Fixed checkpatch.pl error "Qemu -> QEMU" in [Patch
> > V5 1/9]
> > Link:
> > https://lore.kernel.org/qemu-devel/20231011194355.15628-1-
> salil.mehta@
> > huawei.com/
> >
> > Patch-set V4 -> V5
> > 1. Addressed Gavin Shan's comments
> > - Fixed the trace events print string for
> kvm_{create,get,park,destroy}_vcpu
> > - Added Reviewed-by tag for patch {1} 2. Added Shaoqin Huang's
> > Reviewed-by tags for Patches {2,3} 3. Added Tested-by Tag from Vishnu
> > Pajjuri to the patch-set 4. Dropped the ARM specific [Patch V4 10/10]
> > Link:
> > https://lore.kernel.org/qemu-devel/20231009203601.17584-1-
> salil.mehta@
> > huawei.com/
> >
> > Patch-set V3 -> V4
> > 1. Addressed David Hilderbrand's comments
> > - Fixed the wrong doc comment of kvm_park_vcpu API prototype
> > - Added Reviewed-by tags for patches {2,4}
> > Link:
> > https://lore.kernel.org/qemu-devel/20231009112812.10612-1-
> salil.mehta@
> > huawei.com/
> >
> > Patch-set V2 -> V3
> > 1. Addressed Jonathan Cameron's comments
> > - Fixed 'vcpu-id' type wrongly changed from 'unsigned long' to 'integer'
> > - Removed unnecessary use of variable 'vcpu_id' in kvm_park_vcpu
> > - Updated [Patch V2 3/10] commit-log with details of
> ACPI_CPU_SCAN_METHOD macro
> > - Updated [Patch V2 5/10] commit-log with details of conditional event
> handler method
> > - Added Reviewed-by tags for patches {2,3,4,6,7} 2. Addressed Gavin
> > Shan's comments
> > - Remove unnecessary use of variable 'vcpu_id' in kvm_par_vcpu
> > - Fixed return value in kvm_get_vcpu from -1 to -ENOENT
> > - Reset the value of 'gdb_num_g_regs' in
> gdb_unregister_coprocessor_all
> > - Fixed the kvm_{create,park}_vcpu prototypes docs
> > - Added Reviewed-by tags for patches {2,3,4,5,6,7,9,10} 3.
> > Addressed one earlier missed comment by Alex Bennée in RFC V1
> > - Added traces instead of DPRINTF in the newly added and some
> > existing functions
> > Link:
> > https://lore.kernel.org/qemu-devel/20230930001933.2660-1-
> salil.mehta@h
> > uawei.com/
> >
> > Patch-set V1 -> V2
> > 1. Addressed Alex Bennée's comments
> > - Refactored the kvm_create_vcpu logic to get rid of goto
> > - Added the docs for kvm_{create,park}_vcpu prototypes
> > - Splitted the gdbstub and AddressSpace destruction change into
> separate patches
> > - Added Reviewed-by tags for patches {2,10}
> > Link:
> > https://lore.kernel.org/qemu-devel/20230929124304.13672-1-
> salil.mehta@
> > huawei.com/
> >
> > References:
> >
> > [1]
> > https://lore.kernel.org/qemu-devel/20230926100436.28284-1-
> salil.mehta@
> > huawei.com/ [2]
> > https://lore.kernel.org/all/20230913163823.7880-1-
> james.morse@arm.com/
> > [3]
> > https://lore.kernel.org/qemu-devel/cover.1695697701.git.lixianglai@loo
> > ngson.cn/ [4]
> > https://lore.kernel.org/qemu-devel/20240613233639.202896-2-salil.mehta
> > @huawei.com/
> >
> > Salil Mehta (7):
> > accel/kvm: Extract common KVM vCPU {creation,parking} code
> > hw/acpi: Move CPU ctrl-dev MMIO region len macro to common header
> file
> > hw/acpi: Update ACPI GED framework to support vCPU Hotplug
> > hw/acpi: Update GED _EVT method AML with CPU scan
> > hw/acpi: Update CPUs AML with cpu-(ctrl)dev change
> > physmem: Add helper function to destroy CPU AddressSpace
> > gdbstub: Add helper function to unregister GDB register space
> >
> > accel/kvm/kvm-all.c | 95 +++++++++++++++++---------
> > accel/kvm/kvm-cpus.h | 1 -
> > accel/kvm/trace-events | 5 +-
> > docs/specs/acpi_hw_reduced_hotplug.rst | 3 +-
> > gdbstub/gdbstub.c | 13 ++++
> > hw/acpi/acpi-cpu-hotplug-stub.c | 6 ++
> > hw/acpi/cpu.c | 18 +++--
> > hw/acpi/generic_event_device.c | 27 ++++++++
> > hw/core/cpu-common.c | 4 +-
> > hw/i386/acpi-build.c | 3 +-
> > include/exec/cpu-common.h | 8 +++
> > include/exec/gdbstub.h | 6 ++
> > include/hw/acpi/cpu.h | 7 +-
> > include/hw/acpi/generic_event_device.h | 5 ++
> > include/hw/core/cpu.h | 1 +
> > include/sysemu/kvm.h | 25 +++++++
> > system/physmem.c | 29 ++++++++
> > 17 files changed, 212 insertions(+), 44 deletions(-)
> >
> > --
> > 2.34.1
> >
> >
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-13 18:25 [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Salil Mehta via
` (7 preceding siblings ...)
2024-07-15 6:11 ` [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Zhao Liu
@ 2024-07-15 11:03 ` Vishnu Pajjuri
2024-07-15 11:07 ` Salil Mehta
2024-07-15 11:13 ` Michael S. Tsirkin
` (2 subsequent siblings)
11 siblings, 1 reply; 35+ messages in thread
From: Vishnu Pajjuri @ 2024-07-15 11:03 UTC (permalink / raw)
To: Salil Mehta, qemu-devel, qemu-arm
Cc: maz, jean-philippe, jonathan.cameron, lpieralisi, peter.maydell,
richard.henderson, imammedo, andrew.jones, david, philmd,
eric.auger, oliver.upton, pbonzini, mst, will, gshan, rafael,
alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm
[-- Attachment #1: Type: text/plain, Size: 10209 bytes --]
Hi Salil,
On 13-07-2024 23:55, Salil Mehta wrote:
> [Note: References are present at the last after the revision history]
>
> Virtual CPU hotplug support is being added across various architectures [1][3].
> This series adds various code bits common across all architectures:
>
> 1. vCPU creation and Parking code refactor [Patch 1]
> 2. Update ACPI GED framework to support vCPU Hotplug [Patch 2,3]
> 3. ACPI CPUs AML code change [Patch 4,5]
> 4. Helper functions to support unrealization of CPU objects [Patch 6,7]
>
> Repository:
>
> [*] Architecture *Agnostic* Patch-set (This series)
> V14:https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v3.arch.agnostic.v15
>
> NOTE: This series is meant to work in conjunction with the architecture-specific
> patch-set. For ARM, a combined patch-set (architecture agnostic + specific) was
> earlier pushed as RFC V2 [1]. Later, RFC V2 was split into the ARM Architecture
> specific patch-set RFC V3 [4] (a subset of RFC V2) and the architecture agnostic
> patch-set. Patch-set V14 is the latest version in that series. This series
> works in conjunction with RFC V4-rc2, present at the following link.
>
> [*] ARM Architecture *Specific* Patch-set
> RFC V3 [4]:https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v3
> RFC V4-rc2:https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v4-rc2 (combined)
>
>
> Revision History:
>
> Patch-set V14 -> V15
> 1. Addressed commnet from Igor Mammedov's on [PATCH V14 4/7]
> - Removed ACPI_CPU_SCAN_METHOD
> - Introduced AML_GED_EVT_CPU_SCAN_METHOD ("\\_SB.GED.CPSCN") macro
> 2. Fix the stray change of "assert (" in "PATCH V14 3/7"
> Link:https://lore.kernel.org/qemu-devel/20240712134201.214699-4-salil.mehta@huawei.com/
I tried following test cases with rfc-v4-rc2 and kernel patches v10, and
it looks good on Ampere platforms.
* Regular hotplug and hot unplug tests
* Save/restore VM, suspend/resume VM with and with out hot-plugging
vcpus tests
* Live migration with and with out hot-plugging vcpus tests
Please feel free to add,
Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
_Regards_,
-Vishnu.
> Patch-set V13 -> V14
> 1. Addressed Igor Mammedov's following review comments
> - Mentioned abput new external APIs in the header note of [PATCH 1/7]
> - Merged Doc [PATCH V13 8/8] with [PATCH V14 3/7]
> - Introduced GED realize function for various CPU Hotplug regions initializations
> - Added back event handler method to indirectly expose \\_SB.CPUS.CSCN to GED
> _EVT. Like for ARM, it would be through \\_SB.GED.CSCN event handler method
> - Collected the Ack given for [Patch V13 6/8]
> - Added back the gfree'ing of GDB regs in common finalize and made it conditional
> - Updated the header notes of [PATCH V13 3/8,4/8,5/8] to reflect the changes
>
> Patch-set V12 -> V13
> 1. Added Reviewed-by Tag of Harsh Prateek Bora's (IBM) [PATCH V12 1/8]
> 2. Moved the kvm_{create,park,unpark}_vcpu prototypes from accel/kvm/kvm-cpus.h
> to include/sysemu/kvm.h. These can later be exported through AccelOps.
> Link:https://lore.kernel.org/qemu-devel/62f55169-1796-4d8e-a35d-7f003a172750@linux.ibm.com/
>
> Patch-set V11 -> V12
> 1. Addressed Harsh Prateek Bora's (IBM) comment
> - Changed @cpu to @vcpu_id in the kvm_unpark_vcpu protoype header/
> 2. Added Zhao Liu's (Intel) Tested-by for whole series
> - Qtest does not breaks on Intel platforms now.
> 3. Added Zhao Liu's (Intel) Reviewed-by for [PATCH V11 {1/8 - 3/8}]
> Link:https://lore.kernel.org/qemu-devel/ZlRSPuJGBgyEUW6w@intel.com/
> Link:https://lore.kernel.org/qemu-devel/a5f3d78e-cfed-441f-9c56-e3e78fa5edee@linux.ibm.com/
>
> Patch-set V10 -> V11
> 1. Addressed Nicholas Piggin's (IBM) comment
> - moved the traces in kvm_unpark_vcpu and kvm_create_vcpu at the end
> - Added the Reviewed-by Tag for [PATCH V10 1/8]
> 2. Addressed Alex Bennée's (Linaro) comments
> - Added a note explaining dependency of the [PATCH V10 7/8] on Arch specific patch-set
> Link:https://lore.kernel.org/qemu-devel/D1FS5GOOFWWK.2PNRIVL0V6DBL@gmail.com/
> Link:https://lore.kernel.org/qemu-devel/87frubi402.fsf@draig.linaro.org/
>
> Patch-set V9 -> V10
> 1. Addressed Nicholas Piggin's (IBM) & Philippe Mathieu-Daudé (Linaro) comments
> - carved out kvm_unpark_vcpu and added its trace
> - Widened the scope of the kvm_unpark_vcpu so that it can be used by generic framework
> being thought out
> Link:https://lore.kernel.org/qemu-devel/20240519210620.228342-1-salil.mehta@huawei.com/
> Link:https://lore.kernel.org/qemu-devel/e94b0e14-efee-4050-9c9f-08382a36b63a@linaro.org/
>
> Patch-set V8 -> V9
> 1. Addressed Vishnu Pajjuri's (Ampere) comments
> - Added kvm_fd to trace in kvm_create_vcpu
> - Some clean ups: arch vcpu-id and sbd variable
> - Added the missed initialization of cpu->gdb_num_regs
> 2. Addressed the commnet from Zhao Liu (Intel)
> - Make initialization of CPU Hotplug state conditional (possible_cpu_arch_ids!=NULL)
> Link:https://lore.kernel.org/qemu-devel/20240312020000.12992-1-salil.mehta@huawei.com/
>
> Patch-set V7 -> V8
> 1. Rebased and Fixed the conflicts
>
> Patch-set V6 -> V7
> 1. Addressed Alex Bennée's comments
> - Updated the docs
> 2. Addressed Igor Mammedov's comments
> - Merged patches [Patch V6 3/9] & [Patch V6 7/9] with [Patch V6 4/9]
> - Updated commit-log of [Patch V6 1/9] and [Patch V6 5/9]
> 3. Added Shaoqin Huang's Reviewed-by tags for whole series.
> Link:https://lore.kernel.org/qemu-devel/20231013105129.25648-1-salil.mehta@huawei.com/
>
> Patch-set V5 -> V6
> 1. Addressed Gavin Shan's comments
> - Fixed the assert() ranges of address spaces
> - Rebased the patch-set to latest changes in the qemu.git
> - Added Reviewed-by tags for patches {8,9}
> 2. Addressed Jonathan Cameron's comments
> - Updated commit-log for [Patch V5 1/9] with mention of trace events
> - Added Reviewed-by tags for patches {1,5}
> 3. Added Tested-by tags from Xianglai Li
> 4. Fixed checkpatch.pl error "Qemu -> QEMU" in [Patch V5 1/9]
> Link:https://lore.kernel.org/qemu-devel/20231011194355.15628-1-salil.mehta@huawei.com/
>
> Patch-set V4 -> V5
> 1. Addressed Gavin Shan's comments
> - Fixed the trace events print string for kvm_{create,get,park,destroy}_vcpu
> - Added Reviewed-by tag for patch {1}
> 2. Added Shaoqin Huang's Reviewed-by tags for Patches {2,3}
> 3. Added Tested-by Tag from Vishnu Pajjuri to the patch-set
> 4. Dropped the ARM specific [Patch V4 10/10]
> Link:https://lore.kernel.org/qemu-devel/20231009203601.17584-1-salil.mehta@huawei.com/
>
> Patch-set V3 -> V4
> 1. Addressed David Hilderbrand's comments
> - Fixed the wrong doc comment of kvm_park_vcpu API prototype
> - Added Reviewed-by tags for patches {2,4}
> Link:https://lore.kernel.org/qemu-devel/20231009112812.10612-1-salil.mehta@huawei.com/
>
> Patch-set V2 -> V3
> 1. Addressed Jonathan Cameron's comments
> - Fixed 'vcpu-id' type wrongly changed from 'unsigned long' to 'integer'
> - Removed unnecessary use of variable 'vcpu_id' in kvm_park_vcpu
> - Updated [Patch V2 3/10] commit-log with details of ACPI_CPU_SCAN_METHOD macro
> - Updated [Patch V2 5/10] commit-log with details of conditional event handler method
> - Added Reviewed-by tags for patches {2,3,4,6,7}
> 2. Addressed Gavin Shan's comments
> - Remove unnecessary use of variable 'vcpu_id' in kvm_par_vcpu
> - Fixed return value in kvm_get_vcpu from -1 to -ENOENT
> - Reset the value of 'gdb_num_g_regs' in gdb_unregister_coprocessor_all
> - Fixed the kvm_{create,park}_vcpu prototypes docs
> - Added Reviewed-by tags for patches {2,3,4,5,6,7,9,10}
> 3. Addressed one earlier missed comment by Alex Bennée in RFC V1
> - Added traces instead of DPRINTF in the newly added and some existing functions
> Link:https://lore.kernel.org/qemu-devel/20230930001933.2660-1-salil.mehta@huawei.com/
>
> Patch-set V1 -> V2
> 1. Addressed Alex Bennée's comments
> - Refactored the kvm_create_vcpu logic to get rid of goto
> - Added the docs for kvm_{create,park}_vcpu prototypes
> - Splitted the gdbstub and AddressSpace destruction change into separate patches
> - Added Reviewed-by tags for patches {2,10}
> Link:https://lore.kernel.org/qemu-devel/20230929124304.13672-1-salil.mehta@huawei.com/
>
> References:
>
> [1]https://lore.kernel.org/qemu-devel/20230926100436.28284-1-salil.mehta@huawei.com/
> [2]https://lore.kernel.org/all/20230913163823.7880-1-james.morse@arm.com/
> [3]https://lore.kernel.org/qemu-devel/cover.1695697701.git.lixianglai@loongson.cn/
> [4]https://lore.kernel.org/qemu-devel/20240613233639.202896-2-salil.mehta@huawei.com/
>
> Salil Mehta (7):
> accel/kvm: Extract common KVM vCPU {creation,parking} code
> hw/acpi: Move CPU ctrl-dev MMIO region len macro to common header file
> hw/acpi: Update ACPI GED framework to support vCPU Hotplug
> hw/acpi: Update GED _EVT method AML with CPU scan
> hw/acpi: Update CPUs AML with cpu-(ctrl)dev change
> physmem: Add helper function to destroy CPU AddressSpace
> gdbstub: Add helper function to unregister GDB register space
>
> accel/kvm/kvm-all.c | 95 +++++++++++++++++---------
> accel/kvm/kvm-cpus.h | 1 -
> accel/kvm/trace-events | 5 +-
> docs/specs/acpi_hw_reduced_hotplug.rst | 3 +-
> gdbstub/gdbstub.c | 13 ++++
> hw/acpi/acpi-cpu-hotplug-stub.c | 6 ++
> hw/acpi/cpu.c | 18 +++--
> hw/acpi/generic_event_device.c | 27 ++++++++
> hw/core/cpu-common.c | 4 +-
> hw/i386/acpi-build.c | 3 +-
> include/exec/cpu-common.h | 8 +++
> include/exec/gdbstub.h | 6 ++
> include/hw/acpi/cpu.h | 7 +-
> include/hw/acpi/generic_event_device.h | 5 ++
> include/hw/core/cpu.h | 1 +
> include/sysemu/kvm.h | 25 +++++++
> system/physmem.c | 29 ++++++++
> 17 files changed, 212 insertions(+), 44 deletions(-)
>
[-- Attachment #2: Type: text/html, Size: 13217 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-15 11:03 ` Vishnu Pajjuri
@ 2024-07-15 11:07 ` Salil Mehta
0 siblings, 0 replies; 35+ messages in thread
From: Salil Mehta @ 2024-07-15 11:07 UTC (permalink / raw)
To: Vishnu Pajjuri
Cc: Salil Mehta, alex.bennee, andrew.jones, darren, david, eric.auger,
gshan, harshpb, ilkka, imammedo, jean-philippe, jiakernel2,
jonathan.cameron, karl.heubaum, linux, linuxarm, lixianglai,
lpieralisi, maobibo, maz, miguel.luis, mst, npiggin, oliver.upton,
pbonzini, peter.maydell, philmd, qemu-arm, qemu-devel, rafael,
richard.henderson, vishnu, wangxiongfeng2, wangyanan55, will,
zhukeqian1
[-- Attachment #1: Type: text/plain, Size: 10624 bytes --]
Hi Vishnu,
On Mon, 15 Jul 2024 at 12:04, Vishnu Pajjuri <
vishnu@amperemail.onmicrosoft.com> wrote:
> Hi Salil,
> On 13-07-2024 23:55, Salil Mehta wrote:
>
> [Note: References are present at the last after the revision history]
>
> Virtual CPU hotplug support is being added across various architectures [1][3].
> This series adds various code bits common across all architectures:
>
> 1. vCPU creation and Parking code refactor [Patch 1]
> 2. Update ACPI GED framework to support vCPU Hotplug [Patch 2,3]
> 3. ACPI CPUs AML code change [Patch 4,5]
> 4. Helper functions to support unrealization of CPU objects [Patch 6,7]
>
> Repository:
>
> [*] Architecture *Agnostic* Patch-set (This series)
> V14: https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v3.arch.agnostic.v15
>
> NOTE: This series is meant to work in conjunction with the architecture-specific
> patch-set. For ARM, a combined patch-set (architecture agnostic + specific) was
> earlier pushed as RFC V2 [1]. Later, RFC V2 was split into the ARM Architecture
> specific patch-set RFC V3 [4] (a subset of RFC V2) and the architecture agnostic
> patch-set. Patch-set V14 is the latest version in that series. This series
> works in conjunction with RFC V4-rc2, present at the following link.
>
> [*] ARM Architecture *Specific* Patch-set
> RFC V3 [4]: https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v3
> RFC V4-rc2: https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v4-rc2 (combined)
>
>
> Revision History:
>
> Patch-set V14 -> V15
> 1. Addressed commnet from Igor Mammedov's on [PATCH V14 4/7]
> - Removed ACPI_CPU_SCAN_METHOD
> - Introduced AML_GED_EVT_CPU_SCAN_METHOD ("\\_SB.GED.CPSCN") macro
> 2. Fix the stray change of "assert (" in "PATCH V14 3/7"
> Link: https://lore.kernel.org/qemu-devel/20240712134201.214699-4-salil.mehta@huawei.com/
>
> I tried following test cases with rfc-v4-rc2 and kernel patches v10, and
> it looks good on Ampere platforms.
>
> - Regular hotplug and hot unplug tests
> - Save/restore VM, suspend/resume VM with and with out hot-plugging
> vcpus tests
> - Live migration with and with out hot-plugging vcpus tests
>
> Please feel free to add,
> Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> <vishnu@os.amperecomputing.com>
>
Many thanks for confirming this.
Best wishes
Salil.
<vishnu@os.amperecomputing.com>
>
> *Regards*,
> -Vishnu.
>
> Patch-set V13 -> V14
> 1. Addressed Igor Mammedov's following review comments
> - Mentioned abput new external APIs in the header note of [PATCH 1/7]
> - Merged Doc [PATCH V13 8/8] with [PATCH V14 3/7]
> - Introduced GED realize function for various CPU Hotplug regions initializations
> - Added back event handler method to indirectly expose \\_SB.CPUS.CSCN to GED
> _EVT. Like for ARM, it would be through \\_SB.GED.CSCN event handler method
> - Collected the Ack given for [Patch V13 6/8]
> - Added back the gfree'ing of GDB regs in common finalize and made it conditional
> - Updated the header notes of [PATCH V13 3/8,4/8,5/8] to reflect the changes
>
> Patch-set V12 -> V13
> 1. Added Reviewed-by Tag of Harsh Prateek Bora's (IBM) [PATCH V12 1/8]
> 2. Moved the kvm_{create,park,unpark}_vcpu prototypes from accel/kvm/kvm-cpus.h
> to include/sysemu/kvm.h. These can later be exported through AccelOps.
> Link: https://lore.kernel.org/qemu-devel/62f55169-1796-4d8e-a35d-7f003a172750@linux.ibm.com/
>
> Patch-set V11 -> V12
> 1. Addressed Harsh Prateek Bora's (IBM) comment
> - Changed @cpu to @vcpu_id in the kvm_unpark_vcpu protoype header/
> 2. Added Zhao Liu's (Intel) Tested-by for whole series
> - Qtest does not breaks on Intel platforms now.
> 3. Added Zhao Liu's (Intel) Reviewed-by for [PATCH V11 {1/8 - 3/8}]
> Link: https://lore.kernel.org/qemu-devel/ZlRSPuJGBgyEUW6w@intel.com/
> Link: https://lore.kernel.org/qemu-devel/a5f3d78e-cfed-441f-9c56-e3e78fa5edee@linux.ibm.com/
>
> Patch-set V10 -> V11
> 1. Addressed Nicholas Piggin's (IBM) comment
> - moved the traces in kvm_unpark_vcpu and kvm_create_vcpu at the end
> - Added the Reviewed-by Tag for [PATCH V10 1/8]
> 2. Addressed Alex Bennée's (Linaro) comments
> - Added a note explaining dependency of the [PATCH V10 7/8] on Arch specific patch-set
> Link: https://lore.kernel.org/qemu-devel/D1FS5GOOFWWK.2PNRIVL0V6DBL@gmail.com/
> Link: https://lore.kernel.org/qemu-devel/87frubi402.fsf@draig.linaro.org/
>
> Patch-set V9 -> V10
> 1. Addressed Nicholas Piggin's (IBM) & Philippe Mathieu-Daudé (Linaro) comments
> - carved out kvm_unpark_vcpu and added its trace
> - Widened the scope of the kvm_unpark_vcpu so that it can be used by generic framework
> being thought out
> Link: https://lore.kernel.org/qemu-devel/20240519210620.228342-1-salil.mehta@huawei.com/
> Link: https://lore.kernel.org/qemu-devel/e94b0e14-efee-4050-9c9f-08382a36b63a@linaro.org/
>
> Patch-set V8 -> V9
> 1. Addressed Vishnu Pajjuri's (Ampere) comments
> - Added kvm_fd to trace in kvm_create_vcpu
> - Some clean ups: arch vcpu-id and sbd variable
> - Added the missed initialization of cpu->gdb_num_regs
> 2. Addressed the commnet from Zhao Liu (Intel)
> - Make initialization of CPU Hotplug state conditional (possible_cpu_arch_ids!=NULL)
> Link: https://lore.kernel.org/qemu-devel/20240312020000.12992-1-salil.mehta@huawei.com/
>
> Patch-set V7 -> V8
> 1. Rebased and Fixed the conflicts
>
> Patch-set V6 -> V7
> 1. Addressed Alex Bennée's comments
> - Updated the docs
> 2. Addressed Igor Mammedov's comments
> - Merged patches [Patch V6 3/9] & [Patch V6 7/9] with [Patch V6 4/9]
> - Updated commit-log of [Patch V6 1/9] and [Patch V6 5/9]
> 3. Added Shaoqin Huang's Reviewed-by tags for whole series.
> Link: https://lore.kernel.org/qemu-devel/20231013105129.25648-1-salil.mehta@huawei.com/
>
> Patch-set V5 -> V6
> 1. Addressed Gavin Shan's comments
> - Fixed the assert() ranges of address spaces
> - Rebased the patch-set to latest changes in the qemu.git
> - Added Reviewed-by tags for patches {8,9}
> 2. Addressed Jonathan Cameron's comments
> - Updated commit-log for [Patch V5 1/9] with mention of trace events
> - Added Reviewed-by tags for patches {1,5}
> 3. Added Tested-by tags from Xianglai Li
> 4. Fixed checkpatch.pl error "Qemu -> QEMU" in [Patch V5 1/9]
> Link: https://lore.kernel.org/qemu-devel/20231011194355.15628-1-salil.mehta@huawei.com/
>
> Patch-set V4 -> V5
> 1. Addressed Gavin Shan's comments
> - Fixed the trace events print string for kvm_{create,get,park,destroy}_vcpu
> - Added Reviewed-by tag for patch {1}
> 2. Added Shaoqin Huang's Reviewed-by tags for Patches {2,3}
> 3. Added Tested-by Tag from Vishnu Pajjuri to the patch-set
> 4. Dropped the ARM specific [Patch V4 10/10]
> Link: https://lore.kernel.org/qemu-devel/20231009203601.17584-1-salil.mehta@huawei.com/
>
> Patch-set V3 -> V4
> 1. Addressed David Hilderbrand's comments
> - Fixed the wrong doc comment of kvm_park_vcpu API prototype
> - Added Reviewed-by tags for patches {2,4}
> Link: https://lore.kernel.org/qemu-devel/20231009112812.10612-1-salil.mehta@huawei.com/
>
> Patch-set V2 -> V3
> 1. Addressed Jonathan Cameron's comments
> - Fixed 'vcpu-id' type wrongly changed from 'unsigned long' to 'integer'
> - Removed unnecessary use of variable 'vcpu_id' in kvm_park_vcpu
> - Updated [Patch V2 3/10] commit-log with details of ACPI_CPU_SCAN_METHOD macro
> - Updated [Patch V2 5/10] commit-log with details of conditional event handler method
> - Added Reviewed-by tags for patches {2,3,4,6,7}
> 2. Addressed Gavin Shan's comments
> - Remove unnecessary use of variable 'vcpu_id' in kvm_par_vcpu
> - Fixed return value in kvm_get_vcpu from -1 to -ENOENT
> - Reset the value of 'gdb_num_g_regs' in gdb_unregister_coprocessor_all
> - Fixed the kvm_{create,park}_vcpu prototypes docs
> - Added Reviewed-by tags for patches {2,3,4,5,6,7,9,10}
> 3. Addressed one earlier missed comment by Alex Bennée in RFC V1
> - Added traces instead of DPRINTF in the newly added and some existing functions
> Link: https://lore.kernel.org/qemu-devel/20230930001933.2660-1-salil.mehta@huawei.com/
>
> Patch-set V1 -> V2
> 1. Addressed Alex Bennée's comments
> - Refactored the kvm_create_vcpu logic to get rid of goto
> - Added the docs for kvm_{create,park}_vcpu prototypes
> - Splitted the gdbstub and AddressSpace destruction change into separate patches
> - Added Reviewed-by tags for patches {2,10}
> Link: https://lore.kernel.org/qemu-devel/20230929124304.13672-1-salil.mehta@huawei.com/
>
> References:
>
> [1] https://lore.kernel.org/qemu-devel/20230926100436.28284-1-salil.mehta@huawei.com/
> [2] https://lore.kernel.org/all/20230913163823.7880-1-james.morse@arm.com/
> [3] https://lore.kernel.org/qemu-devel/cover.1695697701.git.lixianglai@loongson.cn/
> [4] https://lore.kernel.org/qemu-devel/20240613233639.202896-2-salil.mehta@huawei.com/
>
> Salil Mehta (7):
> accel/kvm: Extract common KVM vCPU {creation,parking} code
> hw/acpi: Move CPU ctrl-dev MMIO region len macro to common header file
> hw/acpi: Update ACPI GED framework to support vCPU Hotplug
> hw/acpi: Update GED _EVT method AML with CPU scan
> hw/acpi: Update CPUs AML with cpu-(ctrl)dev change
> physmem: Add helper function to destroy CPU AddressSpace
> gdbstub: Add helper function to unregister GDB register space
>
> accel/kvm/kvm-all.c | 95 +++++++++++++++++---------
> accel/kvm/kvm-cpus.h | 1 -
> accel/kvm/trace-events | 5 +-
> docs/specs/acpi_hw_reduced_hotplug.rst | 3 +-
> gdbstub/gdbstub.c | 13 ++++
> hw/acpi/acpi-cpu-hotplug-stub.c | 6 ++
> hw/acpi/cpu.c | 18 +++--
> hw/acpi/generic_event_device.c | 27 ++++++++
> hw/core/cpu-common.c | 4 +-
> hw/i386/acpi-build.c | 3 +-
> include/exec/cpu-common.h | 8 +++
> include/exec/gdbstub.h | 6 ++
> include/hw/acpi/cpu.h | 7 +-
> include/hw/acpi/generic_event_device.h | 5 ++
> include/hw/core/cpu.h | 1 +
> include/sysemu/kvm.h | 25 +++++++
> system/physmem.c | 29 ++++++++
> 17 files changed, 212 insertions(+), 44 deletions(-)
>
>
>
[-- Attachment #2: Type: text/html, Size: 13844 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-13 18:25 [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Salil Mehta via
` (8 preceding siblings ...)
2024-07-15 11:03 ` Vishnu Pajjuri
@ 2024-07-15 11:13 ` Michael S. Tsirkin
2024-07-15 11:27 ` Salil Mehta via
2024-07-15 11:14 ` Salil Mehta via
2024-07-15 13:54 ` Igor Mammedov
11 siblings, 1 reply; 35+ messages in thread
From: Michael S. Tsirkin @ 2024-07-15 11:13 UTC (permalink / raw)
To: Salil Mehta
Cc: qemu-devel, qemu-arm, maz, jean-philippe, jonathan.cameron,
lpieralisi, peter.maydell, richard.henderson, imammedo,
andrew.jones, david, philmd, eric.auger, oliver.upton, pbonzini,
will, gshan, rafael, alex.bennee, linux, darren, ilkka, vishnu,
karl.heubaum, miguel.luis, salil.mehta, zhukeqian1,
wangxiongfeng2, wangyanan55, jiakernel2, maobibo, lixianglai,
npiggin, harshpb, linuxarm
On Sat, Jul 13, 2024 at 07:25:09PM +0100, Salil Mehta wrote:
> [Note: References are present at the last after the revision history]
Igor any comments before I merge this?
--
MST
^ permalink raw reply [flat|nested] 35+ messages in thread
* RE: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-13 18:25 [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Salil Mehta via
` (9 preceding siblings ...)
2024-07-15 11:13 ` Michael S. Tsirkin
@ 2024-07-15 11:14 ` Salil Mehta via
2024-07-15 13:54 ` Igor Mammedov
11 siblings, 0 replies; 35+ messages in thread
From: Salil Mehta via @ 2024-07-15 11:14 UTC (permalink / raw)
To: qemu-devel@nongnu.org, qemu-arm@nongnu.org
Cc: maz@kernel.org, jean-philippe@linaro.org, Jonathan Cameron,
lpieralisi@kernel.org, peter.maydell@linaro.org,
richard.henderson@linaro.org, imammedo@redhat.com,
andrew.jones@linux.dev, david@redhat.com, philmd@linaro.org,
eric.auger@redhat.com, oliver.upton@linux.dev,
pbonzini@redhat.com, mst@redhat.com, will@kernel.org,
gshan@redhat.com, rafael@kernel.org, alex.bennee@linaro.org,
linux@armlinux.org.uk, darren@os.amperecomputing.com,
ilkka@os.amperecomputing.com, vishnu@os.amperecomputing.com,
karl.heubaum@oracle.com, miguel.luis@oracle.com,
salil.mehta@opnsrc.net, zhukeqian, wangxiongfeng (C),
wangyanan (Y), jiakernel2@gmail.com, maobibo@loongson.cn,
lixianglai@loongson.cn, npiggin@gmail.com, harshpb@linux.ibm.com,
Linuxarm
Hi Igor,
We are approaching end of this Qemu cycle, I believe that’s on 17th July. If you are
satisfied with the changes. May I request your Reviewed/Acked-Bys for this series?
This series is vouched by many companies. It will be good it to merge it in this cycle.
Best regards
Salil.
> From: Salil Mehta <salil.mehta@huawei.com>
> Sent: Saturday, July 13, 2024 7:25 PM
> To: qemu-devel@nongnu.org; qemu-arm@nongnu.org
>
> [Note: References are present at the last after the revision history]
>
> Virtual CPU hotplug support is being added across various architectures
> [1][3].
> This series adds various code bits common across all architectures:
>
> 1. vCPU creation and Parking code refactor [Patch 1] 2. Update ACPI GED
> framework to support vCPU Hotplug [Patch 2,3] 3. ACPI CPUs AML code
> change [Patch 4,5] 4. Helper functions to support unrealization of CPU
> objects [Patch 6,7]
>
> Repository:
>
> [*] Architecture *Agnostic* Patch-set (This series)
> V14: https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-
> v3.arch.agnostic.v15
>
> NOTE: This series is meant to work in conjunction with the architecture-
> specific
> patch-set. For ARM, a combined patch-set (architecture agnostic +
> specific) was
> earlier pushed as RFC V2 [1]. Later, RFC V2 was split into the ARM
> Architecture
> specific patch-set RFC V3 [4] (a subset of RFC V2) and the architecture
> agnostic
> patch-set. Patch-set V14 is the latest version in that series. This series
> works in conjunction with RFC V4-rc2, present at the following link.
>
> [*] ARM Architecture *Specific* Patch-set
> RFC V3 [4]: https://github.com/salil-mehta/qemu.git virt-cpuhp-
> armv8/rfc-v3
> RFC V4-rc2: https://github.com/salil-mehta/qemu.git virt-cpuhp-
> armv8/rfc-v4-rc2 (combined)
>
>
> Revision History:
>
> Patch-set V14 -> V15
> 1. Addressed commnet from Igor Mammedov's on [PATCH V14 4/7]
> - Removed ACPI_CPU_SCAN_METHOD
> - Introduced AML_GED_EVT_CPU_SCAN_METHOD ("\\_SB.GED.CPSCN")
> macro 2. Fix the stray change of "assert (" in "PATCH V14 3/7"
> Link: https://lore.kernel.org/qemu-devel/20240712134201.214699-4-
> salil.mehta@huawei.com/
>
> Patch-set V13 -> V14
> 1. Addressed Igor Mammedov's following review comments
> - Mentioned abput new external APIs in the header note of [PATCH 1/7]
> - Merged Doc [PATCH V13 8/8] with [PATCH V14 3/7]
> - Introduced GED realize function for various CPU Hotplug regions
> initializations
> - Added back event handler method to indirectly expose
> \\_SB.CPUS.CSCN to GED
> _EVT. Like for ARM, it would be through \\_SB.GED.CSCN event handler
> method
> - Collected the Ack given for [Patch V13 6/8]
> - Added back the gfree'ing of GDB regs in common finalize and made it
> conditional
> - Updated the header notes of [PATCH V13 3/8,4/8,5/8] to reflect the
> changes
>
> Patch-set V12 -> V13
> 1. Added Reviewed-by Tag of Harsh Prateek Bora's (IBM) [PATCH V12 1/8] 2.
> Moved the kvm_{create,park,unpark}_vcpu prototypes from
> accel/kvm/kvm-cpus.h
> to include/sysemu/kvm.h. These can later be exported through AccelOps.
> Link: https://lore.kernel.org/qemu-devel/62f55169-1796-4d8e-a35d-
> 7f003a172750@linux.ibm.com/
>
> Patch-set V11 -> V12
> 1. Addressed Harsh Prateek Bora's (IBM) comment
> - Changed @cpu to @vcpu_id in the kvm_unpark_vcpu protoype header/
> 2. Added Zhao Liu's (Intel) Tested-by for whole series
> - Qtest does not breaks on Intel platforms now.
> 3. Added Zhao Liu's (Intel) Reviewed-by for [PATCH V11 {1/8 - 3/8}]
> Link: https://lore.kernel.org/qemu-devel/ZlRSPuJGBgyEUW6w@intel.com/
> Link: https://lore.kernel.org/qemu-devel/a5f3d78e-cfed-441f-9c56-
> e3e78fa5edee@linux.ibm.com/
>
> Patch-set V10 -> V11
> 1. Addressed Nicholas Piggin's (IBM) comment
> - moved the traces in kvm_unpark_vcpu and kvm_create_vcpu at the end
> - Added the Reviewed-by Tag for [PATCH V10 1/8] 2. Addressed Alex
> Bennée's (Linaro) comments
> - Added a note explaining dependency of the [PATCH V10 7/8] on Arch
> specific patch-set
> Link: https://lore.kernel.org/qemu-
> devel/D1FS5GOOFWWK.2PNRIVL0V6DBL@gmail.com/
> Link: https://lore.kernel.org/qemu-devel/87frubi402.fsf@draig.linaro.org/
>
> Patch-set V9 -> V10
> 1. Addressed Nicholas Piggin's (IBM) & Philippe Mathieu-Daudé (Linaro)
> comments
> - carved out kvm_unpark_vcpu and added its trace
> - Widened the scope of the kvm_unpark_vcpu so that it can be used by
> generic framework
> being thought out
> Link: https://lore.kernel.org/qemu-devel/20240519210620.228342-1-
> salil.mehta@huawei.com/
> Link: https://lore.kernel.org/qemu-devel/e94b0e14-efee-4050-9c9f-
> 08382a36b63a@linaro.org/
>
> Patch-set V8 -> V9
> 1. Addressed Vishnu Pajjuri's (Ampere) comments
> - Added kvm_fd to trace in kvm_create_vcpu
> - Some clean ups: arch vcpu-id and sbd variable
> - Added the missed initialization of cpu->gdb_num_regs 2. Addressed the
> commnet from Zhao Liu (Intel)
> - Make initialization of CPU Hotplug state conditional
> (possible_cpu_arch_ids!=NULL)
> Link: https://lore.kernel.org/qemu-devel/20240312020000.12992-1-
> salil.mehta@huawei.com/
>
> Patch-set V7 -> V8
> 1. Rebased and Fixed the conflicts
>
> Patch-set V6 -> V7
> 1. Addressed Alex Bennée's comments
> - Updated the docs
> 2. Addressed Igor Mammedov's comments
> - Merged patches [Patch V6 3/9] & [Patch V6 7/9] with [Patch V6 4/9]
> - Updated commit-log of [Patch V6 1/9] and [Patch V6 5/9]
> 3. Added Shaoqin Huang's Reviewed-by tags for whole series.
> Link: https://lore.kernel.org/qemu-devel/20231013105129.25648-1-
> salil.mehta@huawei.com/
>
> Patch-set V5 -> V6
> 1. Addressed Gavin Shan's comments
> - Fixed the assert() ranges of address spaces
> - Rebased the patch-set to latest changes in the qemu.git
> - Added Reviewed-by tags for patches {8,9} 2. Addressed Jonathan
> Cameron's comments
> - Updated commit-log for [Patch V5 1/9] with mention of trace events
> - Added Reviewed-by tags for patches {1,5} 3. Added Tested-by tags from
> Xianglai Li 4. Fixed checkpatch.pl error "Qemu -> QEMU" in [Patch V5 1/9]
> Link: https://lore.kernel.org/qemu-devel/20231011194355.15628-1-
> salil.mehta@huawei.com/
>
> Patch-set V4 -> V5
> 1. Addressed Gavin Shan's comments
> - Fixed the trace events print string for
> kvm_{create,get,park,destroy}_vcpu
> - Added Reviewed-by tag for patch {1} 2. Added Shaoqin Huang's
> Reviewed-by tags for Patches {2,3} 3. Added Tested-by Tag from Vishnu
> Pajjuri to the patch-set 4. Dropped the ARM specific [Patch V4 10/10]
> Link: https://lore.kernel.org/qemu-devel/20231009203601.17584-1-
> salil.mehta@huawei.com/
>
> Patch-set V3 -> V4
> 1. Addressed David Hilderbrand's comments
> - Fixed the wrong doc comment of kvm_park_vcpu API prototype
> - Added Reviewed-by tags for patches {2,4}
> Link: https://lore.kernel.org/qemu-devel/20231009112812.10612-1-
> salil.mehta@huawei.com/
>
> Patch-set V2 -> V3
> 1. Addressed Jonathan Cameron's comments
> - Fixed 'vcpu-id' type wrongly changed from 'unsigned long' to 'integer'
> - Removed unnecessary use of variable 'vcpu_id' in kvm_park_vcpu
> - Updated [Patch V2 3/10] commit-log with details of
> ACPI_CPU_SCAN_METHOD macro
> - Updated [Patch V2 5/10] commit-log with details of conditional event
> handler method
> - Added Reviewed-by tags for patches {2,3,4,6,7} 2. Addressed Gavin
> Shan's comments
> - Remove unnecessary use of variable 'vcpu_id' in kvm_par_vcpu
> - Fixed return value in kvm_get_vcpu from -1 to -ENOENT
> - Reset the value of 'gdb_num_g_regs' in gdb_unregister_coprocessor_all
> - Fixed the kvm_{create,park}_vcpu prototypes docs
> - Added Reviewed-by tags for patches {2,3,4,5,6,7,9,10} 3. Addressed one
> earlier missed comment by Alex Bennée in RFC V1
> - Added traces instead of DPRINTF in the newly added and some existing
> functions
> Link: https://lore.kernel.org/qemu-devel/20230930001933.2660-1-
> salil.mehta@huawei.com/
>
> Patch-set V1 -> V2
> 1. Addressed Alex Bennée's comments
> - Refactored the kvm_create_vcpu logic to get rid of goto
> - Added the docs for kvm_{create,park}_vcpu prototypes
> - Splitted the gdbstub and AddressSpace destruction change into separate
> patches
> - Added Reviewed-by tags for patches {2,10}
> Link: https://lore.kernel.org/qemu-devel/20230929124304.13672-1-
> salil.mehta@huawei.com/
>
> References:
>
> [1] https://lore.kernel.org/qemu-devel/20230926100436.28284-1-
> salil.mehta@huawei.com/
> [2] https://lore.kernel.org/all/20230913163823.7880-1-
> james.morse@arm.com/
> [3] https://lore.kernel.org/qemu-
> devel/cover.1695697701.git.lixianglai@loongson.cn/
> [4] https://lore.kernel.org/qemu-devel/20240613233639.202896-2-
> salil.mehta@huawei.com/
>
> Salil Mehta (7):
> accel/kvm: Extract common KVM vCPU {creation,parking} code
> hw/acpi: Move CPU ctrl-dev MMIO region len macro to common header
> file
> hw/acpi: Update ACPI GED framework to support vCPU Hotplug
> hw/acpi: Update GED _EVT method AML with CPU scan
> hw/acpi: Update CPUs AML with cpu-(ctrl)dev change
> physmem: Add helper function to destroy CPU AddressSpace
> gdbstub: Add helper function to unregister GDB register space
>
> accel/kvm/kvm-all.c | 95 +++++++++++++++++---------
> accel/kvm/kvm-cpus.h | 1 -
> accel/kvm/trace-events | 5 +-
> docs/specs/acpi_hw_reduced_hotplug.rst | 3 +-
> gdbstub/gdbstub.c | 13 ++++
> hw/acpi/acpi-cpu-hotplug-stub.c | 6 ++
> hw/acpi/cpu.c | 18 +++--
> hw/acpi/generic_event_device.c | 27 ++++++++
> hw/core/cpu-common.c | 4 +-
> hw/i386/acpi-build.c | 3 +-
> include/exec/cpu-common.h | 8 +++
> include/exec/gdbstub.h | 6 ++
> include/hw/acpi/cpu.h | 7 +-
> include/hw/acpi/generic_event_device.h | 5 ++
> include/hw/core/cpu.h | 1 +
> include/sysemu/kvm.h | 25 +++++++
> system/physmem.c | 29 ++++++++
> 17 files changed, 212 insertions(+), 44 deletions(-)
>
> --
> 2.34.1
^ permalink raw reply [flat|nested] 35+ messages in thread
* RE: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-15 11:13 ` Michael S. Tsirkin
@ 2024-07-15 11:27 ` Salil Mehta via
2024-07-15 11:33 ` Michael S. Tsirkin
2024-07-15 13:55 ` Igor Mammedov
0 siblings, 2 replies; 35+ messages in thread
From: Salil Mehta via @ 2024-07-15 11:27 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: qemu-devel@nongnu.org, qemu-arm@nongnu.org, maz@kernel.org,
jean-philippe@linaro.org, Jonathan Cameron, lpieralisi@kernel.org,
peter.maydell@linaro.org, richard.henderson@linaro.org,
imammedo@redhat.com, andrew.jones@linux.dev, david@redhat.com,
philmd@linaro.org, eric.auger@redhat.com, oliver.upton@linux.dev,
pbonzini@redhat.com, will@kernel.org, gshan@redhat.com,
rafael@kernel.org, alex.bennee@linaro.org, linux@armlinux.org.uk,
darren@os.amperecomputing.com, ilkka@os.amperecomputing.com,
vishnu@os.amperecomputing.com, karl.heubaum@oracle.com,
miguel.luis@oracle.com, salil.mehta@opnsrc.net, zhukeqian,
wangxiongfeng (C), wangyanan (Y), jiakernel2@gmail.com,
maobibo@loongson.cn, lixianglai@loongson.cn, npiggin@gmail.com,
harshpb@linux.ibm.com, Linuxarm
Hi Michael,
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, July 15, 2024 12:13 PM
> To: Salil Mehta <salil.mehta@huawei.com>
>
> On Sat, Jul 13, 2024 at 07:25:09PM +0100, Salil Mehta wrote:
> > [Note: References are present at the last after the revision history]
>
> Igor any comments before I merge this?
Hi Michael,
Assuming there are no last-minute surprises and If you decide to merge this
series, could I kindly request that you collect all the Tags (XXX-Bys) including
the Igor's pending Reviewed/Acked-By Tag for the entire series, so that I won't
have to churn out another version (V16)?
Many thanks!
Best regards
Salil
>
> --
> MST
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-15 11:27 ` Salil Mehta via
@ 2024-07-15 11:33 ` Michael S. Tsirkin
2024-07-15 11:35 ` Salil Mehta via
2024-07-15 13:55 ` Igor Mammedov
1 sibling, 1 reply; 35+ messages in thread
From: Michael S. Tsirkin @ 2024-07-15 11:33 UTC (permalink / raw)
To: Salil Mehta
Cc: qemu-devel@nongnu.org, qemu-arm@nongnu.org, maz@kernel.org,
jean-philippe@linaro.org, Jonathan Cameron, lpieralisi@kernel.org,
peter.maydell@linaro.org, richard.henderson@linaro.org,
imammedo@redhat.com, andrew.jones@linux.dev, david@redhat.com,
philmd@linaro.org, eric.auger@redhat.com, oliver.upton@linux.dev,
pbonzini@redhat.com, will@kernel.org, gshan@redhat.com,
rafael@kernel.org, alex.bennee@linaro.org, linux@armlinux.org.uk,
darren@os.amperecomputing.com, ilkka@os.amperecomputing.com,
vishnu@os.amperecomputing.com, karl.heubaum@oracle.com,
miguel.luis@oracle.com, salil.mehta@opnsrc.net, zhukeqian,
wangxiongfeng (C), wangyanan (Y), jiakernel2@gmail.com,
maobibo@loongson.cn, lixianglai@loongson.cn, npiggin@gmail.com,
harshpb@linux.ibm.com, Linuxarm
On Mon, Jul 15, 2024 at 11:27:57AM +0000, Salil Mehta wrote:
> Hi Michael,
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, July 15, 2024 12:13 PM
> > To: Salil Mehta <salil.mehta@huawei.com>
> >
> > On Sat, Jul 13, 2024 at 07:25:09PM +0100, Salil Mehta wrote:
> > > [Note: References are present at the last after the revision history]
> >
> > Igor any comments before I merge this?
>
> Hi Michael,
>
> Assuming there are no last-minute surprises and If you decide to merge this
> series, could I kindly request that you collect all the Tags (XXX-Bys) including
> the Igor's pending Reviewed/Acked-By Tag for the entire series, so that I won't
> have to churn out another version (V16)?
>
> Many thanks!
>
> Best regards
> Salil
Yes, there's no need to resend just to add acks, I collect them
automatically.
>
> >
> > --
> > MST
> >
^ permalink raw reply [flat|nested] 35+ messages in thread
* RE: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-15 11:33 ` Michael S. Tsirkin
@ 2024-07-15 11:35 ` Salil Mehta via
0 siblings, 0 replies; 35+ messages in thread
From: Salil Mehta via @ 2024-07-15 11:35 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: qemu-devel@nongnu.org, qemu-arm@nongnu.org, maz@kernel.org,
jean-philippe@linaro.org, Jonathan Cameron, lpieralisi@kernel.org,
peter.maydell@linaro.org, richard.henderson@linaro.org,
imammedo@redhat.com, andrew.jones@linux.dev, david@redhat.com,
philmd@linaro.org, eric.auger@redhat.com, oliver.upton@linux.dev,
pbonzini@redhat.com, will@kernel.org, gshan@redhat.com,
rafael@kernel.org, alex.bennee@linaro.org, linux@armlinux.org.uk,
darren@os.amperecomputing.com, ilkka@os.amperecomputing.com,
vishnu@os.amperecomputing.com, karl.heubaum@oracle.com,
miguel.luis@oracle.com, salil.mehta@opnsrc.net, zhukeqian,
wangxiongfeng (C), wangyanan (Y), jiakernel2@gmail.com,
maobibo@loongson.cn, lixianglai@loongson.cn, npiggin@gmail.com,
harshpb@linux.ibm.com, Linuxarm
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, July 15, 2024 12:33 PM
> To: Salil Mehta <salil.mehta@huawei.com>
>
> On Mon, Jul 15, 2024 at 11:27:57AM +0000, Salil Mehta wrote:
> > Hi Michael,
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Monday, July 15, 2024 12:13 PM
> > > To: Salil Mehta <salil.mehta@huawei.com>
> > >
> > > On Sat, Jul 13, 2024 at 07:25:09PM +0100, Salil Mehta wrote:
> > > > [Note: References are present at the last after the revision
> > > history]
> > >
> > > Igor any comments before I merge this?
> >
> > Hi Michael,
> >
> > Assuming there are no last-minute surprises and If you decide to merge
> > this series, could I kindly request that you collect all the Tags
> > (XXX-Bys) including the Igor's pending Reviewed/Acked-By Tag for the
> > entire series, so that I won't have to churn out another version (V16)?
> >
> > Many thanks!
> >
> > Best regards
> > Salil
>
>
> Yes, there's no need to resend just to add acks, I collect them automatically.
Brilliant. Many thanks!
Best regards
Salil.
> > > --
> > > MST
> > >
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 1/7] accel/kvm: Extract common KVM vCPU {creation,parking} code
2024-07-13 18:25 ` [PATCH V15 1/7] accel/kvm: Extract common KVM vCPU {creation, parking} code Salil Mehta via
@ 2024-07-15 12:49 ` Igor Mammedov
2024-07-15 13:28 ` Igor Mammedov
0 siblings, 1 reply; 35+ messages in thread
From: Igor Mammedov @ 2024-07-15 12:49 UTC (permalink / raw)
To: Salil Mehta
Cc: qemu-devel, qemu-arm, maz, jean-philippe, jonathan.cameron,
lpieralisi, peter.maydell, richard.henderson, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm,
Shaoqin Huang, Zhao Liu
On Sat, 13 Jul 2024 19:25:10 +0100
Salil Mehta <salil.mehta@huawei.com> wrote:
> KVM vCPU creation is done once during the vCPU realization when Qemu vCPU thread
> is spawned. This is common to all the architectures as of now.
>
> Hot-unplug of vCPU results in destruction of the vCPU object in QOM but the
> corresponding KVM vCPU object in the Host KVM is not destroyed as KVM doesn't
> support vCPU removal. Therefore, its representative KVM vCPU object/context in
> Qemu is parked.
>
> Refactor architecture common logic so that some APIs could be reused by vCPU
> Hotplug code of some architectures likes ARM, Loongson etc. Update new/old APIs
> with trace events. New APIs qemu_{create,park,unpark}_vcpu() can be externally
> called. No functional change is intended here.
>
> Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
> Reviewed-by: Gavin Shan <gshan@redhat.com>
> Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> Tested-by: Xianglai Li <lixianglai@loongson.cn>
> Tested-by: Miguel Luis <miguel.luis@oracle.com>
> Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
> Reviewed-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
> Tested-by: Zhao Liu <zhao1.liu@intel.com>
> Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
> Reviewed-by: Harsh Prateek Bora <harshpb@linux.ibm.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
> ---
> accel/kvm/kvm-all.c | 95 ++++++++++++++++++++++++++++--------------
> accel/kvm/kvm-cpus.h | 1 -
> accel/kvm/trace-events | 5 ++-
> include/sysemu/kvm.h | 25 +++++++++++
> 4 files changed, 92 insertions(+), 34 deletions(-)
>
> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
> index 2b4ab89679..e446d18944 100644
> --- a/accel/kvm/kvm-all.c
> +++ b/accel/kvm/kvm-all.c
> @@ -340,14 +340,71 @@ err:
> return ret;
> }
>
> +void kvm_park_vcpu(CPUState *cpu)
> +{
> + struct KVMParkedVcpu *vcpu;
> +
> + trace_kvm_park_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu));
> +
> + vcpu = g_malloc0(sizeof(*vcpu));
> + vcpu->vcpu_id = kvm_arch_vcpu_id(cpu);
> + vcpu->kvm_fd = cpu->kvm_fd;
> + QLIST_INSERT_HEAD(&kvm_state->kvm_parked_vcpus, vcpu, node);
> +}
> +
> +int kvm_unpark_vcpu(KVMState *s, unsigned long vcpu_id)
> +{
> + struct KVMParkedVcpu *cpu;
> + int kvm_fd = -ENOENT;
> +
> + QLIST_FOREACH(cpu, &s->kvm_parked_vcpus, node) {
> + if (cpu->vcpu_id == vcpu_id) {
> + QLIST_REMOVE(cpu, node);
> + kvm_fd = cpu->kvm_fd;
> + g_free(cpu);
> + }
> + }
> +
> + trace_kvm_unpark_vcpu(vcpu_id, kvm_fd > 0 ? "unparked" : "not found parked");
> +
> + return kvm_fd;
> +}
> +
> +int kvm_create_vcpu(CPUState *cpu)
> +{
> + unsigned long vcpu_id = kvm_arch_vcpu_id(cpu);
> + KVMState *s = kvm_state;
> + int kvm_fd;
> +
> + /* check if the KVM vCPU already exist but is parked */
> + kvm_fd = kvm_unpark_vcpu(s, vcpu_id);
> + if (kvm_fd < 0) {
> + /* vCPU not parked: create a new KVM vCPU */
> + kvm_fd = kvm_vm_ioctl(s, KVM_CREATE_VCPU, vcpu_id);
> + if (kvm_fd < 0) {
> + error_report("KVM_CREATE_VCPU IOCTL failed for vCPU %lu", vcpu_id);
> + return kvm_fd;
> + }
> + }
> +
> + cpu->kvm_fd = kvm_fd;
> + cpu->kvm_state = s;
> + cpu->vcpu_dirty = true;
> + cpu->dirty_pages = 0;
> + cpu->throttle_us_per_full = 0;
> +
> + trace_kvm_create_vcpu(cpu->cpu_index, vcpu_id, kvm_fd);
> +
> + return 0;
> +}
> +
> static int do_kvm_destroy_vcpu(CPUState *cpu)
> {
> KVMState *s = kvm_state;
> long mmap_size;
> - struct KVMParkedVcpu *vcpu = NULL;
> int ret = 0;
>
> - trace_kvm_destroy_vcpu();
> + trace_kvm_destroy_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu));
>
> ret = kvm_arch_destroy_vcpu(cpu);
> if (ret < 0) {
> @@ -373,10 +430,7 @@ static int do_kvm_destroy_vcpu(CPUState *cpu)
> }
> }
>
> - vcpu = g_malloc0(sizeof(*vcpu));
> - vcpu->vcpu_id = kvm_arch_vcpu_id(cpu);
> - vcpu->kvm_fd = cpu->kvm_fd;
> - QLIST_INSERT_HEAD(&kvm_state->kvm_parked_vcpus, vcpu, node);
> + kvm_park_vcpu(cpu);
> err:
> return ret;
> }
> @@ -389,24 +443,6 @@ void kvm_destroy_vcpu(CPUState *cpu)
> }
> }
>
> -static int kvm_get_vcpu(KVMState *s, unsigned long vcpu_id)
> -{
> - struct KVMParkedVcpu *cpu;
> -
> - QLIST_FOREACH(cpu, &s->kvm_parked_vcpus, node) {
> - if (cpu->vcpu_id == vcpu_id) {
> - int kvm_fd;
> -
> - QLIST_REMOVE(cpu, node);
> - kvm_fd = cpu->kvm_fd;
> - g_free(cpu);
> - return kvm_fd;
> - }
> - }
> -
> - return kvm_vm_ioctl(s, KVM_CREATE_VCPU, (void *)vcpu_id);
> -}
> -
> int kvm_init_vcpu(CPUState *cpu, Error **errp)
> {
> KVMState *s = kvm_state;
> @@ -415,19 +451,14 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
>
> trace_kvm_init_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu));
>
> - ret = kvm_get_vcpu(s, kvm_arch_vcpu_id(cpu));
> + ret = kvm_create_vcpu(cpu);
> if (ret < 0) {
> - error_setg_errno(errp, -ret, "kvm_init_vcpu: kvm_get_vcpu failed (%lu)",
> + error_setg_errno(errp, -ret,
> + "kvm_init_vcpu: kvm_create_vcpu failed (%lu)",
> kvm_arch_vcpu_id(cpu));
> goto err;
> }
>
> - cpu->kvm_fd = ret;
> - cpu->kvm_state = s;
> - cpu->vcpu_dirty = true;
> - cpu->dirty_pages = 0;
> - cpu->throttle_us_per_full = 0;
> -
> mmap_size = kvm_ioctl(s, KVM_GET_VCPU_MMAP_SIZE, 0);
> if (mmap_size < 0) {
> ret = mmap_size;
> diff --git a/accel/kvm/kvm-cpus.h b/accel/kvm/kvm-cpus.h
> index ca40add32c..171b22fd29 100644
> --- a/accel/kvm/kvm-cpus.h
> +++ b/accel/kvm/kvm-cpus.h
> @@ -22,5 +22,4 @@ bool kvm_supports_guest_debug(void);
> int kvm_insert_breakpoint(CPUState *cpu, int type, vaddr addr, vaddr len);
> int kvm_remove_breakpoint(CPUState *cpu, int type, vaddr addr, vaddr len);
> void kvm_remove_all_breakpoints(CPUState *cpu);
> -
> #endif /* KVM_CPUS_H */
> diff --git a/accel/kvm/trace-events b/accel/kvm/trace-events
> index 681ccb667d..37626c1ac5 100644
> --- a/accel/kvm/trace-events
> +++ b/accel/kvm/trace-events
> @@ -9,6 +9,10 @@ kvm_device_ioctl(int fd, int type, void *arg) "dev fd %d, type 0x%x, arg %p"
> kvm_failed_reg_get(uint64_t id, const char *msg) "Warning: Unable to retrieve ONEREG %" PRIu64 " from KVM: %s"
> kvm_failed_reg_set(uint64_t id, const char *msg) "Warning: Unable to set ONEREG %" PRIu64 " to KVM: %s"
> kvm_init_vcpu(int cpu_index, unsigned long arch_cpu_id) "index: %d id: %lu"
> +kvm_create_vcpu(int cpu_index, unsigned long arch_cpu_id, int kvm_fd) "index: %d, id: %lu, kvm fd: %d"
> +kvm_destroy_vcpu(int cpu_index, unsigned long arch_cpu_id) "index: %d id: %lu"
> +kvm_park_vcpu(int cpu_index, unsigned long arch_cpu_id) "index: %d id: %lu"
> +kvm_unpark_vcpu(unsigned long arch_cpu_id, const char *msg) "id: %lu %s"
> kvm_irqchip_commit_routes(void) ""
> kvm_irqchip_add_msi_route(char *name, int vector, int virq) "dev %s vector %d virq %d"
> kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d"
> @@ -25,7 +29,6 @@ kvm_dirty_ring_reaper(const char *s) "%s"
> kvm_dirty_ring_reap(uint64_t count, int64_t t) "reaped %"PRIu64" pages (took %"PRIi64" us)"
> kvm_dirty_ring_reaper_kick(const char *reason) "%s"
> kvm_dirty_ring_flush(int finished) "%d"
> -kvm_destroy_vcpu(void) ""
> kvm_failed_get_vcpu_mmap_size(void) ""
> kvm_cpu_exec(void) ""
> kvm_interrupt_exit_request(void) ""
> diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
> index c31d9c7356..c4a914b3d8 100644
> --- a/include/sysemu/kvm.h
> +++ b/include/sysemu/kvm.h
> @@ -313,6 +313,31 @@ int kvm_create_device(KVMState *s, uint64_t type, bool test);
> */
> bool kvm_device_supported(int vmfd, uint64_t type);
>
> +/**
> + * kvm_create_vcpu - Gets a parked KVM vCPU or creates a KVM vCPU
> + * @cpu: QOM CPUState object for which KVM vCPU has to be fetched/created.
> + *
> + * @returns: 0 when success, errno (<0) when failed.
> + */
> +int kvm_create_vcpu(CPUState *cpu);
> +
> +/**
> + * kvm_park_vcpu - Park QEMU KVM vCPU context
> + * @cpu: QOM CPUState object for which QEMU KVM vCPU context has to be parked.
> + *
> + * @returns: none
> + */
> +void kvm_park_vcpu(CPUState *cpu);
> +
> +/**
> + * kvm_unpark_vcpu - unpark QEMU KVM vCPU context
> + * @s: KVM State
> + * @vcpu_id: Architecture vCPU ID of the parked vCPU
> + *
> + * @returns: KVM fd
> + */
> +int kvm_unpark_vcpu(KVMState *s, unsigned long vcpu_id);
> +
> /* Arch specific hooks */
>
> extern const KVMCapabilityInfo kvm_arch_required_capabilities[];
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 2/7] hw/acpi: Move CPU ctrl-dev MMIO region len macro to common header file
2024-07-13 18:25 ` [PATCH V15 2/7] hw/acpi: Move CPU ctrl-dev MMIO region len macro to common header file Salil Mehta via
@ 2024-07-15 12:50 ` Igor Mammedov
0 siblings, 0 replies; 35+ messages in thread
From: Igor Mammedov @ 2024-07-15 12:50 UTC (permalink / raw)
To: Salil Mehta
Cc: qemu-devel, qemu-arm, maz, jean-philippe, jonathan.cameron,
lpieralisi, peter.maydell, richard.henderson, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm,
Shaoqin Huang, Zhao Liu
On Sat, 13 Jul 2024 19:25:11 +0100
Salil Mehta <salil.mehta@huawei.com> wrote:
> CPU ctrl-dev MMIO region length could be used in ACPI GED and various other
> architecture specific places. Move ACPI_CPU_HOTPLUG_REG_LEN macro to more
> appropriate common header file.
>
> Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> Reviewed-by: Gavin Shan <gshan@redhat.com>
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
> Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> Tested-by: Xianglai Li <lixianglai@loongson.cn>
> Tested-by: Miguel Luis <miguel.luis@oracle.com>
> Tested-by: Zhao Liu <zhao1.liu@intel.com>
> Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
> ---
> hw/acpi/cpu.c | 1 -
> include/hw/acpi/cpu.h | 2 ++
> 2 files changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/hw/acpi/cpu.c b/hw/acpi/cpu.c
> index 2d81c1e790..cf5e9183e4 100644
> --- a/hw/acpi/cpu.c
> +++ b/hw/acpi/cpu.c
> @@ -7,7 +7,6 @@
> #include "trace.h"
> #include "sysemu/numa.h"
>
> -#define ACPI_CPU_HOTPLUG_REG_LEN 12
> #define ACPI_CPU_SELECTOR_OFFSET_WR 0
> #define ACPI_CPU_FLAGS_OFFSET_RW 4
> #define ACPI_CPU_CMD_OFFSET_WR 5
> diff --git a/include/hw/acpi/cpu.h b/include/hw/acpi/cpu.h
> index e6e1a9ef59..df87b15997 100644
> --- a/include/hw/acpi/cpu.h
> +++ b/include/hw/acpi/cpu.h
> @@ -19,6 +19,8 @@
> #include "hw/boards.h"
> #include "hw/hotplug.h"
>
> +#define ACPI_CPU_HOTPLUG_REG_LEN 12
> +
> typedef struct AcpiCpuStatus {
> CPUState *cpu;
> uint64_t arch_id;
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 4/7] hw/acpi: Update GED _EVT method AML with CPU scan
2024-07-13 18:25 ` [PATCH V15 4/7] hw/acpi: Update GED _EVT method AML with CPU scan Salil Mehta via
@ 2024-07-15 12:55 ` Igor Mammedov
0 siblings, 0 replies; 35+ messages in thread
From: Igor Mammedov @ 2024-07-15 12:55 UTC (permalink / raw)
To: Salil Mehta
Cc: qemu-devel, qemu-arm, maz, jean-philippe, jonathan.cameron,
lpieralisi, peter.maydell, richard.henderson, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm,
Shaoqin Huang, Zhao Liu
On Sat, 13 Jul 2024 19:25:13 +0100
Salil Mehta <salil.mehta@huawei.com> wrote:
> OSPM evaluates _EVT method to map the event. The CPU hotplug event eventually
> results in start of the CPU scan. Scan figures out the CPU and the kind of
> event(plug/unplug) and notifies it back to the guest. Update the GED AML _EVT
> method with the call to method \\_SB.CPUS.CSCN (via \\_SB.GED.CSCN)
>
> Architecture specific code [1] might initialize its CPUs AML code by calling
> common function build_cpus_aml() like below for ARM:
>
> build_cpus_aml(scope, ms, opts, xx_madt_cpu_entry, memmap[VIRT_CPUHP_ACPI].base,
> "\\_SB", "\\_SB.GED.CSCN", AML_SYSTEM_MEMORY);
>
> [1] https://lore.kernel.org/qemu-devel/20240613233639.202896-13-salil.mehta@huawei.com/
>
> Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
> Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> Reviewed-by: Gavin Shan <gshan@redhat.com>
> Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> Tested-by: Xianglai Li <lixianglai@loongson.cn>
> Tested-by: Miguel Luis <miguel.luis@oracle.com>
> Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
> Tested-by: Zhao Liu <zhao1.liu@intel.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
> ---
> hw/acpi/generic_event_device.c | 3 +++
> include/hw/acpi/generic_event_device.h | 1 +
> 2 files changed, 4 insertions(+)
>
> diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c
> index 1b31d633ba..15ffa12cb2 100644
> --- a/hw/acpi/generic_event_device.c
> +++ b/hw/acpi/generic_event_device.c
> @@ -108,6 +108,9 @@ void build_ged_aml(Aml *table, const char *name, HotplugHandler *hotplug_dev,
> aml_append(if_ctx, aml_call0(MEMORY_DEVICES_CONTAINER "."
> MEMORY_SLOT_SCAN_METHOD));
> break;
> + case ACPI_GED_CPU_HOTPLUG_EVT:
> + aml_append(if_ctx, aml_call0(AML_GED_EVT_CPU_SCAN_METHOD));
> + break;
> case ACPI_GED_PWR_DOWN_EVT:
> aml_append(if_ctx,
> aml_notify(aml_name(ACPI_POWER_BUTTON_DEVICE),
> diff --git a/include/hw/acpi/generic_event_device.h b/include/hw/acpi/generic_event_device.h
> index e091ac2108..40af3550b5 100644
> --- a/include/hw/acpi/generic_event_device.h
> +++ b/include/hw/acpi/generic_event_device.h
> @@ -87,6 +87,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(AcpiGedState, ACPI_GED)
> #define GED_DEVICE "GED"
> #define AML_GED_EVT_REG "EREG"
> #define AML_GED_EVT_SEL "ESEL"
> +#define AML_GED_EVT_CPU_SCAN_METHOD "\\_SB.GED.CSCN"
>
> /*
> * Platforms need to specify the GED event bitmap
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 3/7] hw/acpi: Update ACPI GED framework to support vCPU Hotplug
2024-07-13 18:25 ` [PATCH V15 3/7] hw/acpi: Update ACPI GED framework to support vCPU Hotplug Salil Mehta via
@ 2024-07-15 13:12 ` Igor Mammedov
2024-07-15 13:55 ` Salil Mehta via
0 siblings, 1 reply; 35+ messages in thread
From: Igor Mammedov @ 2024-07-15 13:12 UTC (permalink / raw)
To: Salil Mehta
Cc: qemu-devel, qemu-arm, maz, jean-philippe, jonathan.cameron,
lpieralisi, peter.maydell, richard.henderson, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm,
Shaoqin Huang, Zhao Liu
On Sat, 13 Jul 2024 19:25:12 +0100
Salil Mehta <salil.mehta@huawei.com> wrote:
> ACPI GED (as described in the ACPI 6.4 spec) uses an interrupt listed in the
> _CRS object of GED to intimate OSPM about an event. Later then demultiplexes the
> notified event by evaluating ACPI _EVT method to know the type of event. Use
> ACPI GED to also notify the guest kernel about any CPU hot(un)plug events.
>
> Note, GED interface is used by many hotplug events like memory hotplug, NVDIMM
> hotplug and non-hotplug events like system power down event. Each of these can
> be selected using a bit in the 32 bit GED IO interface. A bit has been reserved
> for the CPU hotplug event.
> ACPI CPU hotplug related initialization should only happen if ACPI_CPU_HOTPLUG
> support has been enabled for particular architecture. Add cpu_hotplug_hw_init()
> stub to avoid compilation break.
so any target (and machines in it) that has ACPI_CPU_HOTPLUG enabled will use have all CPU hotplug
machinery builtin which is fine.
However any machine that uses GED but do not opt-in into CPU hotplug,
will still have CPU hotplug registers/memory regions enabled/mapped.
It's not much concern for upstream as migration from new to older QEMU
is not supported, however it will break migration downstream (arm/virt) as
new QEMU will try to migrate memory regions/state that do not exists
in older QEMU. Se below for suggestion.
>
> Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
> Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> Reviewed-by: Gavin Shan <gshan@redhat.com>
> Reviewed-by: David Hildenbrand <david@redhat.com>
> Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
> Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> Tested-by: Xianglai Li <lixianglai@loongson.cn>
> Tested-by: Miguel Luis <miguel.luis@oracle.com>
> Reviewed-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> Tested-by: Zhao Liu <zhao1.liu@intel.com>
> Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
> ---
> docs/specs/acpi_hw_reduced_hotplug.rst | 3 ++-
> hw/acpi/acpi-cpu-hotplug-stub.c | 6 ++++++
> hw/acpi/generic_event_device.c | 24 ++++++++++++++++++++++++
> include/hw/acpi/generic_event_device.h | 4 ++++
> 4 files changed, 36 insertions(+), 1 deletion(-)
>
> diff --git a/docs/specs/acpi_hw_reduced_hotplug.rst b/docs/specs/acpi_hw_reduced_hotplug.rst
> index 0bd3f9399f..3acd6fcd8b 100644
> --- a/docs/specs/acpi_hw_reduced_hotplug.rst
> +++ b/docs/specs/acpi_hw_reduced_hotplug.rst
> @@ -64,7 +64,8 @@ GED IO interface (4 byte access)
> 0: Memory hotplug event
> 1: System power down event
> 2: NVDIMM hotplug event
> - 3-31: Reserved
> + 3: CPU hotplug event
> + 4-31: Reserved
>
> **write_access:**
>
> diff --git a/hw/acpi/acpi-cpu-hotplug-stub.c b/hw/acpi/acpi-cpu-hotplug-stub.c
> index 3fc4b14c26..c6c61bb9cd 100644
> --- a/hw/acpi/acpi-cpu-hotplug-stub.c
> +++ b/hw/acpi/acpi-cpu-hotplug-stub.c
> @@ -19,6 +19,12 @@ void legacy_acpi_cpu_hotplug_init(MemoryRegion *parent, Object *owner,
> return;
> }
>
> +void cpu_hotplug_hw_init(MemoryRegion *as, Object *owner,
> + CPUHotplugState *state, hwaddr base_addr)
> +{
> + return;
> +}
> +
> void acpi_cpu_ospm_status(CPUHotplugState *cpu_st, ACPIOSTInfoList ***list)
> {
> return;
> diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c
> index 2d6e91b124..1b31d633ba 100644
> --- a/hw/acpi/generic_event_device.c
> +++ b/hw/acpi/generic_event_device.c
> @@ -25,6 +25,7 @@ static const uint32_t ged_supported_events[] = {
> ACPI_GED_MEM_HOTPLUG_EVT,
> ACPI_GED_PWR_DOWN_EVT,
> ACPI_GED_NVDIMM_HOTPLUG_EVT,
> + ACPI_GED_CPU_HOTPLUG_EVT,
> };
>
> /*
> @@ -234,6 +235,8 @@ static void acpi_ged_device_plug_cb(HotplugHandler *hotplug_dev,
> } else {
> acpi_memory_plug_cb(hotplug_dev, &s->memhp_state, dev, errp);
> }
> + } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> + acpi_cpu_plug_cb(hotplug_dev, &s->cpuhp_state, dev, errp);
> } else {
> error_setg(errp, "virt: device plug request for unsupported device"
> " type: %s", object_get_typename(OBJECT(dev)));
> @@ -248,6 +251,8 @@ static void acpi_ged_unplug_request_cb(HotplugHandler *hotplug_dev,
> if ((object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) &&
> !(object_dynamic_cast(OBJECT(dev), TYPE_NVDIMM)))) {
> acpi_memory_unplug_request_cb(hotplug_dev, &s->memhp_state, dev, errp);
> + } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> + acpi_cpu_unplug_request_cb(hotplug_dev, &s->cpuhp_state, dev, errp);
> } else {
> error_setg(errp, "acpi: device unplug request for unsupported device"
> " type: %s", object_get_typename(OBJECT(dev)));
> @@ -261,6 +266,8 @@ static void acpi_ged_unplug_cb(HotplugHandler *hotplug_dev,
>
> if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
> acpi_memory_unplug_cb(&s->memhp_state, dev, errp);
> + } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> + acpi_cpu_unplug_cb(&s->cpuhp_state, dev, errp);
> } else {
> error_setg(errp, "acpi: device unplug for unsupported device"
> " type: %s", object_get_typename(OBJECT(dev)));
> @@ -272,6 +279,7 @@ static void acpi_ged_ospm_status(AcpiDeviceIf *adev, ACPIOSTInfoList ***list)
> AcpiGedState *s = ACPI_GED(adev);
>
> acpi_memory_ospm_status(&s->memhp_state, list);
> + acpi_cpu_ospm_status(&s->cpuhp_state, list);
> }
>
> static void acpi_ged_send_event(AcpiDeviceIf *adev, AcpiEventStatusBits ev)
> @@ -286,6 +294,8 @@ static void acpi_ged_send_event(AcpiDeviceIf *adev, AcpiEventStatusBits ev)
> sel = ACPI_GED_PWR_DOWN_EVT;
> } else if (ev & ACPI_NVDIMM_HOTPLUG_STATUS) {
> sel = ACPI_GED_NVDIMM_HOTPLUG_EVT;
> + } else if (ev & ACPI_CPU_HOTPLUG_STATUS) {
> + sel = ACPI_GED_CPU_HOTPLUG_EVT;
> } else {
> /* Unknown event. Return without generating interrupt. */
> warn_report("GED: Unsupported event %d. No irq injected", ev);
> @@ -371,6 +381,19 @@ static const VMStateDescription vmstate_acpi_ged = {
> }
> };
>
> +static void acpi_ged_realize(DeviceState *dev, Error **errp)
> +{
> + AcpiGedState *s = ACPI_GED(dev);
> + SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
> +
> + /* initialize CPU Hotplug related regions */
> + memory_region_init(&s->container_cpuhp, OBJECT(dev), "cpuhp container",
> + ACPI_CPU_HOTPLUG_REG_LEN);
> + sysbus_init_mmio(sbd, &s->container_cpuhp);
> + cpu_hotplug_hw_init(&s->container_cpuhp, OBJECT(dev),
> + &s->cpuhp_state, 0);
how about making it conditional on supported events
see hw/arm/virt.c: create_acpi_ged()
if (ms->ram_slots) {
event |= ACPI_GED_MEM_HOTPLUG_EVT;
}
if (ms->nvdimms_state->is_enabled) {
event |= ACPI_GED_NVDIMM_HOTPLUG_EVT;
}
so cpu hotplug would have similar section
and ged realize could use ged-event to enable/disable
cpuhp feature registers and hwinit.
> +}
> +
> static void acpi_ged_initfn(Object *obj)
> {
> DeviceState *dev = DEVICE(obj);
> @@ -411,6 +434,7 @@ static void acpi_ged_class_init(ObjectClass *class, void *data)
> dc->desc = "ACPI Generic Event Device";
> device_class_set_props(dc, acpi_ged_properties);
> dc->vmsd = &vmstate_acpi_ged;
> + dc->realize = acpi_ged_realize;
>
> hc->plug = acpi_ged_device_plug_cb;
> hc->unplug_request = acpi_ged_unplug_request_cb;
> diff --git a/include/hw/acpi/generic_event_device.h b/include/hw/acpi/generic_event_device.h
> index ba84ce0214..e091ac2108 100644
> --- a/include/hw/acpi/generic_event_device.h
> +++ b/include/hw/acpi/generic_event_device.h
> @@ -62,6 +62,7 @@
> #include "hw/sysbus.h"
> #include "hw/acpi/memory_hotplug.h"
> #include "hw/acpi/ghes.h"
> +#include "hw/acpi/cpu.h"
> #include "qom/object.h"
>
> #define ACPI_POWER_BUTTON_DEVICE "PWRB"
> @@ -95,6 +96,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(AcpiGedState, ACPI_GED)
> #define ACPI_GED_MEM_HOTPLUG_EVT 0x1
> #define ACPI_GED_PWR_DOWN_EVT 0x2
> #define ACPI_GED_NVDIMM_HOTPLUG_EVT 0x4
> +#define ACPI_GED_CPU_HOTPLUG_EVT 0x8
>
> typedef struct GEDState {
> MemoryRegion evt;
> @@ -106,6 +108,8 @@ struct AcpiGedState {
> SysBusDevice parent_obj;
> MemHotplugState memhp_state;
> MemoryRegion container_memhp;
> + CPUHotplugState cpuhp_state;
> + MemoryRegion container_cpuhp;
> GEDState ged_state;
> uint32_t ged_event_bitmap;
> qemu_irq irq;
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 1/7] accel/kvm: Extract common KVM vCPU {creation,parking} code
2024-07-15 12:49 ` [PATCH V15 1/7] accel/kvm: Extract common KVM vCPU {creation,parking} code Igor Mammedov
@ 2024-07-15 13:28 ` Igor Mammedov
2024-07-15 13:57 ` Salil Mehta via
0 siblings, 1 reply; 35+ messages in thread
From: Igor Mammedov @ 2024-07-15 13:28 UTC (permalink / raw)
To: Salil Mehta
Cc: qemu-devel, qemu-arm, maz, jean-philippe, jonathan.cameron,
lpieralisi, peter.maydell, richard.henderson, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm,
Shaoqin Huang, Zhao Liu
On Mon, 15 Jul 2024 14:49:25 +0200
Igor Mammedov <imammedo@redhat.com> wrote:
> On Sat, 13 Jul 2024 19:25:10 +0100
> Salil Mehta <salil.mehta@huawei.com> wrote:
>
> > KVM vCPU creation is done once during the vCPU realization when Qemu vCPU thread
> > is spawned. This is common to all the architectures as of now.
> >
> > Hot-unplug of vCPU results in destruction of the vCPU object in QOM but the
> > corresponding KVM vCPU object in the Host KVM is not destroyed as KVM doesn't
> > support vCPU removal. Therefore, its representative KVM vCPU object/context in
> > Qemu is parked.
> >
> > Refactor architecture common logic so that some APIs could be reused by vCPU
> > Hotplug code of some architectures likes ARM, Loongson etc. Update new/old APIs
> > with trace events. New APIs qemu_{create,park,unpark}_vcpu() can be externally
> > called. No functional change is intended here.
> >
> > Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
> > Reviewed-by: Gavin Shan <gshan@redhat.com>
> > Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> > Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> > Tested-by: Xianglai Li <lixianglai@loongson.cn>
> > Tested-by: Miguel Luis <miguel.luis@oracle.com>
> > Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
> > Reviewed-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> > Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
> > Tested-by: Zhao Liu <zhao1.liu@intel.com>
> > Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
> > Reviewed-by: Harsh Prateek Bora <harshpb@linux.ibm.com>
>
> Reviewed-by: Igor Mammedov <imammedo@redhat.com>
this needs fixing, to make checkpatch happy
Checking 0001-accel-kvm-Extract-common-KVM-vCPU-creation-parking-c.patch...
WARNING: line over 80 characters
#120: FILE: accel/kvm/kvm-all.c:368:
+ trace_kvm_unpark_vcpu(vcpu_id, kvm_fd > 0 ? "unparked" : "not found parked");
total: 0 errors, 1 warnings, 183 lines checked
>
> > ---
> > accel/kvm/kvm-all.c | 95 ++++++++++++++++++++++++++++--------------
> > accel/kvm/kvm-cpus.h | 1 -
> > accel/kvm/trace-events | 5 ++-
> > include/sysemu/kvm.h | 25 +++++++++++
> > 4 files changed, 92 insertions(+), 34 deletions(-)
> >
> > diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
> > index 2b4ab89679..e446d18944 100644
> > --- a/accel/kvm/kvm-all.c
> > +++ b/accel/kvm/kvm-all.c
> > @@ -340,14 +340,71 @@ err:
> > return ret;
> > }
> >
> > +void kvm_park_vcpu(CPUState *cpu)
> > +{
> > + struct KVMParkedVcpu *vcpu;
> > +
> > + trace_kvm_park_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu));
> > +
> > + vcpu = g_malloc0(sizeof(*vcpu));
> > + vcpu->vcpu_id = kvm_arch_vcpu_id(cpu);
> > + vcpu->kvm_fd = cpu->kvm_fd;
> > + QLIST_INSERT_HEAD(&kvm_state->kvm_parked_vcpus, vcpu, node);
> > +}
> > +
> > +int kvm_unpark_vcpu(KVMState *s, unsigned long vcpu_id)
> > +{
> > + struct KVMParkedVcpu *cpu;
> > + int kvm_fd = -ENOENT;
> > +
> > + QLIST_FOREACH(cpu, &s->kvm_parked_vcpus, node) {
> > + if (cpu->vcpu_id == vcpu_id) {
> > + QLIST_REMOVE(cpu, node);
> > + kvm_fd = cpu->kvm_fd;
> > + g_free(cpu);
> > + }
> > + }
> > +
> > + trace_kvm_unpark_vcpu(vcpu_id, kvm_fd > 0 ? "unparked" : "not found parked");
> > +
> > + return kvm_fd;
> > +}
> > +
> > +int kvm_create_vcpu(CPUState *cpu)
> > +{
> > + unsigned long vcpu_id = kvm_arch_vcpu_id(cpu);
> > + KVMState *s = kvm_state;
> > + int kvm_fd;
> > +
> > + /* check if the KVM vCPU already exist but is parked */
> > + kvm_fd = kvm_unpark_vcpu(s, vcpu_id);
> > + if (kvm_fd < 0) {
> > + /* vCPU not parked: create a new KVM vCPU */
> > + kvm_fd = kvm_vm_ioctl(s, KVM_CREATE_VCPU, vcpu_id);
> > + if (kvm_fd < 0) {
> > + error_report("KVM_CREATE_VCPU IOCTL failed for vCPU %lu", vcpu_id);
> > + return kvm_fd;
> > + }
> > + }
> > +
> > + cpu->kvm_fd = kvm_fd;
> > + cpu->kvm_state = s;
> > + cpu->vcpu_dirty = true;
> > + cpu->dirty_pages = 0;
> > + cpu->throttle_us_per_full = 0;
> > +
> > + trace_kvm_create_vcpu(cpu->cpu_index, vcpu_id, kvm_fd);
> > +
> > + return 0;
> > +}
> > +
> > static int do_kvm_destroy_vcpu(CPUState *cpu)
> > {
> > KVMState *s = kvm_state;
> > long mmap_size;
> > - struct KVMParkedVcpu *vcpu = NULL;
> > int ret = 0;
> >
> > - trace_kvm_destroy_vcpu();
> > + trace_kvm_destroy_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu));
> >
> > ret = kvm_arch_destroy_vcpu(cpu);
> > if (ret < 0) {
> > @@ -373,10 +430,7 @@ static int do_kvm_destroy_vcpu(CPUState *cpu)
> > }
> > }
> >
> > - vcpu = g_malloc0(sizeof(*vcpu));
> > - vcpu->vcpu_id = kvm_arch_vcpu_id(cpu);
> > - vcpu->kvm_fd = cpu->kvm_fd;
> > - QLIST_INSERT_HEAD(&kvm_state->kvm_parked_vcpus, vcpu, node);
> > + kvm_park_vcpu(cpu);
> > err:
> > return ret;
> > }
> > @@ -389,24 +443,6 @@ void kvm_destroy_vcpu(CPUState *cpu)
> > }
> > }
> >
> > -static int kvm_get_vcpu(KVMState *s, unsigned long vcpu_id)
> > -{
> > - struct KVMParkedVcpu *cpu;
> > -
> > - QLIST_FOREACH(cpu, &s->kvm_parked_vcpus, node) {
> > - if (cpu->vcpu_id == vcpu_id) {
> > - int kvm_fd;
> > -
> > - QLIST_REMOVE(cpu, node);
> > - kvm_fd = cpu->kvm_fd;
> > - g_free(cpu);
> > - return kvm_fd;
> > - }
> > - }
> > -
> > - return kvm_vm_ioctl(s, KVM_CREATE_VCPU, (void *)vcpu_id);
> > -}
> > -
> > int kvm_init_vcpu(CPUState *cpu, Error **errp)
> > {
> > KVMState *s = kvm_state;
> > @@ -415,19 +451,14 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
> >
> > trace_kvm_init_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu));
> >
> > - ret = kvm_get_vcpu(s, kvm_arch_vcpu_id(cpu));
> > + ret = kvm_create_vcpu(cpu);
> > if (ret < 0) {
> > - error_setg_errno(errp, -ret, "kvm_init_vcpu: kvm_get_vcpu failed (%lu)",
> > + error_setg_errno(errp, -ret,
> > + "kvm_init_vcpu: kvm_create_vcpu failed (%lu)",
> > kvm_arch_vcpu_id(cpu));
> > goto err;
> > }
> >
> > - cpu->kvm_fd = ret;
> > - cpu->kvm_state = s;
> > - cpu->vcpu_dirty = true;
> > - cpu->dirty_pages = 0;
> > - cpu->throttle_us_per_full = 0;
> > -
> > mmap_size = kvm_ioctl(s, KVM_GET_VCPU_MMAP_SIZE, 0);
> > if (mmap_size < 0) {
> > ret = mmap_size;
> > diff --git a/accel/kvm/kvm-cpus.h b/accel/kvm/kvm-cpus.h
> > index ca40add32c..171b22fd29 100644
> > --- a/accel/kvm/kvm-cpus.h
> > +++ b/accel/kvm/kvm-cpus.h
> > @@ -22,5 +22,4 @@ bool kvm_supports_guest_debug(void);
> > int kvm_insert_breakpoint(CPUState *cpu, int type, vaddr addr, vaddr len);
> > int kvm_remove_breakpoint(CPUState *cpu, int type, vaddr addr, vaddr len);
> > void kvm_remove_all_breakpoints(CPUState *cpu);
> > -
> > #endif /* KVM_CPUS_H */
> > diff --git a/accel/kvm/trace-events b/accel/kvm/trace-events
> > index 681ccb667d..37626c1ac5 100644
> > --- a/accel/kvm/trace-events
> > +++ b/accel/kvm/trace-events
> > @@ -9,6 +9,10 @@ kvm_device_ioctl(int fd, int type, void *arg) "dev fd %d, type 0x%x, arg %p"
> > kvm_failed_reg_get(uint64_t id, const char *msg) "Warning: Unable to retrieve ONEREG %" PRIu64 " from KVM: %s"
> > kvm_failed_reg_set(uint64_t id, const char *msg) "Warning: Unable to set ONEREG %" PRIu64 " to KVM: %s"
> > kvm_init_vcpu(int cpu_index, unsigned long arch_cpu_id) "index: %d id: %lu"
> > +kvm_create_vcpu(int cpu_index, unsigned long arch_cpu_id, int kvm_fd) "index: %d, id: %lu, kvm fd: %d"
> > +kvm_destroy_vcpu(int cpu_index, unsigned long arch_cpu_id) "index: %d id: %lu"
> > +kvm_park_vcpu(int cpu_index, unsigned long arch_cpu_id) "index: %d id: %lu"
> > +kvm_unpark_vcpu(unsigned long arch_cpu_id, const char *msg) "id: %lu %s"
> > kvm_irqchip_commit_routes(void) ""
> > kvm_irqchip_add_msi_route(char *name, int vector, int virq) "dev %s vector %d virq %d"
> > kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d"
> > @@ -25,7 +29,6 @@ kvm_dirty_ring_reaper(const char *s) "%s"
> > kvm_dirty_ring_reap(uint64_t count, int64_t t) "reaped %"PRIu64" pages (took %"PRIi64" us)"
> > kvm_dirty_ring_reaper_kick(const char *reason) "%s"
> > kvm_dirty_ring_flush(int finished) "%d"
> > -kvm_destroy_vcpu(void) ""
> > kvm_failed_get_vcpu_mmap_size(void) ""
> > kvm_cpu_exec(void) ""
> > kvm_interrupt_exit_request(void) ""
> > diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
> > index c31d9c7356..c4a914b3d8 100644
> > --- a/include/sysemu/kvm.h
> > +++ b/include/sysemu/kvm.h
> > @@ -313,6 +313,31 @@ int kvm_create_device(KVMState *s, uint64_t type, bool test);
> > */
> > bool kvm_device_supported(int vmfd, uint64_t type);
> >
> > +/**
> > + * kvm_create_vcpu - Gets a parked KVM vCPU or creates a KVM vCPU
> > + * @cpu: QOM CPUState object for which KVM vCPU has to be fetched/created.
> > + *
> > + * @returns: 0 when success, errno (<0) when failed.
> > + */
> > +int kvm_create_vcpu(CPUState *cpu);
> > +
> > +/**
> > + * kvm_park_vcpu - Park QEMU KVM vCPU context
> > + * @cpu: QOM CPUState object for which QEMU KVM vCPU context has to be parked.
> > + *
> > + * @returns: none
> > + */
> > +void kvm_park_vcpu(CPUState *cpu);
> > +
> > +/**
> > + * kvm_unpark_vcpu - unpark QEMU KVM vCPU context
> > + * @s: KVM State
> > + * @vcpu_id: Architecture vCPU ID of the parked vCPU
> > + *
> > + * @returns: KVM fd
> > + */
> > +int kvm_unpark_vcpu(KVMState *s, unsigned long vcpu_id);
> > +
> > /* Arch specific hooks */
> >
> > extern const KVMCapabilityInfo kvm_arch_required_capabilities[];
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 5/7] hw/acpi: Update CPUs AML with cpu-(ctrl)dev change
2024-07-13 18:25 ` [PATCH V15 5/7] hw/acpi: Update CPUs AML with cpu-(ctrl)dev change Salil Mehta via
@ 2024-07-15 13:39 ` Igor Mammedov
0 siblings, 0 replies; 35+ messages in thread
From: Igor Mammedov @ 2024-07-15 13:39 UTC (permalink / raw)
To: Salil Mehta
Cc: qemu-devel, qemu-arm, maz, jean-philippe, jonathan.cameron,
lpieralisi, peter.maydell, richard.henderson, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm,
Shaoqin Huang, Zhao Liu
On Sat, 13 Jul 2024 19:25:14 +0100
Salil Mehta <salil.mehta@huawei.com> wrote:
> CPUs Control device(\\_SB.PCI0) register interface for the x86 arch is IO port
> based and existing CPUs AML code assumes _CRS objects would evaluate to a system
> resource which describes IO Port address. But on ARM arch CPUs control
> device(\\_SB.PRES) register interface is memory-mapped hence _CRS object should
> evaluate to system resource which describes memory-mapped base address. Update
> build CPUs AML function to accept both IO/MEMORY region spaces and accordingly
> update the _CRS object.
>
> Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
> Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
> Reviewed-by: Gavin Shan <gshan@redhat.com>
> Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> Tested-by: Xianglai Li <lixianglai@loongson.cn>
> Tested-by: Miguel Luis <miguel.luis@oracle.com>
> Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
> Tested-by: Zhao Liu <zhao1.liu@intel.com>
Reviewed-by: Igor Mammedov <imammedo@redhat.com>
> ---
> hw/acpi/cpu.c | 17 +++++++++++++----
> hw/i386/acpi-build.c | 3 ++-
> include/hw/acpi/cpu.h | 5 +++--
> 3 files changed, 18 insertions(+), 7 deletions(-)
>
> diff --git a/hw/acpi/cpu.c b/hw/acpi/cpu.c
> index cf5e9183e4..5cb60ca8bc 100644
> --- a/hw/acpi/cpu.c
> +++ b/hw/acpi/cpu.c
> @@ -338,9 +338,10 @@ const VMStateDescription vmstate_cpu_hotplug = {
> #define CPU_FW_EJECT_EVENT "CEJF"
>
> void build_cpus_aml(Aml *table, MachineState *machine, CPUHotplugFeatures opts,
> - build_madt_cpu_fn build_madt_cpu, hwaddr io_base,
> + build_madt_cpu_fn build_madt_cpu, hwaddr base_addr,
> const char *res_root,
> - const char *event_handler_method)
> + const char *event_handler_method,
> + AmlRegionSpace rs)
> {
> Aml *ifctx;
> Aml *field;
> @@ -364,14 +365,22 @@ void build_cpus_aml(Aml *table, MachineState *machine, CPUHotplugFeatures opts,
> aml_name_decl("_UID", aml_string("CPU Hotplug resources")));
> aml_append(cpu_ctrl_dev, aml_mutex(CPU_LOCK, 0));
>
> + assert((rs == AML_SYSTEM_IO) || (rs == AML_SYSTEM_MEMORY));
> +
> crs = aml_resource_template();
> - aml_append(crs, aml_io(AML_DECODE16, io_base, io_base, 1,
> + if (rs == AML_SYSTEM_IO) {
> + aml_append(crs, aml_io(AML_DECODE16, base_addr, base_addr, 1,
> ACPI_CPU_HOTPLUG_REG_LEN));
> + } else if (rs == AML_SYSTEM_MEMORY) {
> + aml_append(crs, aml_memory32_fixed(base_addr,
> + ACPI_CPU_HOTPLUG_REG_LEN, AML_READ_WRITE));
> + }
> +
> aml_append(cpu_ctrl_dev, aml_name_decl("_CRS", crs));
>
> /* declare CPU hotplug MMIO region with related access fields */
> aml_append(cpu_ctrl_dev,
> - aml_operation_region("PRST", AML_SYSTEM_IO, aml_int(io_base),
> + aml_operation_region("PRST", rs, aml_int(base_addr),
> ACPI_CPU_HOTPLUG_REG_LEN));
>
> field = aml_field("PRST", AML_BYTE_ACC, AML_NOLOCK,
> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> index f4e366f64f..5d4bd2b710 100644
> --- a/hw/i386/acpi-build.c
> +++ b/hw/i386/acpi-build.c
> @@ -1536,7 +1536,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
> .fw_unplugs_cpu = pm->smi_on_cpu_unplug,
> };
> build_cpus_aml(dsdt, machine, opts, pc_madt_cpu_entry,
> - pm->cpu_hp_io_base, "\\_SB.PCI0", "\\_GPE._E02");
> + pm->cpu_hp_io_base, "\\_SB.PCI0", "\\_GPE._E02",
> + AML_SYSTEM_IO);
> }
>
> if (pcms->memhp_io_base && nr_mem) {
> diff --git a/include/hw/acpi/cpu.h b/include/hw/acpi/cpu.h
> index df87b15997..32654dc274 100644
> --- a/include/hw/acpi/cpu.h
> +++ b/include/hw/acpi/cpu.h
> @@ -63,9 +63,10 @@ typedef void (*build_madt_cpu_fn)(int uid, const CPUArchIdList *apic_ids,
> GArray *entry, bool force_enabled);
>
> void build_cpus_aml(Aml *table, MachineState *machine, CPUHotplugFeatures opts,
> - build_madt_cpu_fn build_madt_cpu, hwaddr io_base,
> + build_madt_cpu_fn build_madt_cpu, hwaddr base_addr,
> const char *res_root,
> - const char *event_handler_method);
> + const char *event_handler_method,
> + AmlRegionSpace rs);
>
> void acpi_cpu_ospm_status(CPUHotplugState *cpu_st, ACPIOSTInfoList ***list);
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 7/7] gdbstub: Add helper function to unregister GDB register space
2024-07-13 18:25 ` [PATCH V15 7/7] gdbstub: Add helper function to unregister GDB register space Salil Mehta via
@ 2024-07-15 13:40 ` Igor Mammedov
0 siblings, 0 replies; 35+ messages in thread
From: Igor Mammedov @ 2024-07-15 13:40 UTC (permalink / raw)
To: Salil Mehta
Cc: qemu-devel, qemu-arm, maz, jean-philippe, jonathan.cameron,
lpieralisi, peter.maydell, richard.henderson, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm,
Shaoqin Huang, Zhao Liu
On Sat, 13 Jul 2024 19:25:16 +0100
Salil Mehta <salil.mehta@huawei.com> wrote:
> Add common function to help unregister the GDB register space. This shall be
> done in context to the CPU unrealization.
>
> Note: These are common functions exported to arch specific code. For example,
> for ARM this code is being referred in associated arch specific patch-set:
>
> Link: https://lore.kernel.org/qemu-devel/20230926103654.34424-1-salil.mehta@huawei.com/
>
> Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
> Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> Reviewed-by: Gavin Shan <gshan@redhat.com>
> Tested-by: Xianglai Li <lixianglai@loongson.cn>
> Tested-by: Miguel Luis <miguel.luis@oracle.com>
> Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
> Reviewed-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> Tested-by: Zhao Liu <zhao1.liu@intel.com>
Acked-by: Igor Mammedov <imammedo@redhat.com>
> ---
> gdbstub/gdbstub.c | 13 +++++++++++++
> hw/core/cpu-common.c | 4 +++-
> include/exec/gdbstub.h | 6 ++++++
> 3 files changed, 22 insertions(+), 1 deletion(-)
>
> diff --git a/gdbstub/gdbstub.c b/gdbstub/gdbstub.c
> index b9ad0a063e..5da17d6530 100644
> --- a/gdbstub/gdbstub.c
> +++ b/gdbstub/gdbstub.c
> @@ -618,6 +618,19 @@ void gdb_register_coprocessor(CPUState *cpu,
> }
> }
>
> +void gdb_unregister_coprocessor_all(CPUState *cpu)
> +{
> + /*
> + * Safe to nuke everything. GDBRegisterState::xml is static const char so
> + * it won't be freed
> + */
> + g_array_free(cpu->gdb_regs, true);
> +
> + cpu->gdb_regs = NULL;
> + cpu->gdb_num_regs = 0;
> + cpu->gdb_num_g_regs = 0;
> +}
> +
> static void gdb_process_breakpoint_remove_all(GDBProcess *p)
> {
> CPUState *cpu = gdb_get_first_cpu_in_process(p);
> diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c
> index b19e1fdacf..fe5383b4f9 100644
> --- a/hw/core/cpu-common.c
> +++ b/hw/core/cpu-common.c
> @@ -281,7 +281,9 @@ static void cpu_common_finalize(Object *obj)
> g_free(cpu->plugin_state);
> }
> #endif
> - g_array_free(cpu->gdb_regs, TRUE);
> + /* If cleanup didn't happen in context to gdb_unregister_coprocessor_all */
> + if (cpu->gdb_regs)
> + g_array_free(cpu->gdb_regs, TRUE);
> qemu_lockcnt_destroy(&cpu->in_ioctl_lock);
> qemu_mutex_destroy(&cpu->work_mutex);
> qemu_cond_destroy(cpu->halt_cond);
> diff --git a/include/exec/gdbstub.h b/include/exec/gdbstub.h
> index 1bd2c4ec2a..d73f424f56 100644
> --- a/include/exec/gdbstub.h
> +++ b/include/exec/gdbstub.h
> @@ -40,6 +40,12 @@ void gdb_register_coprocessor(CPUState *cpu,
> gdb_get_reg_cb get_reg, gdb_set_reg_cb set_reg,
> const GDBFeature *feature, int g_pos);
>
> +/**
> + * gdb_unregister_coprocessor_all() - unregisters supplemental set of registers
> + * @cpu - the CPU associated with registers
> + */
> +void gdb_unregister_coprocessor_all(CPUState *cpu);
> +
> /**
> * gdbserver_start: start the gdb server
> * @port_or_device: connection spec for gdb
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-13 18:25 [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Salil Mehta via
` (10 preceding siblings ...)
2024-07-15 11:14 ` Salil Mehta via
@ 2024-07-15 13:54 ` Igor Mammedov
2024-07-15 14:14 ` Salil Mehta via
11 siblings, 1 reply; 35+ messages in thread
From: Igor Mammedov @ 2024-07-15 13:54 UTC (permalink / raw)
To: Salil Mehta
Cc: qemu-devel, qemu-arm, maz, jean-philippe, jonathan.cameron,
lpieralisi, peter.maydell, richard.henderson, andrew.jones, david,
philmd, eric.auger, oliver.upton, pbonzini, mst, will, gshan,
rafael, alex.bennee, linux, darren, ilkka, vishnu, karl.heubaum,
miguel.luis, salil.mehta, zhukeqian1, wangxiongfeng2, wangyanan55,
jiakernel2, maobibo, lixianglai, npiggin, harshpb, linuxarm
On Sat, 13 Jul 2024 19:25:09 +0100
Salil Mehta <salil.mehta@huawei.com> wrote:
> [Note: References are present at the last after the revision history]
>
> Virtual CPU hotplug support is being added across various architectures [1][3].
> This series adds various code bits common across all architectures:
>
> 1. vCPU creation and Parking code refactor [Patch 1]
> 2. Update ACPI GED framework to support vCPU Hotplug [Patch 2,3]
> 3. ACPI CPUs AML code change [Patch 4,5]
> 4. Helper functions to support unrealization of CPU objects [Patch 6,7]
with patch 1 and 3 fixed should be good to go.
Salil,
Can you remind me what happened to migration part of this?
Ideally it should be a part of of this series as it should be common
for everything that uses GED and should be a conditional part
of GED's VMSTATE.
If this series is just a common base and no actual hotplug
on top of it is merged in this release (provided patch 13 is fixed),
I'm fine with migration bits being a separate series on top.
However if some machine would be introducing cpu hotplug in
the same release, then the migration part should be merged before
it or be a part that cpu hotplug series.
> Repository:
>
> [*] Architecture *Agnostic* Patch-set (This series)
> V14: https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v3.arch.agnostic.v15
>
> NOTE: This series is meant to work in conjunction with the architecture-specific
> patch-set. For ARM, a combined patch-set (architecture agnostic + specific) was
> earlier pushed as RFC V2 [1]. Later, RFC V2 was split into the ARM Architecture
> specific patch-set RFC V3 [4] (a subset of RFC V2) and the architecture agnostic
> patch-set. Patch-set V14 is the latest version in that series. This series
> works in conjunction with RFC V4-rc2, present at the following link.
>
> [*] ARM Architecture *Specific* Patch-set
> RFC V3 [4]: https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v3
> RFC V4-rc2: https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v4-rc2 (combined)
>
>
> Revision History:
>
> Patch-set V14 -> V15
> 1. Addressed commnet from Igor Mammedov's on [PATCH V14 4/7]
> - Removed ACPI_CPU_SCAN_METHOD
> - Introduced AML_GED_EVT_CPU_SCAN_METHOD ("\\_SB.GED.CPSCN") macro
> 2. Fix the stray change of "assert (" in "PATCH V14 3/7"
> Link: https://lore.kernel.org/qemu-devel/20240712134201.214699-4-salil.mehta@huawei.com/
>
> Patch-set V13 -> V14
> 1. Addressed Igor Mammedov's following review comments
> - Mentioned abput new external APIs in the header note of [PATCH 1/7]
> - Merged Doc [PATCH V13 8/8] with [PATCH V14 3/7]
> - Introduced GED realize function for various CPU Hotplug regions initializations
> - Added back event handler method to indirectly expose \\_SB.CPUS.CSCN to GED
> _EVT. Like for ARM, it would be through \\_SB.GED.CSCN event handler method
> - Collected the Ack given for [Patch V13 6/8]
> - Added back the gfree'ing of GDB regs in common finalize and made it conditional
> - Updated the header notes of [PATCH V13 3/8,4/8,5/8] to reflect the changes
>
> Patch-set V12 -> V13
> 1. Added Reviewed-by Tag of Harsh Prateek Bora's (IBM) [PATCH V12 1/8]
> 2. Moved the kvm_{create,park,unpark}_vcpu prototypes from accel/kvm/kvm-cpus.h
> to include/sysemu/kvm.h. These can later be exported through AccelOps.
> Link: https://lore.kernel.org/qemu-devel/62f55169-1796-4d8e-a35d-7f003a172750@linux.ibm.com/
>
> Patch-set V11 -> V12
> 1. Addressed Harsh Prateek Bora's (IBM) comment
> - Changed @cpu to @vcpu_id in the kvm_unpark_vcpu protoype header/
> 2. Added Zhao Liu's (Intel) Tested-by for whole series
> - Qtest does not breaks on Intel platforms now.
> 3. Added Zhao Liu's (Intel) Reviewed-by for [PATCH V11 {1/8 - 3/8}]
> Link: https://lore.kernel.org/qemu-devel/ZlRSPuJGBgyEUW6w@intel.com/
> Link: https://lore.kernel.org/qemu-devel/a5f3d78e-cfed-441f-9c56-e3e78fa5edee@linux.ibm.com/
>
> Patch-set V10 -> V11
> 1. Addressed Nicholas Piggin's (IBM) comment
> - moved the traces in kvm_unpark_vcpu and kvm_create_vcpu at the end
> - Added the Reviewed-by Tag for [PATCH V10 1/8]
> 2. Addressed Alex Bennée's (Linaro) comments
> - Added a note explaining dependency of the [PATCH V10 7/8] on Arch specific patch-set
> Link: https://lore.kernel.org/qemu-devel/D1FS5GOOFWWK.2PNRIVL0V6DBL@gmail.com/
> Link: https://lore.kernel.org/qemu-devel/87frubi402.fsf@draig.linaro.org/
>
> Patch-set V9 -> V10
> 1. Addressed Nicholas Piggin's (IBM) & Philippe Mathieu-Daudé (Linaro) comments
> - carved out kvm_unpark_vcpu and added its trace
> - Widened the scope of the kvm_unpark_vcpu so that it can be used by generic framework
> being thought out
> Link: https://lore.kernel.org/qemu-devel/20240519210620.228342-1-salil.mehta@huawei.com/
> Link: https://lore.kernel.org/qemu-devel/e94b0e14-efee-4050-9c9f-08382a36b63a@linaro.org/
>
> Patch-set V8 -> V9
> 1. Addressed Vishnu Pajjuri's (Ampere) comments
> - Added kvm_fd to trace in kvm_create_vcpu
> - Some clean ups: arch vcpu-id and sbd variable
> - Added the missed initialization of cpu->gdb_num_regs
> 2. Addressed the commnet from Zhao Liu (Intel)
> - Make initialization of CPU Hotplug state conditional (possible_cpu_arch_ids!=NULL)
> Link: https://lore.kernel.org/qemu-devel/20240312020000.12992-1-salil.mehta@huawei.com/
>
> Patch-set V7 -> V8
> 1. Rebased and Fixed the conflicts
>
> Patch-set V6 -> V7
> 1. Addressed Alex Bennée's comments
> - Updated the docs
> 2. Addressed Igor Mammedov's comments
> - Merged patches [Patch V6 3/9] & [Patch V6 7/9] with [Patch V6 4/9]
> - Updated commit-log of [Patch V6 1/9] and [Patch V6 5/9]
> 3. Added Shaoqin Huang's Reviewed-by tags for whole series.
> Link: https://lore.kernel.org/qemu-devel/20231013105129.25648-1-salil.mehta@huawei.com/
>
> Patch-set V5 -> V6
> 1. Addressed Gavin Shan's comments
> - Fixed the assert() ranges of address spaces
> - Rebased the patch-set to latest changes in the qemu.git
> - Added Reviewed-by tags for patches {8,9}
> 2. Addressed Jonathan Cameron's comments
> - Updated commit-log for [Patch V5 1/9] with mention of trace events
> - Added Reviewed-by tags for patches {1,5}
> 3. Added Tested-by tags from Xianglai Li
> 4. Fixed checkpatch.pl error "Qemu -> QEMU" in [Patch V5 1/9]
> Link: https://lore.kernel.org/qemu-devel/20231011194355.15628-1-salil.mehta@huawei.com/
>
> Patch-set V4 -> V5
> 1. Addressed Gavin Shan's comments
> - Fixed the trace events print string for kvm_{create,get,park,destroy}_vcpu
> - Added Reviewed-by tag for patch {1}
> 2. Added Shaoqin Huang's Reviewed-by tags for Patches {2,3}
> 3. Added Tested-by Tag from Vishnu Pajjuri to the patch-set
> 4. Dropped the ARM specific [Patch V4 10/10]
> Link: https://lore.kernel.org/qemu-devel/20231009203601.17584-1-salil.mehta@huawei.com/
>
> Patch-set V3 -> V4
> 1. Addressed David Hilderbrand's comments
> - Fixed the wrong doc comment of kvm_park_vcpu API prototype
> - Added Reviewed-by tags for patches {2,4}
> Link: https://lore.kernel.org/qemu-devel/20231009112812.10612-1-salil.mehta@huawei.com/
>
> Patch-set V2 -> V3
> 1. Addressed Jonathan Cameron's comments
> - Fixed 'vcpu-id' type wrongly changed from 'unsigned long' to 'integer'
> - Removed unnecessary use of variable 'vcpu_id' in kvm_park_vcpu
> - Updated [Patch V2 3/10] commit-log with details of ACPI_CPU_SCAN_METHOD macro
> - Updated [Patch V2 5/10] commit-log with details of conditional event handler method
> - Added Reviewed-by tags for patches {2,3,4,6,7}
> 2. Addressed Gavin Shan's comments
> - Remove unnecessary use of variable 'vcpu_id' in kvm_par_vcpu
> - Fixed return value in kvm_get_vcpu from -1 to -ENOENT
> - Reset the value of 'gdb_num_g_regs' in gdb_unregister_coprocessor_all
> - Fixed the kvm_{create,park}_vcpu prototypes docs
> - Added Reviewed-by tags for patches {2,3,4,5,6,7,9,10}
> 3. Addressed one earlier missed comment by Alex Bennée in RFC V1
> - Added traces instead of DPRINTF in the newly added and some existing functions
> Link: https://lore.kernel.org/qemu-devel/20230930001933.2660-1-salil.mehta@huawei.com/
>
> Patch-set V1 -> V2
> 1. Addressed Alex Bennée's comments
> - Refactored the kvm_create_vcpu logic to get rid of goto
> - Added the docs for kvm_{create,park}_vcpu prototypes
> - Splitted the gdbstub and AddressSpace destruction change into separate patches
> - Added Reviewed-by tags for patches {2,10}
> Link: https://lore.kernel.org/qemu-devel/20230929124304.13672-1-salil.mehta@huawei.com/
>
> References:
>
> [1] https://lore.kernel.org/qemu-devel/20230926100436.28284-1-salil.mehta@huawei.com/
> [2] https://lore.kernel.org/all/20230913163823.7880-1-james.morse@arm.com/
> [3] https://lore.kernel.org/qemu-devel/cover.1695697701.git.lixianglai@loongson.cn/
> [4] https://lore.kernel.org/qemu-devel/20240613233639.202896-2-salil.mehta@huawei.com/
>
> Salil Mehta (7):
> accel/kvm: Extract common KVM vCPU {creation,parking} code
> hw/acpi: Move CPU ctrl-dev MMIO region len macro to common header file
> hw/acpi: Update ACPI GED framework to support vCPU Hotplug
> hw/acpi: Update GED _EVT method AML with CPU scan
> hw/acpi: Update CPUs AML with cpu-(ctrl)dev change
> physmem: Add helper function to destroy CPU AddressSpace
> gdbstub: Add helper function to unregister GDB register space
>
> accel/kvm/kvm-all.c | 95 +++++++++++++++++---------
> accel/kvm/kvm-cpus.h | 1 -
> accel/kvm/trace-events | 5 +-
> docs/specs/acpi_hw_reduced_hotplug.rst | 3 +-
> gdbstub/gdbstub.c | 13 ++++
> hw/acpi/acpi-cpu-hotplug-stub.c | 6 ++
> hw/acpi/cpu.c | 18 +++--
> hw/acpi/generic_event_device.c | 27 ++++++++
> hw/core/cpu-common.c | 4 +-
> hw/i386/acpi-build.c | 3 +-
> include/exec/cpu-common.h | 8 +++
> include/exec/gdbstub.h | 6 ++
> include/hw/acpi/cpu.h | 7 +-
> include/hw/acpi/generic_event_device.h | 5 ++
> include/hw/core/cpu.h | 1 +
> include/sysemu/kvm.h | 25 +++++++
> system/physmem.c | 29 ++++++++
> 17 files changed, 212 insertions(+), 44 deletions(-)
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-15 11:27 ` Salil Mehta via
2024-07-15 11:33 ` Michael S. Tsirkin
@ 2024-07-15 13:55 ` Igor Mammedov
1 sibling, 0 replies; 35+ messages in thread
From: Igor Mammedov @ 2024-07-15 13:55 UTC (permalink / raw)
To: Salil Mehta
Cc: Michael S. Tsirkin, qemu-devel@nongnu.org, qemu-arm@nongnu.org,
maz@kernel.org, jean-philippe@linaro.org, Jonathan Cameron,
lpieralisi@kernel.org, peter.maydell@linaro.org,
richard.henderson@linaro.org, andrew.jones@linux.dev,
david@redhat.com, philmd@linaro.org, eric.auger@redhat.com,
oliver.upton@linux.dev, pbonzini@redhat.com, will@kernel.org,
gshan@redhat.com, rafael@kernel.org, alex.bennee@linaro.org,
linux@armlinux.org.uk, darren@os.amperecomputing.com,
ilkka@os.amperecomputing.com, vishnu@os.amperecomputing.com,
karl.heubaum@oracle.com, miguel.luis@oracle.com,
salil.mehta@opnsrc.net, zhukeqian, wangxiongfeng (C),
wangyanan (Y), jiakernel2@gmail.com, maobibo@loongson.cn,
lixianglai@loongson.cn, npiggin@gmail.com, harshpb@linux.ibm.com,
Linuxarm
On Mon, 15 Jul 2024 11:27:57 +0000
Salil Mehta <salil.mehta@huawei.com> wrote:
> Hi Michael,
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Monday, July 15, 2024 12:13 PM
> > To: Salil Mehta <salil.mehta@huawei.com>
> >
> > On Sat, Jul 13, 2024 at 07:25:09PM +0100, Salil Mehta wrote:
> > > [Note: References are present at the last after the revision history]
> >
> > Igor any comments before I merge this?
>
> Hi Michael,
>
> Assuming there are no last-minute surprises and If you decide to merge this
> series, could I kindly request that you collect all the Tags (XXX-Bys) including
> the Igor's pending Reviewed/Acked-By Tag for the entire series, so that I won't
> have to churn out another version (V16)?
v16 might be necessary, see cover letter.
>
> Many thanks!
>
> Best regards
> Salil
>
>
> >
> > --
> > MST
> >
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* RE: [PATCH V15 3/7] hw/acpi: Update ACPI GED framework to support vCPU Hotplug
2024-07-15 13:12 ` Igor Mammedov
@ 2024-07-15 13:55 ` Salil Mehta via
0 siblings, 0 replies; 35+ messages in thread
From: Salil Mehta via @ 2024-07-15 13:55 UTC (permalink / raw)
To: Igor Mammedov
Cc: qemu-devel@nongnu.org, qemu-arm@nongnu.org, maz@kernel.org,
jean-philippe@linaro.org, Jonathan Cameron, lpieralisi@kernel.org,
peter.maydell@linaro.org, richard.henderson@linaro.org,
andrew.jones@linux.dev, david@redhat.com, philmd@linaro.org,
eric.auger@redhat.com, oliver.upton@linux.dev,
pbonzini@redhat.com, mst@redhat.com, will@kernel.org,
gshan@redhat.com, rafael@kernel.org, alex.bennee@linaro.org,
linux@armlinux.org.uk, darren@os.amperecomputing.com,
ilkka@os.amperecomputing.com, vishnu@os.amperecomputing.com,
karl.heubaum@oracle.com, miguel.luis@oracle.com,
salil.mehta@opnsrc.net, zhukeqian, wangxiongfeng (C),
wangyanan (Y), jiakernel2@gmail.com, maobibo@loongson.cn,
lixianglai@loongson.cn, npiggin@gmail.com, harshpb@linux.ibm.com,
Linuxarm, Shaoqin Huang, Zhao Liu
Hi Igor,
Thanks for the feedback.
> From: qemu-arm-bounces+salil.mehta=huawei.com@nongnu.org <qemu-
> arm-bounces+salil.mehta=huawei.com@nongnu.org> On Behalf Of Igor
> Mammedov
> Sent: Monday, July 15, 2024 2:12 PM
> To: Salil Mehta <salil.mehta@huawei.com>
>
> On Sat, 13 Jul 2024 19:25:12 +0100
> Salil Mehta <salil.mehta@huawei.com> wrote:
>
> > ACPI GED (as described in the ACPI 6.4 spec) uses an interrupt listed
> > in the _CRS object of GED to intimate OSPM about an event. Later then
> > demultiplexes the notified event by evaluating ACPI _EVT method to
> > know the type of event. Use ACPI GED to also notify the guest kernel
> about any CPU hot(un)plug events.
> >
> > Note, GED interface is used by many hotplug events like memory
> > hotplug, NVDIMM hotplug and non-hotplug events like system power
> down
> > event. Each of these can be selected using a bit in the 32 bit GED IO
> > interface. A bit has been reserved for the CPU hotplug event.
>
> > ACPI CPU hotplug related initialization should only happen if
> > ACPI_CPU_HOTPLUG support has been enabled for particular
> architecture.
> > Add cpu_hotplug_hw_init() stub to avoid compilation break.
>
> so any target (and machines in it) that has ACPI_CPU_HOTPLUG enabled will
> use have all CPU hotplug machinery builtin which is fine.
>
> However any machine that uses GED but do not opt-in into CPU hotplug, will
> still have CPU hotplug registers/memory regions enabled/mapped.
>
> It's not much concern for upstream as migration from new to older QEMU is
> not supported, however it will break migration downstream (arm/virt) as
> new QEMU will try to migrate memory regions/state that do not exists in
> older QEMU. Se below for suggestion.
Sorry. I missed this one. Yes. agreed.
> > Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
> > Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
> > Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
> > Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> > Reviewed-by: Gavin Shan <gshan@redhat.com>
> > Reviewed-by: David Hildenbrand <david@redhat.com>
> > Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
> > Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> > Tested-by: Xianglai Li <lixianglai@loongson.cn>
> > Tested-by: Miguel Luis <miguel.luis@oracle.com>
> > Reviewed-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> > Tested-by: Zhao Liu <zhao1.liu@intel.com>
> > Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
> > ---
> > docs/specs/acpi_hw_reduced_hotplug.rst | 3 ++-
> > hw/acpi/acpi-cpu-hotplug-stub.c | 6 ++++++
> > hw/acpi/generic_event_device.c | 24 ++++++++++++++++++++++++
> > include/hw/acpi/generic_event_device.h | 4 ++++
> > 4 files changed, 36 insertions(+), 1 deletion(-)
> >
> > diff --git a/docs/specs/acpi_hw_reduced_hotplug.rst
> > b/docs/specs/acpi_hw_reduced_hotplug.rst
> > index 0bd3f9399f..3acd6fcd8b 100644
> > --- a/docs/specs/acpi_hw_reduced_hotplug.rst
> > +++ b/docs/specs/acpi_hw_reduced_hotplug.rst
> > @@ -64,7 +64,8 @@ GED IO interface (4 byte access)
> > 0: Memory hotplug event
> > 1: System power down event
> > 2: NVDIMM hotplug event
> > - 3-31: Reserved
> > + 3: CPU hotplug event
> > + 4-31: Reserved
> >
> > **write_access:**
> >
> > diff --git a/hw/acpi/acpi-cpu-hotplug-stub.c
> > b/hw/acpi/acpi-cpu-hotplug-stub.c index 3fc4b14c26..c6c61bb9cd 100644
> > --- a/hw/acpi/acpi-cpu-hotplug-stub.c
> > +++ b/hw/acpi/acpi-cpu-hotplug-stub.c
> > @@ -19,6 +19,12 @@ void legacy_acpi_cpu_hotplug_init(MemoryRegion
> *parent, Object *owner,
> > return;
> > }
> >
> > +void cpu_hotplug_hw_init(MemoryRegion *as, Object *owner,
> > + CPUHotplugState *state, hwaddr base_addr) {
> > + return;
> > +}
> > +
> > void acpi_cpu_ospm_status(CPUHotplugState *cpu_st, ACPIOSTInfoList
> > ***list) {
> > return;
> > diff --git a/hw/acpi/generic_event_device.c
> > b/hw/acpi/generic_event_device.c index 2d6e91b124..1b31d633ba
> 100644
> > --- a/hw/acpi/generic_event_device.c
> > +++ b/hw/acpi/generic_event_device.c
> > @@ -25,6 +25,7 @@ static const uint32_t ged_supported_events[] = {
> > ACPI_GED_MEM_HOTPLUG_EVT,
> > ACPI_GED_PWR_DOWN_EVT,
> > ACPI_GED_NVDIMM_HOTPLUG_EVT,
> > + ACPI_GED_CPU_HOTPLUG_EVT,
> > };
> >
> > /*
> > @@ -234,6 +235,8 @@ static void
> acpi_ged_device_plug_cb(HotplugHandler *hotplug_dev,
> > } else {
> > acpi_memory_plug_cb(hotplug_dev, &s->memhp_state, dev,
> errp);
> > }
> > + } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> > + acpi_cpu_plug_cb(hotplug_dev, &s->cpuhp_state, dev, errp);
> > } else {
> > error_setg(errp, "virt: device plug request for unsupported device"
> > " type: %s", object_get_typename(OBJECT(dev)));
> > @@ -248,6 +251,8 @@ static void
> acpi_ged_unplug_request_cb(HotplugHandler *hotplug_dev,
> > if ((object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) &&
> > !(object_dynamic_cast(OBJECT(dev), TYPE_NVDIMM)))) {
> > acpi_memory_unplug_request_cb(hotplug_dev, &s->memhp_state,
> > dev, errp);
> > + } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> > + acpi_cpu_unplug_request_cb(hotplug_dev, &s->cpuhp_state, dev,
> > + errp);
> > } else {
> > error_setg(errp, "acpi: device unplug request for unsupported
> device"
> > " type: %s", object_get_typename(OBJECT(dev)));
> > @@ -261,6 +266,8 @@ static void acpi_ged_unplug_cb(HotplugHandler
> > *hotplug_dev,
> >
> > if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
> > acpi_memory_unplug_cb(&s->memhp_state, dev, errp);
> > + } else if (object_dynamic_cast(OBJECT(dev), TYPE_CPU)) {
> > + acpi_cpu_unplug_cb(&s->cpuhp_state, dev, errp);
> > } else {
> > error_setg(errp, "acpi: device unplug for unsupported device"
> > " type: %s", object_get_typename(OBJECT(dev)));
> > @@ -272,6 +279,7 @@ static void acpi_ged_ospm_status(AcpiDeviceIf
> *adev, ACPIOSTInfoList ***list)
> > AcpiGedState *s = ACPI_GED(adev);
> >
> > acpi_memory_ospm_status(&s->memhp_state, list);
> > + acpi_cpu_ospm_status(&s->cpuhp_state, list);
> > }
> >
> > static void acpi_ged_send_event(AcpiDeviceIf *adev,
> > AcpiEventStatusBits ev) @@ -286,6 +294,8 @@ static void
> acpi_ged_send_event(AcpiDeviceIf *adev, AcpiEventStatusBits ev)
> > sel = ACPI_GED_PWR_DOWN_EVT;
> > } else if (ev & ACPI_NVDIMM_HOTPLUG_STATUS) {
> > sel = ACPI_GED_NVDIMM_HOTPLUG_EVT;
> > + } else if (ev & ACPI_CPU_HOTPLUG_STATUS) {
> > + sel = ACPI_GED_CPU_HOTPLUG_EVT;
> > } else {
> > /* Unknown event. Return without generating interrupt. */
> > warn_report("GED: Unsupported event %d. No irq injected",
> > ev); @@ -371,6 +381,19 @@ static const VMStateDescription
> vmstate_acpi_ged = {
> > }
> > };
> >
> > +static void acpi_ged_realize(DeviceState *dev, Error **errp) {
> > + AcpiGedState *s = ACPI_GED(dev);
> > + SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
> > +
>
> > + /* initialize CPU Hotplug related regions */
> > + memory_region_init(&s->container_cpuhp, OBJECT(dev), "cpuhp
> container",
> > + ACPI_CPU_HOTPLUG_REG_LEN);
> > + sysbus_init_mmio(sbd, &s->container_cpuhp);
> > + cpu_hotplug_hw_init(&s->container_cpuhp, OBJECT(dev),
> > + &s->cpuhp_state, 0);
>
> how about making it conditional on supported events see hw/arm/virt.c:
> create_acpi_ged()
> if (ms->ram_slots) {
> event |= ACPI_GED_MEM_HOTPLUG_EVT;
> }
>
> if (ms->nvdimms_state->is_enabled) {
> event |= ACPI_GED_NVDIMM_HOTPLUG_EVT;
> }
>
> so cpu hotplug would have similar section and ged realize could use ged-
> event to enable/disable cpuhp feature registers and hwinit.
Got it, thanks for pointing out. I will fix this and share V16 by COB today itself.
Best regards
Salil.
>
> > +}
> > +
> > static void acpi_ged_initfn(Object *obj) {
> > DeviceState *dev = DEVICE(obj);
> > @@ -411,6 +434,7 @@ static void acpi_ged_class_init(ObjectClass *class,
> void *data)
> > dc->desc = "ACPI Generic Event Device";
> > device_class_set_props(dc, acpi_ged_properties);
> > dc->vmsd = &vmstate_acpi_ged;
> > + dc->realize = acpi_ged_realize;
> >
> > hc->plug = acpi_ged_device_plug_cb;
> > hc->unplug_request = acpi_ged_unplug_request_cb; diff --git
> > a/include/hw/acpi/generic_event_device.h
> > b/include/hw/acpi/generic_event_device.h
> > index ba84ce0214..e091ac2108 100644
> > --- a/include/hw/acpi/generic_event_device.h
> > +++ b/include/hw/acpi/generic_event_device.h
> > @@ -62,6 +62,7 @@
> > #include "hw/sysbus.h"
> > #include "hw/acpi/memory_hotplug.h"
> > #include "hw/acpi/ghes.h"
> > +#include "hw/acpi/cpu.h"
> > #include "qom/object.h"
> >
> > #define ACPI_POWER_BUTTON_DEVICE "PWRB"
> > @@ -95,6 +96,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(AcpiGedState,
> ACPI_GED)
> > #define ACPI_GED_MEM_HOTPLUG_EVT 0x1
> > #define ACPI_GED_PWR_DOWN_EVT 0x2
> > #define ACPI_GED_NVDIMM_HOTPLUG_EVT 0x4
> > +#define ACPI_GED_CPU_HOTPLUG_EVT 0x8
> >
> > typedef struct GEDState {
> > MemoryRegion evt;
> > @@ -106,6 +108,8 @@ struct AcpiGedState {
> > SysBusDevice parent_obj;
> > MemHotplugState memhp_state;
> > MemoryRegion container_memhp;
> > + CPUHotplugState cpuhp_state;
> > + MemoryRegion container_cpuhp;
> > GEDState ged_state;
> > uint32_t ged_event_bitmap;
> > qemu_irq irq;
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* RE: [PATCH V15 1/7] accel/kvm: Extract common KVM vCPU {creation,parking} code
2024-07-15 13:28 ` Igor Mammedov
@ 2024-07-15 13:57 ` Salil Mehta via
0 siblings, 0 replies; 35+ messages in thread
From: Salil Mehta via @ 2024-07-15 13:57 UTC (permalink / raw)
To: Igor Mammedov
Cc: qemu-devel@nongnu.org, qemu-arm@nongnu.org, maz@kernel.org,
jean-philippe@linaro.org, Jonathan Cameron, lpieralisi@kernel.org,
peter.maydell@linaro.org, richard.henderson@linaro.org,
andrew.jones@linux.dev, david@redhat.com, philmd@linaro.org,
eric.auger@redhat.com, oliver.upton@linux.dev,
pbonzini@redhat.com, mst@redhat.com, will@kernel.org,
gshan@redhat.com, rafael@kernel.org, alex.bennee@linaro.org,
linux@armlinux.org.uk, darren@os.amperecomputing.com,
ilkka@os.amperecomputing.com, vishnu@os.amperecomputing.com,
karl.heubaum@oracle.com, miguel.luis@oracle.com,
salil.mehta@opnsrc.net, zhukeqian, wangxiongfeng (C),
wangyanan (Y), jiakernel2@gmail.com, maobibo@loongson.cn,
lixianglai@loongson.cn, npiggin@gmail.com, harshpb@linux.ibm.com,
Linuxarm, Shaoqin Huang, Zhao Liu
> From: Igor Mammedov <imammedo@redhat.com>
> Sent: Monday, July 15, 2024 2:28 PM
> To: Salil Mehta <salil.mehta@huawei.com>
>
> On Mon, 15 Jul 2024 14:49:25 +0200
> Igor Mammedov <imammedo@redhat.com> wrote:
>
> > On Sat, 13 Jul 2024 19:25:10 +0100
> > Salil Mehta <salil.mehta@huawei.com> wrote:
> >
> > > KVM vCPU creation is done once during the vCPU realization when Qemu
> > > vCPU thread is spawned. This is common to all the architectures as of now.
> > >
> > > Hot-unplug of vCPU results in destruction of the vCPU object in QOM
> > > but the corresponding KVM vCPU object in the Host KVM is not
> > > destroyed as KVM doesn't support vCPU removal. Therefore, its
> > > representative KVM vCPU object/context in Qemu is parked.
> > >
> > > Refactor architecture common logic so that some APIs could be reused
> > > by vCPU Hotplug code of some architectures likes ARM, Loongson etc.
> > > Update new/old APIs with trace events. New APIs
> > > qemu_{create,park,unpark}_vcpu() can be externally called. No
> functional change is intended here.
> > >
> > > Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
> > > Reviewed-by: Gavin Shan <gshan@redhat.com>
> > > Tested-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> > > Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
> > > Tested-by: Xianglai Li <lixianglai@loongson.cn>
> > > Tested-by: Miguel Luis <miguel.luis@oracle.com>
> > > Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
> > > Reviewed-by: Vishnu Pajjuri <vishnu@os.amperecomputing.com>
> > > Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
> > > Tested-by: Zhao Liu <zhao1.liu@intel.com>
> > > Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
> > > Reviewed-by: Harsh Prateek Bora <harshpb@linux.ibm.com>
> >
> > Reviewed-by: Igor Mammedov <imammedo@redhat.com>
>
> this needs fixing, to make checkpatch happy
>
> Checking 0001-accel-kvm-Extract-common-KVM-vCPU-creation-parking-
> c.patch...
> WARNING: line over 80 characters
> #120: FILE: accel/kvm/kvm-all.c:368:
> + trace_kvm_unpark_vcpu(vcpu_id, kvm_fd > 0 ? "unparked" : "not found
> + parked");
Thanks. I will fix this as well.
>
> total: 0 errors, 1 warnings, 183 lines checked
>
> >
> > > ---
> > > accel/kvm/kvm-all.c | 95 ++++++++++++++++++++++++++++----------
> ----
> > > accel/kvm/kvm-cpus.h | 1 -
> > > accel/kvm/trace-events | 5 ++-
> > > include/sysemu/kvm.h | 25 +++++++++++
> > > 4 files changed, 92 insertions(+), 34 deletions(-)
> > >
> > > diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index
> > > 2b4ab89679..e446d18944 100644
> > > --- a/accel/kvm/kvm-all.c
> > > +++ b/accel/kvm/kvm-all.c
> > > @@ -340,14 +340,71 @@ err:
> > > return ret;
> > > }
> > >
> > > +void kvm_park_vcpu(CPUState *cpu)
> > > +{
> > > + struct KVMParkedVcpu *vcpu;
> > > +
> > > + trace_kvm_park_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu));
> > > +
> > > + vcpu = g_malloc0(sizeof(*vcpu));
> > > + vcpu->vcpu_id = kvm_arch_vcpu_id(cpu);
> > > + vcpu->kvm_fd = cpu->kvm_fd;
> > > + QLIST_INSERT_HEAD(&kvm_state->kvm_parked_vcpus, vcpu,
> node); }
> > > +
> > > +int kvm_unpark_vcpu(KVMState *s, unsigned long vcpu_id) {
> > > + struct KVMParkedVcpu *cpu;
> > > + int kvm_fd = -ENOENT;
> > > +
> > > + QLIST_FOREACH(cpu, &s->kvm_parked_vcpus, node) {
> > > + if (cpu->vcpu_id == vcpu_id) {
> > > + QLIST_REMOVE(cpu, node);
> > > + kvm_fd = cpu->kvm_fd;
> > > + g_free(cpu);
> > > + }
> > > + }
> > > +
> > > + trace_kvm_unpark_vcpu(vcpu_id, kvm_fd > 0 ? "unparked" : "not
> > > + found parked");
> > > +
> > > + return kvm_fd;
> > > +}
> > > +
> > > +int kvm_create_vcpu(CPUState *cpu)
> > > +{
> > > + unsigned long vcpu_id = kvm_arch_vcpu_id(cpu);
> > > + KVMState *s = kvm_state;
> > > + int kvm_fd;
> > > +
> > > + /* check if the KVM vCPU already exist but is parked */
> > > + kvm_fd = kvm_unpark_vcpu(s, vcpu_id);
> > > + if (kvm_fd < 0) {
> > > + /* vCPU not parked: create a new KVM vCPU */
> > > + kvm_fd = kvm_vm_ioctl(s, KVM_CREATE_VCPU, vcpu_id);
> > > + if (kvm_fd < 0) {
> > > + error_report("KVM_CREATE_VCPU IOCTL failed for vCPU %lu",
> vcpu_id);
> > > + return kvm_fd;
> > > + }
> > > + }
> > > +
> > > + cpu->kvm_fd = kvm_fd;
> > > + cpu->kvm_state = s;
> > > + cpu->vcpu_dirty = true;
> > > + cpu->dirty_pages = 0;
> > > + cpu->throttle_us_per_full = 0;
> > > +
> > > + trace_kvm_create_vcpu(cpu->cpu_index, vcpu_id, kvm_fd);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > static int do_kvm_destroy_vcpu(CPUState *cpu) {
> > > KVMState *s = kvm_state;
> > > long mmap_size;
> > > - struct KVMParkedVcpu *vcpu = NULL;
> > > int ret = 0;
> > >
> > > - trace_kvm_destroy_vcpu();
> > > + trace_kvm_destroy_vcpu(cpu->cpu_index,
> kvm_arch_vcpu_id(cpu));
> > >
> > > ret = kvm_arch_destroy_vcpu(cpu);
> > > if (ret < 0) {
> > > @@ -373,10 +430,7 @@ static int do_kvm_destroy_vcpu(CPUState
> *cpu)
> > > }
> > > }
> > >
> > > - vcpu = g_malloc0(sizeof(*vcpu));
> > > - vcpu->vcpu_id = kvm_arch_vcpu_id(cpu);
> > > - vcpu->kvm_fd = cpu->kvm_fd;
> > > - QLIST_INSERT_HEAD(&kvm_state->kvm_parked_vcpus, vcpu, node);
> > > + kvm_park_vcpu(cpu);
> > > err:
> > > return ret;
> > > }
> > > @@ -389,24 +443,6 @@ void kvm_destroy_vcpu(CPUState *cpu)
> > > }
> > > }
> > >
> > > -static int kvm_get_vcpu(KVMState *s, unsigned long vcpu_id) -{
> > > - struct KVMParkedVcpu *cpu;
> > > -
> > > - QLIST_FOREACH(cpu, &s->kvm_parked_vcpus, node) {
> > > - if (cpu->vcpu_id == vcpu_id) {
> > > - int kvm_fd;
> > > -
> > > - QLIST_REMOVE(cpu, node);
> > > - kvm_fd = cpu->kvm_fd;
> > > - g_free(cpu);
> > > - return kvm_fd;
> > > - }
> > > - }
> > > -
> > > - return kvm_vm_ioctl(s, KVM_CREATE_VCPU, (void *)vcpu_id);
> > > -}
> > > -
> > > int kvm_init_vcpu(CPUState *cpu, Error **errp) {
> > > KVMState *s = kvm_state;
> > > @@ -415,19 +451,14 @@ int kvm_init_vcpu(CPUState *cpu, Error
> **errp)
> > >
> > > trace_kvm_init_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu));
> > >
> > > - ret = kvm_get_vcpu(s, kvm_arch_vcpu_id(cpu));
> > > + ret = kvm_create_vcpu(cpu);
> > > if (ret < 0) {
> > > - error_setg_errno(errp, -ret, "kvm_init_vcpu: kvm_get_vcpu failed
> (%lu)",
> > > + error_setg_errno(errp, -ret,
> > > + "kvm_init_vcpu: kvm_create_vcpu failed
> > > + (%lu)",
> > > kvm_arch_vcpu_id(cpu));
> > > goto err;
> > > }
> > >
> > > - cpu->kvm_fd = ret;
> > > - cpu->kvm_state = s;
> > > - cpu->vcpu_dirty = true;
> > > - cpu->dirty_pages = 0;
> > > - cpu->throttle_us_per_full = 0;
> > > -
> > > mmap_size = kvm_ioctl(s, KVM_GET_VCPU_MMAP_SIZE, 0);
> > > if (mmap_size < 0) {
> > > ret = mmap_size;
> > > diff --git a/accel/kvm/kvm-cpus.h b/accel/kvm/kvm-cpus.h index
> > > ca40add32c..171b22fd29 100644
> > > --- a/accel/kvm/kvm-cpus.h
> > > +++ b/accel/kvm/kvm-cpus.h
> > > @@ -22,5 +22,4 @@ bool kvm_supports_guest_debug(void); int
> > > kvm_insert_breakpoint(CPUState *cpu, int type, vaddr addr, vaddr
> > > len); int kvm_remove_breakpoint(CPUState *cpu, int type, vaddr
> > > addr, vaddr len); void kvm_remove_all_breakpoints(CPUState *cpu);
> > > -
> > > #endif /* KVM_CPUS_H */
> > > diff --git a/accel/kvm/trace-events b/accel/kvm/trace-events index
> > > 681ccb667d..37626c1ac5 100644
> > > --- a/accel/kvm/trace-events
> > > +++ b/accel/kvm/trace-events
> > > @@ -9,6 +9,10 @@ kvm_device_ioctl(int fd, int type, void *arg) "dev fd
> %d, type 0x%x, arg %p"
> > > kvm_failed_reg_get(uint64_t id, const char *msg) "Warning: Unable to
> retrieve ONEREG %" PRIu64 " from KVM: %s"
> > > kvm_failed_reg_set(uint64_t id, const char *msg) "Warning: Unable to
> set ONEREG %" PRIu64 " to KVM: %s"
> > > kvm_init_vcpu(int cpu_index, unsigned long arch_cpu_id) "index: %d
> id: %lu"
> > > +kvm_create_vcpu(int cpu_index, unsigned long arch_cpu_id, int
> kvm_fd) "index: %d, id: %lu, kvm fd: %d"
> > > +kvm_destroy_vcpu(int cpu_index, unsigned long arch_cpu_id) "index:
> %d id: %lu"
> > > +kvm_park_vcpu(int cpu_index, unsigned long arch_cpu_id) "index: %d
> id: %lu"
> > > +kvm_unpark_vcpu(unsigned long arch_cpu_id, const char *msg) "id:
> %lu %s"
> > > kvm_irqchip_commit_routes(void) ""
> > > kvm_irqchip_add_msi_route(char *name, int vector, int virq) "dev %s
> vector %d virq %d"
> > > kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d"
> > > @@ -25,7 +29,6 @@ kvm_dirty_ring_reaper(const char *s) "%s"
> > > kvm_dirty_ring_reap(uint64_t count, int64_t t) "reaped %"PRIu64"
> pages (took %"PRIi64" us)"
> > > kvm_dirty_ring_reaper_kick(const char *reason) "%s"
> > > kvm_dirty_ring_flush(int finished) "%d"
> > > -kvm_destroy_vcpu(void) ""
> > > kvm_failed_get_vcpu_mmap_size(void) ""
> > > kvm_cpu_exec(void) ""
> > > kvm_interrupt_exit_request(void) ""
> > > diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h index
> > > c31d9c7356..c4a914b3d8 100644
> > > --- a/include/sysemu/kvm.h
> > > +++ b/include/sysemu/kvm.h
> > > @@ -313,6 +313,31 @@ int kvm_create_device(KVMState *s, uint64_t
> type, bool test);
> > > */
> > > bool kvm_device_supported(int vmfd, uint64_t type);
> > >
> > > +/**
> > > + * kvm_create_vcpu - Gets a parked KVM vCPU or creates a KVM vCPU
> > > + * @cpu: QOM CPUState object for which KVM vCPU has to be
> fetched/created.
> > > + *
> > > + * @returns: 0 when success, errno (<0) when failed.
> > > + */
> > > +int kvm_create_vcpu(CPUState *cpu);
> > > +
> > > +/**
> > > + * kvm_park_vcpu - Park QEMU KVM vCPU context
> > > + * @cpu: QOM CPUState object for which QEMU KVM vCPU context
> has to be parked.
> > > + *
> > > + * @returns: none
> > > + */
> > > +void kvm_park_vcpu(CPUState *cpu);
> > > +
> > > +/**
> > > + * kvm_unpark_vcpu - unpark QEMU KVM vCPU context
> > > + * @s: KVM State
> > > + * @vcpu_id: Architecture vCPU ID of the parked vCPU
> > > + *
> > > + * @returns: KVM fd
> > > + */
> > > +int kvm_unpark_vcpu(KVMState *s, unsigned long vcpu_id);
> > > +
> > > /* Arch specific hooks */
> > >
> > > extern const KVMCapabilityInfo kvm_arch_required_capabilities[];
> >
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* RE: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-15 13:54 ` Igor Mammedov
@ 2024-07-15 14:14 ` Salil Mehta via
2024-07-15 14:19 ` Salil Mehta via
0 siblings, 1 reply; 35+ messages in thread
From: Salil Mehta via @ 2024-07-15 14:14 UTC (permalink / raw)
To: Igor Mammedov
Cc: qemu-devel@nongnu.org, qemu-arm@nongnu.org, maz@kernel.org,
jean-philippe@linaro.org, Jonathan Cameron, lpieralisi@kernel.org,
peter.maydell@linaro.org, richard.henderson@linaro.org,
andrew.jones@linux.dev, david@redhat.com, philmd@linaro.org,
eric.auger@redhat.com, oliver.upton@linux.dev,
pbonzini@redhat.com, mst@redhat.com, will@kernel.org,
gshan@redhat.com, rafael@kernel.org, alex.bennee@linaro.org,
linux@armlinux.org.uk, darren@os.amperecomputing.com,
ilkka@os.amperecomputing.com, vishnu@os.amperecomputing.com,
karl.heubaum@oracle.com, miguel.luis@oracle.com,
salil.mehta@opnsrc.net, zhukeqian, wangxiongfeng (C),
wangyanan (Y), jiakernel2@gmail.com, maobibo@loongson.cn,
lixianglai@loongson.cn, npiggin@gmail.com, harshpb@linux.ibm.com,
Linuxarm
Hi Igor,
> From: Igor Mammedov <imammedo@redhat.com>
> Sent: Monday, July 15, 2024 2:55 PM
> To: Salil Mehta <salil.mehta@huawei.com>
>
> On Sat, 13 Jul 2024 19:25:09 +0100
> Salil Mehta <salil.mehta@huawei.com> wrote:
>
> > [Note: References are present at the last after the revision history]
> >
> > Virtual CPU hotplug support is being added across various architectures
> [1][3].
> > This series adds various code bits common across all architectures:
> >
> > 1. vCPU creation and Parking code refactor [Patch 1] 2. Update ACPI
> > GED framework to support vCPU Hotplug [Patch 2,3] 3. ACPI CPUs AML
> > code change [Patch 4,5] 4. Helper functions to support unrealization
> > of CPU objects [Patch 6,7]
>
> with patch 1 and 3 fixed should be good to go.
>
> Salil,
> Can you remind me what happened to migration part of this?
> Ideally it should be a part of of this series as it should be common for
> everything that uses GED and should be a conditional part of GED's
> VMSTATE.
>
> If this series is just a common base and no actual hotplug on top of it is
> merged in this release (provided patch 13 is fixed), I'm fine with migration
> bits being a separate series on top.
>
> However if some machine would be introducing cpu hotplug in the same
> release, then the migration part should be merged before it or be a part
> that cpu hotplug series.
We have tested Live/Pseudo Migration and it seem to work with the changes
part of the architecture specific patch-set.
Ampere: https://lore.kernel.org/all/e17e28ac-28c7-496f-b212-2c9b552dbf63@amperemail.onmicrosoft.com/
Oracle: https://lore.kernel.org/all/46D74D30-EE54-4AD2-8F0E-BA5627FAA63E@oracle.com/
For ARM, please check below patch part of RFC V3 for changes related to migration:
https://lore.kernel.org/qemu-devel/20240613233639.202896-15-salil.mehta@huawei.com/
Thanks
Salil.
>
> > Repository:
> >
> > [*] Architecture *Agnostic* Patch-set (This series)
> > V14: https://github.com/salil-mehta/qemu.git
> > virt-cpuhp-armv8/rfc-v3.arch.agnostic.v15
> >
> > NOTE: This series is meant to work in conjunction with the architecture-
> specific
> > patch-set. For ARM, a combined patch-set (architecture agnostic +
> specific) was
> > earlier pushed as RFC V2 [1]. Later, RFC V2 was split into the ARM
> Architecture
> > specific patch-set RFC V3 [4] (a subset of RFC V2) and the architecture
> agnostic
> > patch-set. Patch-set V14 is the latest version in that series. This series
> > works in conjunction with RFC V4-rc2, present at the following link.
> >
> > [*] ARM Architecture *Specific* Patch-set
> > RFC V3 [4]: https://github.com/salil-mehta/qemu.git virt-cpuhp-
> armv8/rfc-v3
> > RFC V4-rc2: https://github.com/salil-mehta/qemu.git
> > virt-cpuhp-armv8/rfc-v4-rc2 (combined)
> >
> >
> > Revision History:
> >
> > Patch-set V14 -> V15
> > 1. Addressed commnet from Igor Mammedov's on [PATCH V14 4/7]
> > - Removed ACPI_CPU_SCAN_METHOD
> > - Introduced AML_GED_EVT_CPU_SCAN_METHOD
> ("\\_SB.GED.CPSCN") macro
> > 2. Fix the stray change of "assert (" in "PATCH V14 3/7"
> > Link:
> > https://lore.kernel.org/qemu-devel/20240712134201.214699-4-salil.mehta
> > @huawei.com/
> >
> > Patch-set V13 -> V14
> > 1. Addressed Igor Mammedov's following review comments
> > - Mentioned abput new external APIs in the header note of [PATCH 1/7]
> > - Merged Doc [PATCH V13 8/8] with [PATCH V14 3/7]
> > - Introduced GED realize function for various CPU Hotplug regions
> initializations
> > - Added back event handler method to indirectly expose
> \\_SB.CPUS.CSCN to GED
> > _EVT. Like for ARM, it would be through \\_SB.GED.CSCN event handler
> method
> > - Collected the Ack given for [Patch V13 6/8]
> > - Added back the gfree'ing of GDB regs in common finalize and made it
> conditional
> > - Updated the header notes of [PATCH V13 3/8,4/8,5/8] to reflect
> > the changes
> >
> > Patch-set V12 -> V13
> > 1. Added Reviewed-by Tag of Harsh Prateek Bora's (IBM) [PATCH V12 1/8]
> > 2. Moved the kvm_{create,park,unpark}_vcpu prototypes from
> accel/kvm/kvm-cpus.h
> > to include/sysemu/kvm.h. These can later be exported through
> AccelOps.
> > Link:
> > https://lore.kernel.org/qemu-devel/62f55169-1796-4d8e-a35d-
> 7f003a17275
> > 0@linux.ibm.com/
> >
> > Patch-set V11 -> V12
> > 1. Addressed Harsh Prateek Bora's (IBM) comment
> > - Changed @cpu to @vcpu_id in the kvm_unpark_vcpu protoype
> header/
> > 2. Added Zhao Liu's (Intel) Tested-by for whole series
> > - Qtest does not breaks on Intel platforms now.
> > 3. Added Zhao Liu's (Intel) Reviewed-by for [PATCH V11 {1/8 - 3/8}]
> > Link: https://lore.kernel.org/qemu-
> devel/ZlRSPuJGBgyEUW6w@intel.com/
> > Link:
> > https://lore.kernel.org/qemu-devel/a5f3d78e-cfed-441f-9c56-
> e3e78fa5ede
> > e@linux.ibm.com/
> >
> > Patch-set V10 -> V11
> > 1. Addressed Nicholas Piggin's (IBM) comment
> > - moved the traces in kvm_unpark_vcpu and kvm_create_vcpu at the
> end
> > - Added the Reviewed-by Tag for [PATCH V10 1/8] 2. Addressed Alex
> > Bennée's (Linaro) comments
> > - Added a note explaining dependency of the [PATCH V10 7/8] on Arch
> > specific patch-set
> > Link:
> > https://lore.kernel.org/qemu-
> devel/D1FS5GOOFWWK.2PNRIVL0V6DBL@gmail.co
> > m/
> > Link:
> > https://lore.kernel.org/qemu-devel/87frubi402.fsf@draig.linaro.org/
> >
> > Patch-set V9 -> V10
> > 1. Addressed Nicholas Piggin's (IBM) & Philippe Mathieu-Daudé (Linaro)
> comments
> > - carved out kvm_unpark_vcpu and added its trace
> > - Widened the scope of the kvm_unpark_vcpu so that it can be used by
> generic framework
> > being thought out
> > Link:
> > https://lore.kernel.org/qemu-devel/20240519210620.228342-1-salil.mehta
> > @huawei.com/
> > Link:
> > https://lore.kernel.org/qemu-devel/e94b0e14-efee-4050-9c9f-
> 08382a36b63
> > a@linaro.org/
> >
> > Patch-set V8 -> V9
> > 1. Addressed Vishnu Pajjuri's (Ampere) comments
> > - Added kvm_fd to trace in kvm_create_vcpu
> > - Some clean ups: arch vcpu-id and sbd variable
> > - Added the missed initialization of cpu->gdb_num_regs 2. Addressed
> > the commnet from Zhao Liu (Intel)
> > - Make initialization of CPU Hotplug state conditional
> > (possible_cpu_arch_ids!=NULL)
> > Link:
> > https://lore.kernel.org/qemu-devel/20240312020000.12992-1-
> salil.mehta@
> > huawei.com/
> >
> > Patch-set V7 -> V8
> > 1. Rebased and Fixed the conflicts
> >
> > Patch-set V6 -> V7
> > 1. Addressed Alex Bennée's comments
> > - Updated the docs
> > 2. Addressed Igor Mammedov's comments
> > - Merged patches [Patch V6 3/9] & [Patch V6 7/9] with [Patch V6 4/9]
> > - Updated commit-log of [Patch V6 1/9] and [Patch V6 5/9]
> > 3. Added Shaoqin Huang's Reviewed-by tags for whole series.
> > Link:
> > https://lore.kernel.org/qemu-devel/20231013105129.25648-1-
> salil.mehta@
> > huawei.com/
> >
> > Patch-set V5 -> V6
> > 1. Addressed Gavin Shan's comments
> > - Fixed the assert() ranges of address spaces
> > - Rebased the patch-set to latest changes in the qemu.git
> > - Added Reviewed-by tags for patches {8,9} 2. Addressed Jonathan
> > Cameron's comments
> > - Updated commit-log for [Patch V5 1/9] with mention of trace events
> > - Added Reviewed-by tags for patches {1,5} 3. Added Tested-by tags
> > from Xianglai Li 4. Fixed checkpatch.pl error "Qemu -> QEMU" in [Patch
> > V5 1/9]
> > Link:
> > https://lore.kernel.org/qemu-devel/20231011194355.15628-1-
> salil.mehta@
> > huawei.com/
> >
> > Patch-set V4 -> V5
> > 1. Addressed Gavin Shan's comments
> > - Fixed the trace events print string for
> kvm_{create,get,park,destroy}_vcpu
> > - Added Reviewed-by tag for patch {1} 2. Added Shaoqin Huang's
> > Reviewed-by tags for Patches {2,3} 3. Added Tested-by Tag from Vishnu
> > Pajjuri to the patch-set 4. Dropped the ARM specific [Patch V4 10/10]
> > Link:
> > https://lore.kernel.org/qemu-devel/20231009203601.17584-1-
> salil.mehta@
> > huawei.com/
> >
> > Patch-set V3 -> V4
> > 1. Addressed David Hilderbrand's comments
> > - Fixed the wrong doc comment of kvm_park_vcpu API prototype
> > - Added Reviewed-by tags for patches {2,4}
> > Link:
> > https://lore.kernel.org/qemu-devel/20231009112812.10612-1-
> salil.mehta@
> > huawei.com/
> >
> > Patch-set V2 -> V3
> > 1. Addressed Jonathan Cameron's comments
> > - Fixed 'vcpu-id' type wrongly changed from 'unsigned long' to 'integer'
> > - Removed unnecessary use of variable 'vcpu_id' in kvm_park_vcpu
> > - Updated [Patch V2 3/10] commit-log with details of
> ACPI_CPU_SCAN_METHOD macro
> > - Updated [Patch V2 5/10] commit-log with details of conditional event
> handler method
> > - Added Reviewed-by tags for patches {2,3,4,6,7} 2. Addressed Gavin
> > Shan's comments
> > - Remove unnecessary use of variable 'vcpu_id' in kvm_par_vcpu
> > - Fixed return value in kvm_get_vcpu from -1 to -ENOENT
> > - Reset the value of 'gdb_num_g_regs' in
> gdb_unregister_coprocessor_all
> > - Fixed the kvm_{create,park}_vcpu prototypes docs
> > - Added Reviewed-by tags for patches {2,3,4,5,6,7,9,10} 3.
> > Addressed one earlier missed comment by Alex Bennée in RFC V1
> > - Added traces instead of DPRINTF in the newly added and some
> > existing functions
> > Link:
> > https://lore.kernel.org/qemu-devel/20230930001933.2660-1-
> salil.mehta@h
> > uawei.com/
> >
> > Patch-set V1 -> V2
> > 1. Addressed Alex Bennée's comments
> > - Refactored the kvm_create_vcpu logic to get rid of goto
> > - Added the docs for kvm_{create,park}_vcpu prototypes
> > - Splitted the gdbstub and AddressSpace destruction change into
> separate patches
> > - Added Reviewed-by tags for patches {2,10}
> > Link:
> > https://lore.kernel.org/qemu-devel/20230929124304.13672-1-
> salil.mehta@
> > huawei.com/
> >
> > References:
> >
> > [1]
> > https://lore.kernel.org/qemu-devel/20230926100436.28284-1-
> salil.mehta@
> > huawei.com/ [2]
> > https://lore.kernel.org/all/20230913163823.7880-1-
> james.morse@arm.com/
> > [3]
> > https://lore.kernel.org/qemu-devel/cover.1695697701.git.lixianglai@loo
> > ngson.cn/ [4]
> > https://lore.kernel.org/qemu-devel/20240613233639.202896-2-salil.mehta
> > @huawei.com/
> >
> > Salil Mehta (7):
> > accel/kvm: Extract common KVM vCPU {creation,parking} code
> > hw/acpi: Move CPU ctrl-dev MMIO region len macro to common header
> file
> > hw/acpi: Update ACPI GED framework to support vCPU Hotplug
> > hw/acpi: Update GED _EVT method AML with CPU scan
> > hw/acpi: Update CPUs AML with cpu-(ctrl)dev change
> > physmem: Add helper function to destroy CPU AddressSpace
> > gdbstub: Add helper function to unregister GDB register space
> >
> > accel/kvm/kvm-all.c | 95 +++++++++++++++++---------
> > accel/kvm/kvm-cpus.h | 1 -
> > accel/kvm/trace-events | 5 +-
> > docs/specs/acpi_hw_reduced_hotplug.rst | 3 +-
> > gdbstub/gdbstub.c | 13 ++++
> > hw/acpi/acpi-cpu-hotplug-stub.c | 6 ++
> > hw/acpi/cpu.c | 18 +++--
> > hw/acpi/generic_event_device.c | 27 ++++++++
> > hw/core/cpu-common.c | 4 +-
> > hw/i386/acpi-build.c | 3 +-
> > include/exec/cpu-common.h | 8 +++
> > include/exec/gdbstub.h | 6 ++
> > include/hw/acpi/cpu.h | 7 +-
> > include/hw/acpi/generic_event_device.h | 5 ++
> > include/hw/core/cpu.h | 1 +
> > include/sysemu/kvm.h | 25 +++++++
> > system/physmem.c | 29 ++++++++
> > 17 files changed, 212 insertions(+), 44 deletions(-)
> >
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* RE: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-15 14:14 ` Salil Mehta via
@ 2024-07-15 14:19 ` Salil Mehta via
2024-07-15 15:11 ` Igor Mammedov
0 siblings, 1 reply; 35+ messages in thread
From: Salil Mehta via @ 2024-07-15 14:19 UTC (permalink / raw)
To: Salil Mehta, Igor Mammedov
Cc: qemu-devel@nongnu.org, maz@kernel.org, jean-philippe@linaro.org,
Jonathan Cameron, lpieralisi@kernel.org, peter.maydell@linaro.org,
richard.henderson@linaro.org, andrew.jones@linux.dev,
david@redhat.com, philmd@linaro.org, eric.auger@redhat.com,
oliver.upton@linux.dev, pbonzini@redhat.com, mst@redhat.com,
will@kernel.org, gshan@redhat.com, rafael@kernel.org,
alex.bennee@linaro.org, linux@armlinux.org.uk,
darren@os.amperecomputing.com, ilkka@os.amperecomputing.com,
vishnu@os.amperecomputing.com, karl.heubaum@oracle.com,
miguel.luis@oracle.com, salil.mehta@opnsrc.net, zhukeqian,
wangxiongfeng (C), wangyanan (Y), jiakernel2@gmail.com,
maobibo@loongson.cn, lixianglai@loongson.cn, npiggin@gmail.com,
harshpb@linux.ibm.com, Linuxarm
> From: qemu-arm-bounces+salil.mehta=huawei.com@nongnu.org <qemu-
> arm-bounces+salil.mehta=huawei.com@nongnu.org> On Behalf Of Salil
> Mehta via
> Sent: Monday, July 15, 2024 3:14 PM
> To: Igor Mammedov <imammedo@redhat.com>
>
> Hi Igor,
>
> > From: Igor Mammedov <imammedo@redhat.com>
> > Sent: Monday, July 15, 2024 2:55 PM
> > To: Salil Mehta <salil.mehta@huawei.com>
> >
> > On Sat, 13 Jul 2024 19:25:09 +0100
> > Salil Mehta <salil.mehta@huawei.com> wrote:
> >
> > > [Note: References are present at the last after the revision
> > history] > > Virtual CPU hotplug support is being added across
> > various architectures [1][3].
> > > This series adds various code bits common across all architectures:
> > >
> > > 1. vCPU creation and Parking code refactor [Patch 1] 2. Update ACPI
> > > GED framework to support vCPU Hotplug [Patch 2,3] 3. ACPI CPUs AML
> > > code change [Patch 4,5] 4. Helper functions to support unrealization
> > > of CPU objects [Patch 6,7]
> >
> > with patch 1 and 3 fixed should be good to go.
> >
> > Salil,
> > Can you remind me what happened to migration part of this?
> > Ideally it should be a part of of this series as it should be common
> > for everything that uses GED and should be a conditional part of
> > GED's VMSTATE.
> >
> > If this series is just a common base and no actual hotplug on top of
> > it is merged in this release (provided patch 13 is fixed), I'm fine
> > with migration bits being a separate series on top.
> >
> > However if some machine would be introducing cpu hotplug in the same
> > release, then the migration part should be merged before it or be a
> > part that cpu hotplug series.
>
> We have tested Live/Pseudo Migration and it seem to work with the
> changes part of the architecture specific patch-set.
>
> Ampere: https://lore.kernel.org/all/e17e28ac-28c7-496f-b212-
> 2c9b552dbf63@amperemail.onmicrosoft.com/
> Oracle: https://lore.kernel.org/all/46D74D30-EE54-4AD2-8F0E-
> BA5627FAA63E@oracle.com/
>
>
> For ARM, please check below patch part of RFC V3 for changes related to
> migration:
> https://lore.kernel.org/qemu-devel/20240613233639.202896-15-
> salil.mehta@huawei.com/
Do you wish to move below change into this path-set and make it common
to all instead?
diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c
index 63226b0040..e92ce07955 100644
--- a/hw/acpi/generic_event_device.c
+++ b/hw/acpi/generic_event_device.c
@@ -333,6 +333,16 @@ static const VMStateDescription vmstate_memhp_state = {
}
};
+static const VMStateDescription vmstate_cpuhp_state = {
+ .name = "acpi-ged/cpuhp",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .fields = (VMStateField[]) {
+ VMSTATE_CPU_HOTPLUG(cpuhp_state, AcpiGedState),
+ VMSTATE_END_OF_LIST()
+ }
+};
+
static const VMStateDescription vmstate_ged_state = {
.name = "acpi-ged-state",
.version_id = 1,
@@ -381,6 +391,7 @@ static const VMStateDescription vmstate_acpi_ged = {
},
.subsections = (const VMStateDescription * const []) {
&vmstate_memhp_state,
+ &vmstate_cpuhp_state,
&vmstate_ghes_state,
NULL
}
Maybe I can add a separate patch for this in the end? Please confirm.
Thanks
Salil.
^ permalink raw reply related [flat|nested] 35+ messages in thread
* Re: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-15 14:19 ` Salil Mehta via
@ 2024-07-15 15:11 ` Igor Mammedov
2024-07-16 3:38 ` Salil Mehta
0 siblings, 1 reply; 35+ messages in thread
From: Igor Mammedov @ 2024-07-15 15:11 UTC (permalink / raw)
To: Salil Mehta
Cc: qemu-devel@nongnu.org, maz@kernel.org, jean-philippe@linaro.org,
Jonathan Cameron, lpieralisi@kernel.org, peter.maydell@linaro.org,
richard.henderson@linaro.org, andrew.jones@linux.dev,
david@redhat.com, philmd@linaro.org, eric.auger@redhat.com,
oliver.upton@linux.dev, pbonzini@redhat.com, mst@redhat.com,
will@kernel.org, gshan@redhat.com, rafael@kernel.org,
alex.bennee@linaro.org, linux@armlinux.org.uk,
darren@os.amperecomputing.com, ilkka@os.amperecomputing.com,
vishnu@os.amperecomputing.com, karl.heubaum@oracle.com,
miguel.luis@oracle.com, salil.mehta@opnsrc.net, zhukeqian,
wangxiongfeng (C), wangyanan (Y), jiakernel2@gmail.com,
maobibo@loongson.cn, lixianglai@loongson.cn, npiggin@gmail.com,
harshpb@linux.ibm.com, Linuxarm, peterx
On Mon, 15 Jul 2024 14:19:12 +0000
Salil Mehta <salil.mehta@huawei.com> wrote:
> > From: qemu-arm-bounces+salil.mehta=huawei.com@nongnu.org <qemu-
> > arm-bounces+salil.mehta=huawei.com@nongnu.org> On Behalf Of Salil
> > Mehta via
> > Sent: Monday, July 15, 2024 3:14 PM
> > To: Igor Mammedov <imammedo@redhat.com>
> >
> > Hi Igor,
> >
> > > From: Igor Mammedov <imammedo@redhat.com>
> > > Sent: Monday, July 15, 2024 2:55 PM
> > > To: Salil Mehta <salil.mehta@huawei.com>
> > >
> > > On Sat, 13 Jul 2024 19:25:09 +0100
> > > Salil Mehta <salil.mehta@huawei.com> wrote:
> > >
> > > > [Note: References are present at the last after the revision
> > > history] > > Virtual CPU hotplug support is being added across
> > > various architectures [1][3].
> > > > This series adds various code bits common across all architectures:
> > > >
> > > > 1. vCPU creation and Parking code refactor [Patch 1] 2. Update ACPI
> > > > GED framework to support vCPU Hotplug [Patch 2,3] 3. ACPI CPUs AML
> > > > code change [Patch 4,5] 4. Helper functions to support unrealization
> > > > of CPU objects [Patch 6,7]
> > >
> > > with patch 1 and 3 fixed should be good to go.
> > >
> > > Salil,
> > > Can you remind me what happened to migration part of this?
> > > Ideally it should be a part of of this series as it should be common
> > > for everything that uses GED and should be a conditional part of
> > > GED's VMSTATE.
> > >
> > > If this series is just a common base and no actual hotplug on top of
> > > it is merged in this release (provided patch 13 is fixed), I'm fine
> > > with migration bits being a separate series on top.
> > >
> > > However if some machine would be introducing cpu hotplug in the same
> > > release, then the migration part should be merged before it or be a
> > > part that cpu hotplug series.
> >
> > We have tested Live/Pseudo Migration and it seem to work with the
> > changes part of the architecture specific patch-set.
have you tested, migration from new QEMU to an older one (that doesn't have cpuhotplug builtin)?
> >
> > Ampere: https://lore.kernel.org/all/e17e28ac-28c7-496f-b212-
> > 2c9b552dbf63@amperemail.onmicrosoft.com/
> > Oracle: https://lore.kernel.org/all/46D74D30-EE54-4AD2-8F0E-
> > BA5627FAA63E@oracle.com/
> >
> >
> > For ARM, please check below patch part of RFC V3 for changes related to
> > migration:
> > https://lore.kernel.org/qemu-devel/20240613233639.202896-15-
> > salil.mehta@huawei.com/
>
>
> Do you wish to move below change into this path-set and make it common
> to all instead?
it would be the best to include this with here.
>
>
> diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c
> index 63226b0040..e92ce07955 100644
> --- a/hw/acpi/generic_event_device.c
> +++ b/hw/acpi/generic_event_device.c
> @@ -333,6 +333,16 @@ static const VMStateDescription vmstate_memhp_state = {
> }
> };
>
> +static const VMStateDescription vmstate_cpuhp_state = {
> + .name = "acpi-ged/cpuhp",
> + .version_id = 1,
> + .minimum_version_id = 1,
> + .fields = (VMStateField[]) {
> + VMSTATE_CPU_HOTPLUG(cpuhp_state, AcpiGedState),
> + VMSTATE_END_OF_LIST()
> + }
> +};
> +
> static const VMStateDescription vmstate_ged_state = {
> .name = "acpi-ged-state",
> .version_id = 1,
> @@ -381,6 +391,7 @@ static const VMStateDescription vmstate_acpi_ged = {
> },
> .subsections = (const VMStateDescription * const []) {
> &vmstate_memhp_state,
> + &vmstate_cpuhp_state,
I'm not migration guru but I believe this should be conditional
to avoid breaking cross-version migration.
See 679dd1a957d '.needed = vmstate_test_use_cpuhp. part
CCing Peter
> &vmstate_ghes_state,
> NULL
> }
>
> Maybe I can add a separate patch for this in the end? Please confirm.
>
> Thanks
> Salil.
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-15 15:11 ` Igor Mammedov
@ 2024-07-16 3:38 ` Salil Mehta
2024-07-16 9:52 ` Igor Mammedov
0 siblings, 1 reply; 35+ messages in thread
From: Salil Mehta @ 2024-07-16 3:38 UTC (permalink / raw)
To: Igor Mammedov, Salil Mehta
Cc: qemu-devel@nongnu.org, maz@kernel.org, jean-philippe@linaro.org,
Jonathan Cameron, lpieralisi@kernel.org, peter.maydell@linaro.org,
richard.henderson@linaro.org, andrew.jones@linux.dev,
david@redhat.com, philmd@linaro.org, eric.auger@redhat.com,
oliver.upton@linux.dev, pbonzini@redhat.com, mst@redhat.com,
will@kernel.org, gshan@redhat.com, rafael@kernel.org,
alex.bennee@linaro.org, linux@armlinux.org.uk,
darren@os.amperecomputing.com, ilkka@os.amperecomputing.com,
vishnu@os.amperecomputing.com, karl.heubaum@oracle.com,
miguel.luis@oracle.com, zhukeqian, wangxiongfeng (C),
wangyanan (Y), jiakernel2@gmail.com, maobibo@loongson.cn,
lixianglai@loongson.cn, npiggin@gmail.com, harshpb@linux.ibm.com,
Linuxarm, peterx
Hi Igor,
On 15/07/2024 15:11, Igor Mammedov wrote:
> On Mon, 15 Jul 2024 14:19:12 +0000
> Salil Mehta <salil.mehta@huawei.com> wrote:
>
>>> From: qemu-arm-bounces+salil.mehta=huawei.com@nongnu.org <qemu-
>>> arm-bounces+salil.mehta=huawei.com@nongnu.org> On Behalf Of Salil
>>> Mehta via
>>> Sent: Monday, July 15, 2024 3:14 PM
>>> To: Igor Mammedov <imammedo@redhat.com>
>>>
>>> Hi Igor,
>>>
>>> > From: Igor Mammedov <imammedo@redhat.com>
>>> > Sent: Monday, July 15, 2024 2:55 PM
>>> > To: Salil Mehta <salil.mehta@huawei.com>
>>> >
>>> > On Sat, 13 Jul 2024 19:25:09 +0100
>>> > Salil Mehta <salil.mehta@huawei.com> wrote:
>>> >
>>> > > [Note: References are present at the last after the revision
>>> > history] > > Virtual CPU hotplug support is being added across
>>> > various architectures [1][3].
>>> > > This series adds various code bits common across all architectures:
>>> > >
>>> > > 1. vCPU creation and Parking code refactor [Patch 1] 2. Update ACPI
>>> > > GED framework to support vCPU Hotplug [Patch 2,3] 3. ACPI CPUs AML
>>> > > code change [Patch 4,5] 4. Helper functions to support unrealization
>>> > > of CPU objects [Patch 6,7]
>>> >
>>> > with patch 1 and 3 fixed should be good to go.
>>> >
>>> > Salil,
>>> > Can you remind me what happened to migration part of this?
>>> > Ideally it should be a part of of this series as it should be common
>>> > for everything that uses GED and should be a conditional part of
>>> > GED's VMSTATE.
>>> >
>>> > If this series is just a common base and no actual hotplug on top of
>>> > it is merged in this release (provided patch 13 is fixed), I'm fine
>>> > with migration bits being a separate series on top.
>>> >
>>> > However if some machine would be introducing cpu hotplug in the same
>>> > release, then the migration part should be merged before it or be a
>>> > part that cpu hotplug series.
>>>
>>> We have tested Live/Pseudo Migration and it seem to work with the
>>> changes part of the architecture specific patch-set.
>
> have you tested, migration from new QEMU to an older one (that doesn't have cpuhotplug builtin)?
Just curious, how can we detect at source Qemu what version of the Qemu
destination is running. We require some sort of compatibility check but
then this is a problem not specific to CPU Hotplug?
We are not initializing CPU Hotplug VMSD in this patch-set. I was
wondering then how can a new machine attempt to migrate VMSD state from
new Qemu to older Qemu.
ARM vCPU Hotplug patches will be on top of this later in next Qemu cycle.
>
>>>
>>> Ampere: https://lore.kernel.org/all/e17e28ac-28c7-496f-b212-
>>> 2c9b552dbf63@amperemail.onmicrosoft.com/
>>> Oracle: https://lore.kernel.org/all/46D74D30-EE54-4AD2-8F0E-
>>> BA5627FAA63E@oracle.com/
>>>
>>>
>>> For ARM, please check below patch part of RFC V3 for changes related to
>>> migration:
>>> https://lore.kernel.org/qemu-devel/20240613233639.202896-15-
>>> salil.mehta@huawei.com/
>>
>>
>> Do you wish to move below change into this path-set and make it common
>> to all instead?
>
> it would be the best to include this with here.
>
>>
>>
>> diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c
>> index 63226b0040..e92ce07955 100644
>> --- a/hw/acpi/generic_event_device.c
>> +++ b/hw/acpi/generic_event_device.c
>> @@ -333,6 +333,16 @@ static const VMStateDescription vmstate_memhp_state = {
>> }
>> };
>>
>> +static const VMStateDescription vmstate_cpuhp_state = {
>> + .name = "acpi-ged/cpuhp",
>> + .version_id = 1,
>> + .minimum_version_id = 1,
>> + .fields = (VMStateField[]) {
>> + VMSTATE_CPU_HOTPLUG(cpuhp_state, AcpiGedState),
>> + VMSTATE_END_OF_LIST()
>> + }
>> +};
>> +
>> static const VMStateDescription vmstate_ged_state = {
>> .name = "acpi-ged-state",
>> .version_id = 1,
>> @@ -381,6 +391,7 @@ static const VMStateDescription vmstate_acpi_ged = {
>> },
>> .subsections = (const VMStateDescription * const []) {
>> &vmstate_memhp_state,
>> + &vmstate_cpuhp_state,
>
> I'm not migration guru but I believe this should be conditional
> to avoid breaking cross-version migration.
> See 679dd1a957d '.needed = vmstate_test_use_cpuhp. part
Sure, thanks for this. As I can see, the needed() function is used at
the source to decide if the state corresponding to a particular device
can be forwarded to the destination QEMU/VM. But how can this be used
to check for cross-version migration?
BTW, I've prepared V16. May I request a quick peek at:
https://github.com/salil-mehta/qemu/commits/virt-cpuhp-armv8/rfc-v3.arch.agnostic.v16/
Above does not have the suggested migration change yet. I can add it as
a separate path
Best regards,
Salil
>
> CCing Peter
>
>> &vmstate_ghes_state,
>> NULL
>> }
>>
>> Maybe I can add a separate patch for this in the end? Please confirm.
>>
>> Thanks
>> Salil.
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-16 3:38 ` Salil Mehta
@ 2024-07-16 9:52 ` Igor Mammedov
2024-07-16 11:43 ` Salil Mehta via
0 siblings, 1 reply; 35+ messages in thread
From: Igor Mammedov @ 2024-07-16 9:52 UTC (permalink / raw)
To: Salil Mehta
Cc: Salil Mehta, qemu-devel@nongnu.org, maz@kernel.org,
jean-philippe@linaro.org, Jonathan Cameron, lpieralisi@kernel.org,
peter.maydell@linaro.org, richard.henderson@linaro.org,
andrew.jones@linux.dev, david@redhat.com, philmd@linaro.org,
eric.auger@redhat.com, oliver.upton@linux.dev,
pbonzini@redhat.com, mst@redhat.com, will@kernel.org,
gshan@redhat.com, rafael@kernel.org, alex.bennee@linaro.org,
linux@armlinux.org.uk, darren@os.amperecomputing.com,
ilkka@os.amperecomputing.com, vishnu@os.amperecomputing.com,
karl.heubaum@oracle.com, miguel.luis@oracle.com, zhukeqian,
wangxiongfeng (C), wangyanan (Y), jiakernel2@gmail.com,
maobibo@loongson.cn, lixianglai@loongson.cn, npiggin@gmail.com,
harshpb@linux.ibm.com, Linuxarm, peterx
On Tue, 16 Jul 2024 03:38:29 +0000
Salil Mehta <salil.mehta@opnsrc.net> wrote:
> Hi Igor,
>
> On 15/07/2024 15:11, Igor Mammedov wrote:
> > On Mon, 15 Jul 2024 14:19:12 +0000
> > Salil Mehta <salil.mehta@huawei.com> wrote:
> >
> >>> From: qemu-arm-bounces+salil.mehta=huawei.com@nongnu.org <qemu-
> >>> arm-bounces+salil.mehta=huawei.com@nongnu.org> On Behalf Of Salil
> >>> Mehta via
> >>> Sent: Monday, July 15, 2024 3:14 PM
> >>> To: Igor Mammedov <imammedo@redhat.com>
> >>>
> >>> Hi Igor,
> >>>
> >>> > From: Igor Mammedov <imammedo@redhat.com>
> >>> > Sent: Monday, July 15, 2024 2:55 PM
> >>> > To: Salil Mehta <salil.mehta@huawei.com>
> >>> >
> >>> > On Sat, 13 Jul 2024 19:25:09 +0100
> >>> > Salil Mehta <salil.mehta@huawei.com> wrote:
> >>> >
> >>> > > [Note: References are present at the last after the revision
> >>> > history] > > Virtual CPU hotplug support is being added across
> >>> > various architectures [1][3].
> >>> > > This series adds various code bits common across all architectures:
> >>> > >
> >>> > > 1. vCPU creation and Parking code refactor [Patch 1] 2. Update ACPI
> >>> > > GED framework to support vCPU Hotplug [Patch 2,3] 3. ACPI CPUs AML
> >>> > > code change [Patch 4,5] 4. Helper functions to support unrealization
> >>> > > of CPU objects [Patch 6,7]
> >>> >
> >>> > with patch 1 and 3 fixed should be good to go.
> >>> >
> >>> > Salil,
> >>> > Can you remind me what happened to migration part of this?
> >>> > Ideally it should be a part of of this series as it should be common
> >>> > for everything that uses GED and should be a conditional part of
> >>> > GED's VMSTATE.
> >>> >
> >>> > If this series is just a common base and no actual hotplug on top of
> >>> > it is merged in this release (provided patch 13 is fixed), I'm fine
> >>> > with migration bits being a separate series on top.
> >>> >
> >>> > However if some machine would be introducing cpu hotplug in the same
> >>> > release, then the migration part should be merged before it or be a
> >>> > part that cpu hotplug series.
> >>>
> >>> We have tested Live/Pseudo Migration and it seem to work with the
> >>> changes part of the architecture specific patch-set.
> >
> > have you tested, migration from new QEMU to an older one (that doesn't have cpuhotplug builtin)?
>
>
> Just curious, how can we detect at source Qemu what version of the Qemu
> destination is running. We require some sort of compatibility check but
> then this is a problem not specific to CPU Hotplug?
it's usually managed by version machine types + compat settings for
machine/device.
> We are not initializing CPU Hotplug VMSD in this patch-set. I was
> wondering then how can a new machine attempt to migrate VMSD state from
> new Qemu to older Qemu.
If I'm not mistaken without VMSD it shouldn't explode, since CPUHP
code shouldn't create memory-regions that are migrated.
(If I recall correctly, mmio regions aren't going into migration stream)
> ARM vCPU Hotplug patches will be on top of this later in next Qemu cycle.
then it's fine to introduce VMSD later on, just make sure others
who adding cpu hotplug elsewhere also aware of it and pickup the same patch.
>
>
> >
> >>>
> >>> Ampere: https://lore.kernel.org/all/e17e28ac-28c7-496f-b212-
> >>> 2c9b552dbf63@amperemail.onmicrosoft.com/
> >>> Oracle: https://lore.kernel.org/all/46D74D30-EE54-4AD2-8F0E-
> >>> BA5627FAA63E@oracle.com/
> >>>
> >>>
> >>> For ARM, please check below patch part of RFC V3 for changes related to
> >>> migration:
> >>> https://lore.kernel.org/qemu-devel/20240613233639.202896-15-
> >>> salil.mehta@huawei.com/
> >>
> >>
> >> Do you wish to move below change into this path-set and make it common
> >> to all instead?
> >
> > it would be the best to include this with here.
> >
> >>
> >>
> >> diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c
> >> index 63226b0040..e92ce07955 100644
> >> --- a/hw/acpi/generic_event_device.c
> >> +++ b/hw/acpi/generic_event_device.c
> >> @@ -333,6 +333,16 @@ static const VMStateDescription vmstate_memhp_state = {
> >> }
> >> };
> >>
> >> +static const VMStateDescription vmstate_cpuhp_state = {
> >> + .name = "acpi-ged/cpuhp",
> >> + .version_id = 1,
> >> + .minimum_version_id = 1,
> >> + .fields = (VMStateField[]) {
> >> + VMSTATE_CPU_HOTPLUG(cpuhp_state, AcpiGedState),
> >> + VMSTATE_END_OF_LIST()
> >> + }
> >> +};
> >> +
> >> static const VMStateDescription vmstate_ged_state = {
> >> .name = "acpi-ged-state",
> >> .version_id = 1,
> >> @@ -381,6 +391,7 @@ static const VMStateDescription vmstate_acpi_ged = {
> >> },
> >> .subsections = (const VMStateDescription * const []) {
> >> &vmstate_memhp_state,
> >> + &vmstate_cpuhp_state,
> >
> > I'm not migration guru but I believe this should be conditional
> > to avoid breaking cross-version migration.
> > See 679dd1a957d '.needed = vmstate_test_use_cpuhp. part
>
>
> Sure, thanks for this. As I can see, the needed() function is used at
> the source to decide if the state corresponding to a particular device
> can be forwarded to the destination QEMU/VM. But how can this be used
> to check for cross-version migration?
what I'd do is to make sure that older machine types to not have
cpu hotplug enabled in supported events, and only machine that
has full support for hotplug enabled the bit. And then
machine_init depending on that would manage actual 'ged-event'
property.
Then later VMSD.needed would check ged-event to decide
if section should be used or omitted.
>
> BTW, I've prepared V16. May I request a quick peek at:
>
> https://github.com/salil-mehta/qemu/commits/virt-cpuhp-armv8/rfc-v3.arch.agnostic.v16/
looked at
hw/acpi: Update ACPI GED framework to support vCPU Hotplug
I get that ged_event loop in realize was copy-pasted from _EVT handler,
but that looks a bit complicated (though I won't object, it's matter of taste)
I'd prefer simpler condition than for() {} loop, and just use simpler 'if'
if (enabled_events & ACPI_GED_CPU_HOTPLUG_EVT) {
init cpu hp code
}
PS:
if you keep for loop, I'd replace error_report() + abort() with 'error_abort'
>
>
> Above does not have the suggested migration change yet. I can add it as
> a separate path
>
>
> Best regards,
> Salil
>
> >
> > CCing Peter
> >
> >> &vmstate_ghes_state,
> >> NULL
> >> }
> >>
> >> Maybe I can add a separate patch for this in the end? Please confirm.
> >>
> >> Thanks
> >> Salil.
> >
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* RE: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-16 9:52 ` Igor Mammedov
@ 2024-07-16 11:43 ` Salil Mehta via
2024-07-16 15:21 ` Igor Mammedov
0 siblings, 1 reply; 35+ messages in thread
From: Salil Mehta via @ 2024-07-16 11:43 UTC (permalink / raw)
To: Igor Mammedov, Salil Mehta
Cc: qemu-devel@nongnu.org, maz@kernel.org, jean-philippe@linaro.org,
Jonathan Cameron, lpieralisi@kernel.org, peter.maydell@linaro.org,
richard.henderson@linaro.org, andrew.jones@linux.dev,
david@redhat.com, philmd@linaro.org, eric.auger@redhat.com,
oliver.upton@linux.dev, pbonzini@redhat.com, mst@redhat.com,
will@kernel.org, gshan@redhat.com, rafael@kernel.org,
alex.bennee@linaro.org, linux@armlinux.org.uk,
darren@os.amperecomputing.com, ilkka@os.amperecomputing.com,
vishnu@os.amperecomputing.com, karl.heubaum@oracle.com,
miguel.luis@oracle.com, zhukeqian, wangxiongfeng (C),
wangyanan (Y), jiakernel2@gmail.com, maobibo@loongson.cn,
lixianglai@loongson.cn, npiggin@gmail.com, harshpb@linux.ibm.com,
Linuxarm, peterx@redhat.com
Hi Igor,
> From: Igor Mammedov <imammedo@redhat.com>
> Sent: Tuesday, July 16, 2024 10:52 AM
> To: Salil Mehta <salil.mehta@opnsrc.net>
>
> On Tue, 16 Jul 2024 03:38:29 +0000
> Salil Mehta <salil.mehta@opnsrc.net> wrote:
>
> > Hi Igor,
> >
> > On 15/07/2024 15:11, Igor Mammedov wrote:
> > > On Mon, 15 Jul 2024 14:19:12 +0000
> > > Salil Mehta <salil.mehta@huawei.com> wrote:
> > >
> > >>> From: qemu-arm-bounces+salil.mehta=huawei.com@nongnu.org <qemu-
> > >>> arm-bounces+salil.mehta=huawei.com@nongnu.org> On Behalf Of Salil
> > >>> Mehta via
> > >>> Sent: Monday, July 15, 2024 3:14 PM
> > >>> To: Igor Mammedov <imammedo@redhat.com>
> > >>>
> > >>> Hi Igor,
> > >>>
> > >>> > From: Igor Mammedov <imammedo@redhat.com>
> > >>> > Sent: Monday, July 15, 2024 2:55 PM
> > >>> > To: Salil Mehta <salil.mehta@huawei.com>
> > >>> >
> > >>> > On Sat, 13 Jul 2024 19:25:09 +0100
> > >>> > Salil Mehta <salil.mehta@huawei.com> wrote:
> > >>> >
> > >>> > > [Note: References are present at the last after the revision
> > >>> > history] > > Virtual CPU hotplug support is being added across
> > >>> > various architectures [1][3].
> > >>> > > This series adds various code bits common across all architectures:
> > >>> > >
> > >>> > > 1. vCPU creation and Parking code refactor [Patch 1] 2. Update ACPI
> > >>> > > GED framework to support vCPU Hotplug [Patch 2,3] 3. ACPI CPUs AML
> > >>> > > code change [Patch 4,5] 4. Helper functions to support unrealization
> > >>> > > of CPU objects [Patch 6,7]
> > >>> >
> > >>> > with patch 1 and 3 fixed should be good to go.
> > >>> >
> > >>> > Salil,
> > >>> > Can you remind me what happened to migration part of this?
> > >>> > Ideally it should be a part of of this series as it should be common
> > >>> > for everything that uses GED and should be a conditional part of
> > >>> > GED's VMSTATE.
> > >>> >
> > >>> > If this series is just a common base and no actual hotplug on top of
> > >>> > it is merged in this release (provided patch 13 is fixed), I'm fine
> > >>> > with migration bits being a separate series on top.
> > >>> >
> > >>> > However if some machine would be introducing cpu hotplug in the same
> > >>> > release, then the migration part should be merged before it or be a
> > >>> > part that cpu hotplug series.
> > >>>
> > >>> We have tested Live/Pseudo Migration and it seem to work with the
> > >>> changes part of the architecture specific patch-set.
> > >
> > > have you tested, migration from new QEMU to an older one (that doesn't have cpuhotplug builtin)?
> >
> >
> > Just curious, how can we detect at source Qemu what version of the
> > Qemu destination is running. We require some sort of compatibility
> > check but then this is a problem not specific to CPU Hotplug?
>
> it's usually managed by version machine types + compat settings for
> machine/device.
Ok. it looks to be a static checking at the source. I'm sure there must be
a way to dynamically do the same by negotiating the features i.e. only
enabling the common subset at the destination. I quickly skimmed the
migration code and I cannot find any thing like this being done as of now.
And this problem looks to be a pandoras box to me.
>
> > We are not initializing CPU Hotplug VMSD in this patch-set. I was
> > wondering then how can a new machine attempt to migrate VMSD state
> > from new Qemu to older Qemu.
>
> If I'm not mistaken without VMSD it shouldn't explode, since CPUHP code
> shouldn't create memory-regions that are migrated.
> (If I recall correctly, mmio regions aren't going into migration stream)
Correct.
>
> > ARM vCPU Hotplug patches will be on top of this later in next Qemu cycle.
> then it's fine to introduce VMSD later on, just make sure others who adding
> cpu hotplug elsewhere also aware of it and pickup the same patch.
Yes, thanks.
> > >>>
> > >>> Ampere: https://lore.kernel.org/all/e17e28ac-28c7-496f-b212-
> > >>> 2c9b552dbf63@amperemail.onmicrosoft.com/
> > >>> Oracle: https://lore.kernel.org/all/46D74D30-EE54-4AD2-8F0E-
> > >>> BA5627FAA63E@oracle.com/
> > >>>
> > >>>
> > >>> For ARM, please check below patch part of RFC V3 for changes related to
> > >>> migration:
> > >>> https://lore.kernel.org/qemu-devel/20240613233639.202896-15-
> > >>> salil.mehta@huawei.com/
> > >>
> > >>
> > >> Do you wish to move below change into this path-set and make it
> > >> common to all instead?
> > >
> > > it would be the best to include this with here.
> > >
> > >>
> > >>
> > >> diff --git a/hw/acpi/generic_event_device.c
> > >> b/hw/acpi/generic_event_device.c index 63226b0040..e92ce07955
> > >> 100644
> > >> --- a/hw/acpi/generic_event_device.c
> > >> +++ b/hw/acpi/generic_event_device.c
> > >> @@ -333,6 +333,16 @@ static const VMStateDescription
> vmstate_memhp_state = {
> > >> }
> > >> };
> > >>
> > >> +static const VMStateDescription vmstate_cpuhp_state = {
> > >> + .name = "acpi-ged/cpuhp",
> > >> + .version_id = 1,
> > >> + .minimum_version_id = 1,
> > >> + .fields = (VMStateField[]) {
> > >> + VMSTATE_CPU_HOTPLUG(cpuhp_state, AcpiGedState),
> > >> + VMSTATE_END_OF_LIST()
> > >> + }
> > >> +};
> > >> +
> > >> static const VMStateDescription vmstate_ged_state = {
> > >> .name = "acpi-ged-state",
> > >> .version_id = 1,
> > >> @@ -381,6 +391,7 @@ static const VMStateDescription
> vmstate_acpi_ged = {
> > >> },
> > >> .subsections = (const VMStateDescription * const []) {
> > >> &vmstate_memhp_state,
> > >> + &vmstate_cpuhp_state,
> > >
> > > I'm not migration guru but I believe this should be conditional to
> > > avoid breaking cross-version migration.
> > > See 679dd1a957d '.needed = vmstate_test_use_cpuhp. part
> >
> >
> > Sure, thanks for this. As I can see, the needed() function is used at
> > the source to decide if the state corresponding to a particular device
> > can be forwarded to the destination QEMU/VM. But how can this be used
> > to check for cross-version migration?
>
> what I'd do is to make sure that older machine types to not have cpu
> hotplug enabled in supported events, and only machine that has full
> support for hotplug enabled the bit. And then machine_init depending on
> that would manage actual 'ged-event'
> property.
>
> Then later VMSD.needed would check ged-event to decide if section should
> be used or omitted.
I understand this part, as discussed above this is not relevant to this patch-set
as we are not adding CPU Hotplug specific VMSD. But yes, I take your point and
will add above suggested change in the following patches in the next Qemu Cycle.
> > BTW, I've prepared V16. May I request a quick peek at:
> >
> > https://github.com/salil-mehta/qemu/commits/virt-cpuhp-armv8/rfc-
> v3.ar
> > ch.agnostic.v16/
>
> looked at
> hw/acpi: Update ACPI GED framework to support vCPU Hotplug
>
> I get that ged_event loop in realize was copy-pasted from _EVT handler, but
> that looks a bit complicated (though I won't object, it's matter of taste)
Yes, I did that intentionally to increase the code reuse which usually is encouraged.
This reduces the amount of re-testing required. But yes, there is a con side as
well to this approach if not done properly i.e. any issues in the older code also
get propagated in the newer code.
Generally we bank upon FFS function to get the next bit and use bit shifting to
deal with this kind of logic efficiently.
>
> I'd prefer simpler condition than for() {} loop, and just use simpler 'if'
>
> if (enabled_events & ACPI_GED_CPU_HOTPLUG_EVT) {
> init cpu hp code
> }
>
> PS:
> if you keep for loop, I'd replace error_report() + abort() with 'error_abort'
Maybe I should have looked a this earlier. I've missed to address this in V16.
Do you mean error_setg(&error_abort,....) kind of logic? it has been discouraged
in favor of assert() I think?
Can we live with this for now or do you want me to send V17 for this change?
Thanks
Salil.
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug
2024-07-16 11:43 ` Salil Mehta via
@ 2024-07-16 15:21 ` Igor Mammedov
0 siblings, 0 replies; 35+ messages in thread
From: Igor Mammedov @ 2024-07-16 15:21 UTC (permalink / raw)
To: Salil Mehta
Cc: Salil Mehta, qemu-devel@nongnu.org, maz@kernel.org,
jean-philippe@linaro.org, Jonathan Cameron, lpieralisi@kernel.org,
peter.maydell@linaro.org, richard.henderson@linaro.org,
andrew.jones@linux.dev, david@redhat.com, philmd@linaro.org,
eric.auger@redhat.com, oliver.upton@linux.dev,
pbonzini@redhat.com, mst@redhat.com, will@kernel.org,
gshan@redhat.com, rafael@kernel.org, alex.bennee@linaro.org,
linux@armlinux.org.uk, darren@os.amperecomputing.com,
ilkka@os.amperecomputing.com, vishnu@os.amperecomputing.com,
karl.heubaum@oracle.com, miguel.luis@oracle.com, zhukeqian,
wangxiongfeng (C), wangyanan (Y), jiakernel2@gmail.com,
maobibo@loongson.cn, lixianglai@loongson.cn, npiggin@gmail.com,
harshpb@linux.ibm.com, Linuxarm, peterx@redhat.com
On Tue, 16 Jul 2024 11:43:00 +0000
Salil Mehta <salil.mehta@huawei.com> wrote:
> Hi Igor,
>
> > From: Igor Mammedov <imammedo@redhat.com>
> > Sent: Tuesday, July 16, 2024 10:52 AM
> > To: Salil Mehta <salil.mehta@opnsrc.net>
> >
> > On Tue, 16 Jul 2024 03:38:29 +0000
> > Salil Mehta <salil.mehta@opnsrc.net> wrote:
> >
> > > Hi Igor,
> > >
> > > On 15/07/2024 15:11, Igor Mammedov wrote:
> > > > On Mon, 15 Jul 2024 14:19:12 +0000
> > > > Salil Mehta <salil.mehta@huawei.com> wrote:
> > > >
> > > >>> From: qemu-arm-bounces+salil.mehta=huawei.com@nongnu.org <qemu-
> > > >>> arm-bounces+salil.mehta=huawei.com@nongnu.org> On Behalf Of Salil
> > > >>> Mehta via
> > > >>> Sent: Monday, July 15, 2024 3:14 PM
> > > >>> To: Igor Mammedov <imammedo@redhat.com>
> > > >>>
> > > >>> Hi Igor,
> > > >>>
> > > >>> > From: Igor Mammedov <imammedo@redhat.com>
> > > >>> > Sent: Monday, July 15, 2024 2:55 PM
> > > >>> > To: Salil Mehta <salil.mehta@huawei.com>
> > > >>> >
> > > >>> > On Sat, 13 Jul 2024 19:25:09 +0100
> > > >>> > Salil Mehta <salil.mehta@huawei.com> wrote:
> > > >>> >
> > > >>> > > [Note: References are present at the last after the revision
> > > >>> > history] > > Virtual CPU hotplug support is being added across
> > > >>> > various architectures [1][3].
> > > >>> > > This series adds various code bits common across all architectures:
> > > >>> > >
> > > >>> > > 1. vCPU creation and Parking code refactor [Patch 1] 2. Update ACPI
> > > >>> > > GED framework to support vCPU Hotplug [Patch 2,3] 3. ACPI CPUs AML
> > > >>> > > code change [Patch 4,5] 4. Helper functions to support unrealization
> > > >>> > > of CPU objects [Patch 6,7]
> > > >>> >
> > > >>> > with patch 1 and 3 fixed should be good to go.
> > > >>> >
> > > >>> > Salil,
> > > >>> > Can you remind me what happened to migration part of this?
> > > >>> > Ideally it should be a part of of this series as it should be common
> > > >>> > for everything that uses GED and should be a conditional part of
> > > >>> > GED's VMSTATE.
> > > >>> >
> > > >>> > If this series is just a common base and no actual hotplug on top of
> > > >>> > it is merged in this release (provided patch 13 is fixed), I'm fine
> > > >>> > with migration bits being a separate series on top.
> > > >>> >
> > > >>> > However if some machine would be introducing cpu hotplug in the same
> > > >>> > release, then the migration part should be merged before it or be a
> > > >>> > part that cpu hotplug series.
> > > >>>
> > > >>> We have tested Live/Pseudo Migration and it seem to work with the
> > > >>> changes part of the architecture specific patch-set.
> > > >
> > > > have you tested, migration from new QEMU to an older one (that doesn't have cpuhotplug builtin)?
> > >
> > >
> > > Just curious, how can we detect at source Qemu what version of the
> > > Qemu destination is running. We require some sort of compatibility
> > > check but then this is a problem not specific to CPU Hotplug?
> >
> > it's usually managed by version machine types + compat settings for
> > machine/device.
>
> Ok. it looks to be a static checking at the source. I'm sure there must be
> a way to dynamically do the same by negotiating the features i.e. only
> enabling the common subset at the destination. I quickly skimmed the
> migration code and I cannot find any thing like this being done as of now.
> And this problem looks to be a pandoras box to me.
no dynamic negotiating as far as I'm aware.
We've managed to survive so far with static compat knobs
(with an occasional disaster along the way)
...
>
> Thanks
> Salil.
>
^ permalink raw reply [flat|nested] 35+ messages in thread
end of thread, other threads:[~2024-07-16 15:23 UTC | newest]
Thread overview: 35+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-07-13 18:25 [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Salil Mehta via
2024-07-13 18:25 ` [PATCH V15 1/7] accel/kvm: Extract common KVM vCPU {creation, parking} code Salil Mehta via
2024-07-15 12:49 ` [PATCH V15 1/7] accel/kvm: Extract common KVM vCPU {creation,parking} code Igor Mammedov
2024-07-15 13:28 ` Igor Mammedov
2024-07-15 13:57 ` Salil Mehta via
2024-07-13 18:25 ` [PATCH V15 2/7] hw/acpi: Move CPU ctrl-dev MMIO region len macro to common header file Salil Mehta via
2024-07-15 12:50 ` Igor Mammedov
2024-07-13 18:25 ` [PATCH V15 3/7] hw/acpi: Update ACPI GED framework to support vCPU Hotplug Salil Mehta via
2024-07-15 13:12 ` Igor Mammedov
2024-07-15 13:55 ` Salil Mehta via
2024-07-13 18:25 ` [PATCH V15 4/7] hw/acpi: Update GED _EVT method AML with CPU scan Salil Mehta via
2024-07-15 12:55 ` Igor Mammedov
2024-07-13 18:25 ` [PATCH V15 5/7] hw/acpi: Update CPUs AML with cpu-(ctrl)dev change Salil Mehta via
2024-07-15 13:39 ` Igor Mammedov
2024-07-13 18:25 ` [PATCH V15 6/7] physmem: Add helper function to destroy CPU AddressSpace Salil Mehta via
2024-07-13 18:25 ` [PATCH V15 7/7] gdbstub: Add helper function to unregister GDB register space Salil Mehta via
2024-07-15 13:40 ` Igor Mammedov
2024-07-15 6:11 ` [PATCH V15 0/7] Add architecture agnostic code to support vCPU Hotplug Zhao Liu
2024-07-15 8:45 ` Salil Mehta via
2024-07-15 11:03 ` Vishnu Pajjuri
2024-07-15 11:07 ` Salil Mehta
2024-07-15 11:13 ` Michael S. Tsirkin
2024-07-15 11:27 ` Salil Mehta via
2024-07-15 11:33 ` Michael S. Tsirkin
2024-07-15 11:35 ` Salil Mehta via
2024-07-15 13:55 ` Igor Mammedov
2024-07-15 11:14 ` Salil Mehta via
2024-07-15 13:54 ` Igor Mammedov
2024-07-15 14:14 ` Salil Mehta via
2024-07-15 14:19 ` Salil Mehta via
2024-07-15 15:11 ` Igor Mammedov
2024-07-16 3:38 ` Salil Mehta
2024-07-16 9:52 ` Igor Mammedov
2024-07-16 11:43 ` Salil Mehta via
2024-07-16 15:21 ` Igor Mammedov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).