[Qemu-devel] [PATCH 00/21 v5] target-i386: CPU hot-add with cpu-add QMP command

[Qemu-devel] [PATCH 00/21 v5] target-i386: CPU hot-add with cpu-add QMP command

[Igor Mammedov]

Implements alternative way for hot-adding CPU using cpu-add QMP command,
wich could be useful until it would be possible to add CPUs via device_add.

All patches except the last are also applicable to device_add aprroach.

To hot-add CPU use following command from qmp-shell:
 cpu-add id=[0..max-cpus - 1)

git tree for testing: https://github.com/imammedo/qemu/tree/cpu_add.v5

based on qom-cpu-nex tree due to dependency on "mc146818rtc: QOM'ify"

v5->v4:
  * style fixes
  * new helper qemu_for_each_cpu()
  * switch to qemu_for_each_cpu() in cpu_exists()
  * "pc: update rtc ..." patch make depend it on "mc146818rtc: QOM'ify"
    and use QOM cast style
  * call CPU added notifier right before CPU becomes runable
  * s/resume_vcpu/cpu_resume/
  * acpi/piix4: add spec documentation for QEMU<->Seabios CPU hotplug
    interface
  * use error_propagate() in pc_new_cpu()
  * skip cpu_exists() check in apic-id property setter if new value is
    the same as current
  * embed icc-bus inside icc-bridge and use qbus_create_inplace()
  * move include/hw/i386/icc_bus.h into include/hw/cpu/
  * make missing icc-bus fatal error for softmmu target
  * split "move APIC to ICC bus" and "move IOAPIC to ICC bus" on smaller
    patches
  * use qdev_get_parent_bus() to get parent bus
  * split "add cpu-add command..." on smaller patches

v4->v3:
  * 'id' in cpu-add command will be a thread number instead of APIC ID
  * split off resume_vcpu() into separate patch
  * move notifier from rtc code into pc.c

v2->v3:
  * use local error & propagate_error() instead of operating on
    passed in errp in several places
  * replace CPUClass.get_firmware_id() with CPUClass.get_arch_id()
  * leave IOAPIC creation to board and just set bus to icc-bus
  * include kvm-stub.o in cpu libary if no KVM is configured
  * create resume_vcpu() stub and include it in libqemustub,
    and use it directly instead of CPU method
  * acpi_piix4: s/cpu_add_notifier/cpu_added_notifier/

v1->v2:
  * generalize cpu sync to KVM, resume and hot-plug notification and
    invoke them form CPUClass, to make available to all targets.
  * introduce cpu_exists() and CPUClass.get_firmware_id() and use
    the last one in acpi_piix to make code target independent.
  * move IOAPIC to ICC bus, it was suggested and easy to convert.
  * leave kvmvapic as SysBusDevice, it doesn't affect hot-plug and
    created only once for all APIC instances. I haven't found yet
    good/clean enough way to convert it to ICCDevice. May be follow-up
    though.
  * split one big ICC patch into several, one per converted device
  * add cpu_hot_add hook to machine and implement it for target-i386,
    instead of adding stabs. Could be used by other targets to
    implement cpu-add.
  * pre-allocate links<CPU> for all possible CPUs and make them available
    at /machine/icc-bridge/cpu[0..N] QOM path, so users could find out
    possible/free CPU IDs to use in cpu-add command.

CC: pbonzini@redhat.com
CC: gleb@redhat.com
CC: mst@redhat.com
CC: afaerber@suse.de
CC: ehabkost@redhat.com

Igor Mammedov (21):
  cpu: make kvm-stub.o a part of CPU library
  cpu: call cpu_synchronize_post_init() from CPUClass.realize() if
    hotplugged
  introduce cpu_resume(), for single CPU
  cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is
    hot-plugged
  introduce CPU hot-plug notifier
  target-i386: pc: update rtc_cmos on CPU hot-plug
  cpu: introduce get_arch_id() method and override it for target-i386
  exec: add qemu_for_each_cpu
  cpu: add helper cpu_exists(), to check if CPU with specified id
    exists
  acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest
  target-i386: introduce apic-id property
  target-i386: introduce ICC bus/device/bridge
  target-i386: cpu: attach ICC bus to CPU on its creation
  target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE
  target-i386: kvmvapic: make expilict dependency on sysbus.h
  target-i386: move APIC to ICC bus
  introduce memory_region_get_address() and use it in kvm/ioapic
  target-i386: move IOAPIC to ICC bus
  add hot_add_cpu hook to QEMUMachine and export machine_args
  target-i386: implement machine->hot_add_cpu hook
  QMP: add cpu-add command

 MAINTAINERS                        |    6 ++
 Makefile.target                    |   14 ++--
 cpus.c                             |   24 +++++---
 default-configs/i386-softmmu.mak   |    1 +
 default-configs/x86_64-softmmu.mak |    1 +
 docs/specs/acpi_cpu_hotplug.txt    |   22 +++++++
 exec.c                             |   10 +++
 hw/acpi/piix4.c                    |  117 ++++++++++++++++++++++++++++++++++-
 hw/cpu/Makefile.objs               |    1 +
 hw/cpu/icc_bus.c                   |  120 ++++++++++++++++++++++++++++++++++++
 hw/i386/kvm/apic.c                 |    2 +-
 hw/i386/kvm/ioapic.c               |    2 +-
 hw/i386/kvmvapic.c                 |    1 +
 hw/i386/pc.c                       |   93 ++++++++++++++++++++++++++--
 hw/i386/pc_piix.c                  |    7 ++
 hw/i386/pc_q35.c                   |    7 ++
 hw/intc/apic.c                     |    2 +-
 hw/intc/apic_common.c              |   18 ++++--
 hw/intc/ioapic_common.c            |   14 +++-
 hw/timer/mc146818rtc.c             |    7 ++
 hw/xen/xen_apic.c                  |    2 +-
 include/exec/memory.h              |   10 +++
 include/hw/boards.h                |    3 +
 include/hw/cpu/icc_bus.h           |   63 +++++++++++++++++++
 include/hw/i386/apic_internal.h    |    8 +--
 include/hw/i386/ioapic_internal.h  |    6 +-
 include/hw/timer/mc146818rtc.h     |    1 +
 include/qom/cpu.h                  |   27 ++++++++
 include/sysemu/kvm.h               |   24 ++++----
 include/sysemu/sysemu.h            |    3 +
 kvm-all.c                          |    1 +
 kvm-stub.c                         |    8 ++-
 memory.c                           |   11 +++
 qapi-schema.json                   |   11 +++
 qmp-commands.hx                    |   23 +++++++
 qmp.c                              |   10 +++
 qom/cpu.c                          |   49 +++++++++++++++
 stubs/Makefile.objs                |    1 +
 stubs/cpus.c                       |    5 ++
 target-i386/cpu.c                  |   83 +++++++++++++++++++++----
 target-i386/cpu.h                  |    1 +
 vl.c                               |    7 ++-
 42 files changed, 752 insertions(+), 74 deletions(-)
 create mode 100644 docs/specs/acpi_cpu_hotplug.txt
 create mode 100644 hw/cpu/icc_bus.c
 create mode 100644 include/hw/cpu/icc_bus.h
 create mode 100644 stubs/cpus.c

[Qemu-devel] [PATCH 01/21] cpu: make kvm-stub.o a part of CPU library

[Igor Mammedov]

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
---
v3:
  put ifndef around msi.h in kvm-stub.c instead of in msi.h

v2:
  remove unnecessary kvm-stub.o from "System emulator target" section
---
 Makefile.target      |   14 +++++++-------
 include/sysemu/kvm.h |    4 ++--
 kvm-stub.c           |    7 ++++++-
 3 files changed, 15 insertions(+), 10 deletions(-)

diff --git a/Makefile.target b/Makefile.target
index 2636103..0783c13 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -60,6 +60,12 @@ all: $(PROGS) stap
 # Dummy command so that make thinks it has done something
 	@true
 
+CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
+CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
+CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
+CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
+CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
+
 #########################################################
 # cpu emulator library
 obj-y = exec.o translate-all.o cpu-exec.o
@@ -70,6 +76,7 @@ obj-y += fpu/softfloat.o
 obj-y += target-$(TARGET_BASE_ARCH)/
 obj-y += disas.o
 obj-$(CONFIG_GDBSTUB_XML) += gdbstub-xml.o
+obj-$(CONFIG_NO_KVM) += kvm-stub.o
 
 #########################################################
 # Linux user emulator target
@@ -98,18 +105,11 @@ endif #CONFIG_BSD_USER
 #########################################################
 # System emulator target
 ifdef CONFIG_SOFTMMU
-CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
-CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
-CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
-CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
-CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
-
 obj-y += arch_init.o cpus.o monitor.o gdbstub.o balloon.o ioport.o
 obj-y += qtest.o
 obj-y += hw/
 obj-$(CONFIG_FDT) += device_tree.o
 obj-$(CONFIG_KVM) += kvm-all.o
-obj-$(CONFIG_NO_KVM) += kvm-stub.o
 obj-y += memory.o savevm.o cputlb.o
 obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
 obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 75bd7d9..2bc1f6b 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
 #if !defined(CONFIG_USER_ONLY)
 void *kvm_vmalloc(ram_addr_t size);
 void *kvm_arch_vmalloc(ram_addr_t size);
-void kvm_setup_guest_memory(void *start, size_t size);
+#endif
 
+void kvm_setup_guest_memory(void *start, size_t size);
 void kvm_flush_coalesced_mmio_buffer(void);
-#endif
 
 int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
                           target_ulong len, int type);
diff --git a/kvm-stub.c b/kvm-stub.c
index 5f52186..b228378 100644
--- a/kvm-stub.c
+++ b/kvm-stub.c
@@ -12,10 +12,13 @@
 
 #include "qemu-common.h"
 #include "hw/hw.h"
-#include "hw/pci/msi.h"
 #include "cpu.h"
 #include "sysemu/kvm.h"
 
+#ifndef CONFIG_USER_ONLY
+#include "hw/pci/msi.h"
+#endif
+
 KVMState *kvm_state;
 bool kvm_kernel_irqchip;
 bool kvm_async_interrupts_allowed;
@@ -111,6 +114,7 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
+#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;
@@ -134,3 +138,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
 {
     return -ENOSYS;
 }
+#endif
-- 
1.7.1

Re: [Qemu-devel] [PATCH 01/21] cpu: make kvm-stub.o a part of CPU library

[Andreas Färber]

Am 23.04.2013 10:29, schrieb Igor Mammedov:
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
> ---
> v3:
>   put ifndef around msi.h in kvm-stub.c instead of in msi.h
> 
> v2:
>   remove unnecessary kvm-stub.o from "System emulator target" section

Thanks, applied to qom-cpu (with jointly reworded commit message):
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

[Qemu-devel] [PATCH 02/21] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

[Igor Mammedov]

... to synchronize CPU state to KVM

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
---
v2:
  * linking kvm-stub.o to *-user target moved in separate patch
---
 include/sysemu/kvm.h |   20 ++++++++++----------
 kvm-all.c            |    1 +
 kvm-stub.c           |    1 +
 qom/cpu.c            |    4 ++++
 vl.c                 |    1 -
 5 files changed, 16 insertions(+), 11 deletions(-)

diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 2bc1f6b..9735c1d 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -250,8 +250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension);
 uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
                                       uint32_t index, int reg);
 void kvm_cpu_synchronize_state(CPUArchState *env);
-void kvm_cpu_synchronize_post_reset(CPUState *cpu);
-void kvm_cpu_synchronize_post_init(CPUState *cpu);
 
 /* generic hooks - to be moved/refactored once there are more users */
 
@@ -262,6 +260,16 @@ static inline void cpu_synchronize_state(CPUArchState *env)
     }
 }
 
+#if !defined(CONFIG_USER_ONLY)
+int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
+                                       hwaddr *phys_addr);
+#endif
+
+#endif /* NEED_CPU_H */
+
+void kvm_cpu_synchronize_post_reset(CPUState *cpu);
+void kvm_cpu_synchronize_post_init(CPUState *cpu);
+
 static inline void cpu_synchronize_post_reset(CPUState *cpu)
 {
     if (kvm_enabled()) {
@@ -276,14 +284,6 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
     }
 }
 
-
-#if !defined(CONFIG_USER_ONLY)
-int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
-                                       hwaddr *phys_addr);
-#endif
-
-#endif
-
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg);
 int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg);
 void kvm_irqchip_release_virq(KVMState *s, int virq);
diff --git a/kvm-all.c b/kvm-all.c
index 2d92721..f6c0f4a 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
     KVM_CAP_INFO(USER_MEMORY),
diff --git a/kvm-stub.c b/kvm-stub.c
index b228378..b2c8f9b 100644
--- a/kvm-stub.c
+++ b/kvm-stub.c
@@ -25,6 +25,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 int kvm_init_vcpu(CPUState *cpu)
 {
diff --git a/qom/cpu.c b/qom/cpu.c
index e242dcb..0c76712 100644
--- a/qom/cpu.c
+++ b/qom/cpu.c
@@ -20,6 +20,7 @@
 
 #include "qom/cpu.h"
 #include "qemu-common.h"
+#include "sysemu/kvm.h"
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -57,6 +58,9 @@ static ObjectClass *cpu_common_class_by_name(const char *cpu_model)
 
 static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
+    if (dev->hotplugged) {
+        cpu_synchronize_post_init(CPU(dev));
+    }
 }
 
 static void cpu_class_init(ObjectClass *klass, void *data)
diff --git a/vl.c b/vl.c
index 6caa5f4..5612c33 100644
--- a/vl.c
+++ b/vl.c
@@ -267,7 +267,6 @@ static NotifierList machine_init_done_notifiers =
     NOTIFIER_LIST_INITIALIZER(machine_init_done_notifiers);
 
 static bool tcg_allowed = true;
-bool kvm_allowed;
 bool xen_allowed;
 uint32_t xen_domid;
 enum xen_mode xen_mode = XEN_EMULATE;
-- 
1.7.1

Re: [Qemu-devel] [PATCH 02/21] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

[Andreas Färber]

Am 23.04.2013 10:29, schrieb Igor Mammedov:
> ... to synchronize CPU state to KVM
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
> ---
> v2:
>   * linking kvm-stub.o to *-user target moved in separate patch

Thanks, applied to qom-cpu:
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 02/21] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

[Andreas Färber]

Am 23.04.2013 17:59, schrieb Andreas Färber:
> Am 23.04.2013 10:29, schrieb Igor Mammedov:
>> ... to synchronize CPU state to KVM
>>
>> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
>> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
>> ---
>> v2:
>>   * linking kvm-stub.o to *-user target moved in separate patch
> 
> Thanks, applied to qom-cpu:
> https://github.com/afaerber/qemu-cpu/commits/qom-cpu

To facilitate conflict resolution for future patches, I have modified
this to match other functions in there and to spare refactorings:

diff --git a/qom/cpu.c b/qom/cpu.c
index 0c76712..66f7c00 100644
--- a/qom/cpu.c
+++ b/qom/cpu.c
@@ -58,8 +58,10 @@ static ObjectClass *cpu_common_class_by_name(const
char *cpu_model)

 static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
+    CPUState *cpu = CPU(dev);
+
     if (dev->hotplugged) {
-        cpu_synchronize_post_init(CPU(dev));
+        cpu_synchronize_post_init(cpu);
     }
 }


Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 02/21] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

[Igor Mammedov]

On Wed, 24 Apr 2013 14:08:45 +0200
Andreas Färber <afaerber@suse.de> wrote:

> Am 23.04.2013 17:59, schrieb Andreas Färber:
> > Am 23.04.2013 10:29, schrieb Igor Mammedov:
> >> ... to synchronize CPU state to KVM
> >>
> >> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> >> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
> >> ---
> >> v2:
> >>   * linking kvm-stub.o to *-user target moved in separate patch
> > 
> > Thanks, applied to qom-cpu:
> > https://github.com/afaerber/qemu-cpu/commits/qom-cpu
> 
> To facilitate conflict resolution for future patches, I have modified
> this to match other functions in there and to spare refactorings:
> 
> diff --git a/qom/cpu.c b/qom/cpu.c
> index 0c76712..66f7c00 100644
> --- a/qom/cpu.c
> +++ b/qom/cpu.c
> @@ -58,8 +58,10 @@ static ObjectClass *cpu_common_class_by_name(const
> char *cpu_model)
> 
>  static void cpu_common_realizefn(DeviceState *dev, Error **errp)
>  {
> +    CPUState *cpu = CPU(dev);
> +
>      if (dev->hotplugged) {
> -        cpu_synchronize_post_init(CPU(dev));
> +        cpu_synchronize_post_init(cpu);
>      }
>  }
> 

Fixed conflicts in 4-5/21,
available at https://github.com/imammedo/qemu/commits/cpu_set.WIP

> Andreas
> 

[Qemu-devel] [PATCH 03/21] introduce cpu_resume(), for single CPU

[Igor Mammedov]

also add stub for it, to make possible to use it in qom/cpu.c,
which is shared with user emulators.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v2:
 * rename resume_vcpu() to cpu_resume()
 * remove comment about neccessity to call cpu_synchronize_post_init()
   which is applicable only to CPU creation. But not to generic CPU
   resume.

Note:
  it will be re-used from qom/cpu.c by hotpluged CPU
---
 cpus.c              |   11 ++++++++---
 include/qom/cpu.h   |    7 +++++++
 stubs/Makefile.objs |    1 +
 stubs/cpus.c        |    5 +++++
 4 files changed, 21 insertions(+), 3 deletions(-)
 create mode 100644 stubs/cpus.c

diff --git a/cpus.c b/cpus.c
index 5a98a37..1d88761 100644
--- a/cpus.c
+++ b/cpus.c
@@ -993,6 +993,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void cpu_resume(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        cpu_resume(pcpu);
         penv = penv->next_cpu;
     }
 }
diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index 3664a1b..d6a0e80 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -256,5 +256,12 @@ void cpu_interrupt(CPUState *cpu, int mask);
  */
 void cpu_reset_interrupt(CPUState *cpu, int mask);
 
+/**
+ * cpu_resume:
+ * @cpu: The CPU to resume.
+ *
+ * Resumes CPU, i.e. puts CPU into runable state.
+ */
+void cpu_resume(CPUState *cpu);
 
 #endif
diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs
index 9c55b34..03dff20 100644
--- a/stubs/Makefile.objs
+++ b/stubs/Makefile.objs
@@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
 stub-obj-y += vm-stop.o
 stub-obj-y += vmstate.o
 stub-obj-$(CONFIG_WIN32) += fd-register.o
+stub-obj-y += cpus.o
diff --git a/stubs/cpus.c b/stubs/cpus.c
new file mode 100644
index 0000000..37000dd
--- /dev/null
+++ b/stubs/cpus.c
@@ -0,0 +1,5 @@
+#include "qom/cpu.h"
+
+void cpu_resume(CPUState *cpu)
+{
+}
-- 
1.7.1

Re: [Qemu-devel] [PATCH 03/21] introduce cpu_resume(), for single CPU

[Andreas Färber]

Am 23.04.2013 10:29, schrieb Igor Mammedov:
> also add stub for it, to make possible to use it in qom/cpu.c,
> which is shared with user emulators.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
> v2:
>  * rename resume_vcpu() to cpu_resume()
>  * remove comment about neccessity to call cpu_synchronize_post_init()
>    which is applicable only to CPU creation. But not to generic CPU
>    resume.

Thanks, applied to qom-cpu (with a minor change below):
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

> 
> Note:
>   it will be re-used from qom/cpu.c by hotpluged CPU
> ---
>  cpus.c              |   11 ++++++++---
>  include/qom/cpu.h   |    7 +++++++
>  stubs/Makefile.objs |    1 +
>  stubs/cpus.c        |    5 +++++
>  4 files changed, 21 insertions(+), 3 deletions(-)
>  create mode 100644 stubs/cpus.c
> 
> diff --git a/cpus.c b/cpus.c
> index 5a98a37..1d88761 100644
> --- a/cpus.c
> +++ b/cpus.c
> @@ -993,6 +993,13 @@ void pause_all_vcpus(void)
>      }
>  }
>  
> +void cpu_resume(CPUState *cpu)
> +{
> +    cpu->stop = false;
> +    cpu->stopped = false;
> +    qemu_cpu_kick(cpu);
> +}
> +
>  void resume_all_vcpus(void)
>  {
>      CPUArchState *penv = first_cpu;
> @@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
>      qemu_clock_enable(vm_clock, true);
>      while (penv) {
>          CPUState *pcpu = ENV_GET_CPU(penv);
> -        pcpu->stop = false;
> -        pcpu->stopped = false;
> -        qemu_cpu_kick(pcpu);
> +        cpu_resume(pcpu);
>          penv = penv->next_cpu;
>      }
>  }
> diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> index 3664a1b..d6a0e80 100644
> --- a/include/qom/cpu.h
> +++ b/include/qom/cpu.h
> @@ -256,5 +256,12 @@ void cpu_interrupt(CPUState *cpu, int mask);
>   */
>  void cpu_reset_interrupt(CPUState *cpu, int mask);
>  
> +/**
> + * cpu_resume:
> + * @cpu: The CPU to resume.
> + *
> + * Resumes CPU, i.e. puts CPU into runable state.

"runnable"

> + */
> +void cpu_resume(CPUState *cpu);
>  
>  #endif

I note that you're dropping a white line, but okay.

> diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs
> index 9c55b34..03dff20 100644
> --- a/stubs/Makefile.objs
> +++ b/stubs/Makefile.objs
> @@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
>  stub-obj-y += vm-stop.o
>  stub-obj-y += vmstate.o
>  stub-obj-$(CONFIG_WIN32) += fd-register.o
> +stub-obj-y += cpus.o
> diff --git a/stubs/cpus.c b/stubs/cpus.c
> new file mode 100644
> index 0000000..37000dd
> --- /dev/null
> +++ b/stubs/cpus.c
> @@ -0,0 +1,5 @@
> +#include "qom/cpu.h"
> +
> +void cpu_resume(CPUState *cpu)
> +{
> +}

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

[Qemu-devel] [PATCH 04/21] cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is hot-plugged

[Igor Mammedov]

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v2:
  s/resume_vcpu/cpu_resume/
v2:
 * remove unnecessary now sysemu/cpus.h header
 * move out introducing resume_vcpu() into a separate patch
---
 qom/cpu.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/qom/cpu.c b/qom/cpu.c
index 0c76712..abb20d0 100644
--- a/qom/cpu.c
+++ b/qom/cpu.c
@@ -60,6 +60,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
+        cpu_resume(CPU(dev));
     }
 }
 
-- 
1.7.1

Re: [Qemu-devel] [PATCH 04/21] cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is hot-plugged

[Andreas Färber]

Am 23.04.2013 10:29, schrieb Igor Mammedov:
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
> v2:
>   s/resume_vcpu/cpu_resume/
> v2:
>  * remove unnecessary now sysemu/cpus.h header
>  * move out introducing resume_vcpu() into a separate patch
> ---
>  qom/cpu.c |    1 +
>  1 files changed, 1 insertions(+), 0 deletions(-)

Thanks, applied to qom-cpu (rebased on introduction of cpu variable and
aligning commit messages wrt realize):

https://github.com/afaerber/qemu-cpu/commits/qom-cpu

> diff --git a/qom/cpu.c b/qom/cpu.c
> index 0c76712..abb20d0 100644
> --- a/qom/cpu.c
> +++ b/qom/cpu.c
> @@ -60,6 +60,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
>  {
>      if (dev->hotplugged) {
>          cpu_synchronize_post_init(CPU(dev));
> +        cpu_resume(CPU(dev));

Having introduced CPUState *cpu in a preceding patch, we can avoid the
duplicate CPU() cast here.

>      }
>  }
>  

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

[Qemu-devel] [PATCH 05/21] introduce CPU hot-plug notifier

[Igor Mammedov]

hot-add CPU event will be distributed to acpi_piix4 and rtc_cmos

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v3:
  * call notifier right before CPU becomes runable
v2:
  * move notifier to qom/cpu.c and call it from CPUClass.realize() on hotplug
  * remove get_firmware_id() since it belong to other patch
---
 include/sysemu/sysemu.h |    3 +++
 qom/cpu.c               |   12 ++++++++++++
 2 files changed, 15 insertions(+), 0 deletions(-)

diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 6578782..a8c3de1 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const QDict *qdict);
 /* generic hotplug */
 void drive_hot_add(Monitor *mon, const QDict *qdict);
 
+/* CPU hotplug */
+void qemu_register_cpu_added_notifier(Notifier *notifier);
+
 /* pcie aer error injection */
 void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
 int do_pcie_aer_inject_error(Monitor *mon,
diff --git a/qom/cpu.c b/qom/cpu.c
index abb20d0..9f78114 100644
--- a/qom/cpu.c
+++ b/qom/cpu.c
@@ -21,6 +21,17 @@
 #include "qom/cpu.h"
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
+#include "qemu/notify.h"
+#include "sysemu/sysemu.h"
+
+/* CPU hot-plug notifiers */
+static NotifierList cpu_added_notifiers =
+    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
+
+void qemu_register_cpu_added_notifier(Notifier *notifier)
+{
+    notifier_list_add(&cpu_added_notifiers, notifier);
+}
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -60,6 +71,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
+        notifier_list_notify(&cpu_added_notifiers, dev);
         cpu_resume(CPU(dev));
     }
 }
-- 
1.7.1

Re: [Qemu-devel] [PATCH 05/21] introduce CPU hot-plug notifier

[Andreas Färber]

Am 23.04.2013 10:29, schrieb Igor Mammedov:
> hot-add CPU event will be distributed to acpi_piix4 and rtc_cmos
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
> v3:
>   * call notifier right before CPU becomes runable
> v2:
>   * move notifier to qom/cpu.c and call it from CPUClass.realize() on hotplug
>   * remove get_firmware_id() since it belong to other patch
> ---
>  include/sysemu/sysemu.h |    3 +++
>  qom/cpu.c               |   12 ++++++++++++
>  2 files changed, 15 insertions(+), 0 deletions(-)

Thanks, applied to qom-cpu:
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

One comment below...

> 
> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> index 6578782..a8c3de1 100644
> --- a/include/sysemu/sysemu.h
> +++ b/include/sysemu/sysemu.h
> @@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const QDict *qdict);
>  /* generic hotplug */
>  void drive_hot_add(Monitor *mon, const QDict *qdict);
>  
> +/* CPU hotplug */
> +void qemu_register_cpu_added_notifier(Notifier *notifier);

Would be nice to turn this into a gtk-doc comment explaining when the
notification is fired, as a follow-up.

Andreas

> +
>  /* pcie aer error injection */
>  void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
>  int do_pcie_aer_inject_error(Monitor *mon,
> diff --git a/qom/cpu.c b/qom/cpu.c
> index abb20d0..9f78114 100644
> --- a/qom/cpu.c
> +++ b/qom/cpu.c
> @@ -21,6 +21,17 @@
>  #include "qom/cpu.h"
>  #include "qemu-common.h"
>  #include "sysemu/kvm.h"
> +#include "qemu/notify.h"
> +#include "sysemu/sysemu.h"
> +
> +/* CPU hot-plug notifiers */
> +static NotifierList cpu_added_notifiers =
> +    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
> +
> +void qemu_register_cpu_added_notifier(Notifier *notifier)
> +{
> +    notifier_list_add(&cpu_added_notifiers, notifier);
> +}
>  
>  void cpu_reset_interrupt(CPUState *cpu, int mask)
>  {
> @@ -60,6 +71,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
>  {
>      if (dev->hotplugged) {
>          cpu_synchronize_post_init(CPU(dev));
> +        notifier_list_notify(&cpu_added_notifiers, dev);
>          cpu_resume(CPU(dev));
>      }
>  }
> 


-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

[Qemu-devel] [PATCH 06/21] target-i386: pc: update rtc_cmos on CPU hot-plug

[Igor Mammedov]

it provides updated currently available CPUs count to BIOS on reboot

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v2:
  * reorder fields in rtc_cpu_hotplug_arg, make cpu_added_notifier first.
  * rebase on top of "mc146818rtc: QOM'ify" and use QOM style cast
---
 hw/i386/pc.c                   |   20 ++++++++++++++++++++
 hw/timer/mc146818rtc.c         |    7 +++++++
 include/hw/timer/mc146818rtc.h |    1 +
 3 files changed, 28 insertions(+), 0 deletions(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 0d6e72b..bbc3bea 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -338,6 +338,21 @@ static void pc_cmos_init_late(void *opaque)
     qemu_unregister_reset(pc_cmos_init_late, opaque);
 }
 
+typedef struct rtc_cpu_hotplug_arg {
+    Notifier cpu_added_notifier;
+    ISADevice *rtc_state;
+} rtc_cpu_hotplug_arg;
+
+static void rtc_notify_cpu_added(Notifier *notifier, void *data)
+{
+    rtc_cpu_hotplug_arg *arg = container_of(notifier, rtc_cpu_hotplug_arg,
+                                           cpu_added_notifier);
+    ISADevice *s = arg->rtc_state;
+
+    /* increment the number of CPUs */
+    rtc_set_memory(s, 0x5f, rtc_get_memory(s, 0x5f) + 1);
+}
+
 void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
                   const char *boot_device,
                   ISADevice *floppy, BusState *idebus0, BusState *idebus1,
@@ -346,6 +361,7 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
     int val, nb, i;
     FDriveType fd_type[2] = { FDRIVE_DRV_NONE, FDRIVE_DRV_NONE };
     static pc_cmos_init_late_arg arg;
+    static rtc_cpu_hotplug_arg cpu_hotplug_cb;
 
     /* various important CMOS locations needed by PC/Bochs bios */
 
@@ -384,6 +400,10 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
 
     /* set the number of CPU */
     rtc_set_memory(s, 0x5f, smp_cpus - 1);
+    /* init CPU hotplug notifier */
+    cpu_hotplug_cb.rtc_state = s;
+    cpu_hotplug_cb.cpu_added_notifier.notify = rtc_notify_cpu_added;
+    qemu_register_cpu_added_notifier(&cpu_hotplug_cb.cpu_added_notifier);
 
     /* set boot devices, and disable floppy signature check if requested */
     if (set_boot_dev(s, boot_device, fd_bootchk)) {
diff --git a/hw/timer/mc146818rtc.c b/hw/timer/mc146818rtc.c
index afbd0db..481604d 100644
--- a/hw/timer/mc146818rtc.c
+++ b/hw/timer/mc146818rtc.c
@@ -680,6 +680,13 @@ void rtc_set_memory(ISADevice *dev, int addr, int val)
         s->cmos_data[addr] = val;
 }
 
+int rtc_get_memory(ISADevice *dev, int addr)
+{
+    RTCState *s = MC146818_RTC(dev);
+    assert(addr >= 0 && addr <= 127);
+    return s->cmos_data[addr];
+}
+
 static void rtc_set_date_from_host(ISADevice *dev)
 {
     RTCState *s = MC146818_RTC(dev);
diff --git a/include/hw/timer/mc146818rtc.h b/include/hw/timer/mc146818rtc.h
index 425bd17..753dda6 100644
--- a/include/hw/timer/mc146818rtc.h
+++ b/include/hw/timer/mc146818rtc.h
@@ -8,6 +8,7 @@
 
 ISADevice *rtc_init(ISABus *bus, int base_year, qemu_irq intercept_irq);
 void rtc_set_memory(ISADevice *dev, int addr, int val);
+int rtc_get_memory(ISADevice *dev, int addr);
 void rtc_set_date(ISADevice *dev, const struct tm *tm);
 
 #endif /* !MC146818RTC_H */
-- 
1.7.1

Re: [Qemu-devel] [PATCH 06/21] target-i386: pc: update rtc_cmos on CPU hot-plug

[Andreas Färber]

Am 23.04.2013 10:29, schrieb Igor Mammedov:
> it provides updated currently available CPUs count to BIOS on reboot
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
> v2:
>   * reorder fields in rtc_cpu_hotplug_arg, make cpu_added_notifier first.
>   * rebase on top of "mc146818rtc: QOM'ify" and use QOM style cast
> ---
>  hw/i386/pc.c                   |   20 ++++++++++++++++++++
>  hw/timer/mc146818rtc.c         |    7 +++++++
>  include/hw/timer/mc146818rtc.h |    1 +
>  3 files changed, 28 insertions(+), 0 deletions(-)
> 
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 0d6e72b..bbc3bea 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -338,6 +338,21 @@ static void pc_cmos_init_late(void *opaque)
>      qemu_unregister_reset(pc_cmos_init_late, opaque);
>  }
>  
> +typedef struct rtc_cpu_hotplug_arg {
> +    Notifier cpu_added_notifier;
> +    ISADevice *rtc_state;
> +} rtc_cpu_hotplug_arg;

RTCCPUHotplugArg?

> +
> +static void rtc_notify_cpu_added(Notifier *notifier, void *data)
> +{
> +    rtc_cpu_hotplug_arg *arg = container_of(notifier, rtc_cpu_hotplug_arg,
> +                                           cpu_added_notifier);

Off-by-one (could fix myself)

Rest looks good.

Andreas

> +    ISADevice *s = arg->rtc_state;
> +
> +    /* increment the number of CPUs */
> +    rtc_set_memory(s, 0x5f, rtc_get_memory(s, 0x5f) + 1);
> +}
> +
>  void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
>                    const char *boot_device,
>                    ISADevice *floppy, BusState *idebus0, BusState *idebus1,
> @@ -346,6 +361,7 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
>      int val, nb, i;
>      FDriveType fd_type[2] = { FDRIVE_DRV_NONE, FDRIVE_DRV_NONE };
>      static pc_cmos_init_late_arg arg;
> +    static rtc_cpu_hotplug_arg cpu_hotplug_cb;
>  
>      /* various important CMOS locations needed by PC/Bochs bios */
>  
> @@ -384,6 +400,10 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
>  
>      /* set the number of CPU */
>      rtc_set_memory(s, 0x5f, smp_cpus - 1);
> +    /* init CPU hotplug notifier */
> +    cpu_hotplug_cb.rtc_state = s;
> +    cpu_hotplug_cb.cpu_added_notifier.notify = rtc_notify_cpu_added;
> +    qemu_register_cpu_added_notifier(&cpu_hotplug_cb.cpu_added_notifier);
>  
>      /* set boot devices, and disable floppy signature check if requested */
>      if (set_boot_dev(s, boot_device, fd_bootchk)) {
> diff --git a/hw/timer/mc146818rtc.c b/hw/timer/mc146818rtc.c
> index afbd0db..481604d 100644
> --- a/hw/timer/mc146818rtc.c
> +++ b/hw/timer/mc146818rtc.c
> @@ -680,6 +680,13 @@ void rtc_set_memory(ISADevice *dev, int addr, int val)
>          s->cmos_data[addr] = val;
>  }
>  
> +int rtc_get_memory(ISADevice *dev, int addr)
> +{
> +    RTCState *s = MC146818_RTC(dev);
> +    assert(addr >= 0 && addr <= 127);
> +    return s->cmos_data[addr];
> +}
> +
>  static void rtc_set_date_from_host(ISADevice *dev)
>  {
>      RTCState *s = MC146818_RTC(dev);
> diff --git a/include/hw/timer/mc146818rtc.h b/include/hw/timer/mc146818rtc.h
> index 425bd17..753dda6 100644
> --- a/include/hw/timer/mc146818rtc.h
> +++ b/include/hw/timer/mc146818rtc.h
> @@ -8,6 +8,7 @@
>  
>  ISADevice *rtc_init(ISABus *bus, int base_year, qemu_irq intercept_irq);
>  void rtc_set_memory(ISADevice *dev, int addr, int val);
> +int rtc_get_memory(ISADevice *dev, int addr);
>  void rtc_set_date(ISADevice *dev, const struct tm *tm);
>  
>  #endif /* !MC146818RTC_H */
> 


-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 06/21] target-i386: pc: update rtc_cmos on CPU hot-plug

[Andreas Färber]

Am 24.04.2013 19:03, schrieb Andreas Färber:
> Am 23.04.2013 10:29, schrieb Igor Mammedov:
>> it provides updated currently available CPUs count to BIOS on reboot
>>
>> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
>> ---
>> v2:
>>   * reorder fields in rtc_cpu_hotplug_arg, make cpu_added_notifier first.
>>   * rebase on top of "mc146818rtc: QOM'ify" and use QOM style cast
>> ---
>>  hw/i386/pc.c                   |   20 ++++++++++++++++++++
>>  hw/timer/mc146818rtc.c         |    7 +++++++
>>  include/hw/timer/mc146818rtc.h |    1 +
>>  3 files changed, 28 insertions(+), 0 deletions(-)
>>
>> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
>> index 0d6e72b..bbc3bea 100644
>> --- a/hw/i386/pc.c
>> +++ b/hw/i386/pc.c
>> @@ -338,6 +338,21 @@ static void pc_cmos_init_late(void *opaque)
>>      qemu_unregister_reset(pc_cmos_init_late, opaque);
>>  }
>>  
>> +typedef struct rtc_cpu_hotplug_arg {
>> +    Notifier cpu_added_notifier;
>> +    ISADevice *rtc_state;
>> +} rtc_cpu_hotplug_arg;
> 
> RTCCPUHotplugArg?

Igor agreed to this variant on IRC.

Thanks, applied to qom-cpu:
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

[Qemu-devel] [PATCH 07/21] cpu: introduce get_arch_id() method and override it for target-i386

[Igor Mammedov]

get_arch_id() adds possibility for generic code to get guest visible
CPU ID without accessing CPUArchState. If target doesn't override it,
it will return cpu_index.

Override it on target-i386 to return APIC ID.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: liguang <lig.fnst@cn.fujitsu.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
---
 * it will be used later by new cpu_exists() generic function and
   acpi_piix.
 * s/cpu_firmware_id/cpu_arch_id/
---
 include/qom/cpu.h |    2 ++
 qom/cpu.c         |    6 ++++++
 target-i386/cpu.c |   10 ++++++++++
 3 files changed, 18 insertions(+), 0 deletions(-)

diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index d6a0e80..639b436 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
  * instantiatable CPU type.
  * @reset: Callback to reset the #CPUState to its initial state.
  * @do_interrupt: Callback for interrupt handling.
+ * @get_arch_id: Callback for getting architecture depended CPU ID
  * @vmsd: State description for migration.
  *
  * Represents a CPU family or model.
@@ -58,6 +59,7 @@ typedef struct CPUClass {
 
     void (*reset)(CPUState *cpu);
     void (*do_interrupt)(CPUState *cpu);
+    int64_t (*get_arch_id)(CPUState *cpu);
 
     const struct VMStateDescription *vmsd;
 } CPUClass;
diff --git a/qom/cpu.c b/qom/cpu.c
index 9f78114..d44f987 100644
--- a/qom/cpu.c
+++ b/qom/cpu.c
@@ -76,6 +76,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     }
 }
 
+static int64_t cpu_common_get_arch_id(CPUState *cpu)
+{
+    return cpu->cpu_index;
+}
+
 static void cpu_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -83,6 +88,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
 
     k->class_by_name = cpu_common_class_by_name;
     k->reset = cpu_common_reset;
+    k->get_arch_id = cpu_common_get_arch_id;
     dc->realize = cpu_common_realizefn;
     dc->no_user = 1;
 }
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index e2302d8..a415fa3 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
     }
 }
 
+static int64_t x86_cpu_get_arch_id(CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+
+    return env->cpuid_apic_id;
+}
+
 static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 {
     X86CPUClass *xcc = X86_CPU_CLASS(oc);
@@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     cc->do_interrupt = x86_cpu_do_interrupt;
     cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
+
+    cc->get_arch_id = x86_cpu_get_arch_id;
 }
 
 static const TypeInfo x86_cpu_type_info = {
-- 
1.7.1

Re: [Qemu-devel] [PATCH 07/21] cpu: introduce get_arch_id() method and override it for target-i386

[Andreas Färber]

Am 23.04.2013 10:29, schrieb Igor Mammedov:
> get_arch_id() adds possibility for generic code to get guest visible
> CPU ID without accessing CPUArchState. If target doesn't override it,
> it will return cpu_index.
> 
> Override it on target-i386 to return APIC ID.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
> Reviewed-by: liguang <lig.fnst@cn.fujitsu.com>
> Acked-by: Michael S. Tsirkin <mst@redhat.com>
> ---
>  * it will be used later by new cpu_exists() generic function and
>    acpi_piix.
>  * s/cpu_firmware_id/cpu_arch_id/
> ---
>  include/qom/cpu.h |    2 ++
>  qom/cpu.c         |    6 ++++++
>  target-i386/cpu.c |   10 ++++++++++
>  3 files changed, 18 insertions(+), 0 deletions(-)

Thanks, applied to qom-cpu (with two minor changes below and rewording
wrt target vs. QOM class):

https://github.com/afaerber/qemu-cpu/commits/qom-cpu

> 
> diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> index d6a0e80..639b436 100644
> --- a/include/qom/cpu.h
> +++ b/include/qom/cpu.h
> @@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
>   * instantiatable CPU type.
>   * @reset: Callback to reset the #CPUState to its initial state.
>   * @do_interrupt: Callback for interrupt handling.
> + * @get_arch_id: Callback for getting architecture depended CPU ID

"dependent"

>   * @vmsd: State description for migration.
>   *
>   * Represents a CPU family or model.
[...]
> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> index e2302d8..a415fa3 100644
> --- a/target-i386/cpu.c
> +++ b/target-i386/cpu.c
> @@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
>      }
>  }
>  
> +static int64_t x86_cpu_get_arch_id(CPUState *cpu)
> +{
> +    X86CPU *x86cpu = X86_CPU(cpu);
> +    CPUX86State *env = &x86cpu->env;
> +
> +    return env->cpuid_apic_id;
> +}
> +
>  static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
>  {
>      X86CPUClass *xcc = X86_CPU_CLASS(oc);
[snip]

To stay consistent in our X86CPU naming conventions:

diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index a415fa3..f34ba23 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -2272,10 +2272,10 @@ static void x86_cpu_initfn(Object *obj)
     }
 }

-static int64_t x86_cpu_get_arch_id(CPUState *cpu)
+static int64_t x86_cpu_get_arch_id(CPUState *cs)
 {
-    X86CPU *x86cpu = X86_CPU(cpu);
-    CPUX86State *env = &x86cpu->env;
+    X86CPU *cpu = X86_CPU(cs);
+    CPUX86State *env = &cpu->env;

     return env->cpuid_apic_id;
 }

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

[Qemu-devel] [PATCH 08/21] exec: add qemu_for_each_cpu

[Igor Mammedov]

wrapper will help to remove open-coded loops

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
Note:
  Will be used by ACPI table generation and cpu_exists()
---
 cpus.c            |   13 +++++++------
 exec.c            |   10 ++++++++++
 include/qom/cpu.h |    8 ++++++++
 3 files changed, 25 insertions(+), 6 deletions(-)

diff --git a/cpus.c b/cpus.c
index 1d88761..5850151 100644
--- a/cpus.c
+++ b/cpus.c
@@ -812,6 +812,12 @@ static void *qemu_dummy_cpu_thread_fn(void *arg)
 
 static void tcg_exec_all(void);
 
+static void signal_cpu_creation(CPUState *cpu, void *data)
+{
+    cpu->thread_id = qemu_get_thread_id();
+    cpu->created = true;
+}
+
 static void *qemu_tcg_cpu_thread_fn(void *arg)
 {
     CPUState *cpu = arg;
@@ -820,13 +826,8 @@ static void *qemu_tcg_cpu_thread_fn(void *arg)
     qemu_tcg_init_cpu_signals();
     qemu_thread_get_self(cpu->thread);
 
-    /* signal CPU creation */
     qemu_mutex_lock(&qemu_global_mutex);
-    for (env = first_cpu; env != NULL; env = env->next_cpu) {
-        cpu = ENV_GET_CPU(env);
-        cpu->thread_id = qemu_get_thread_id();
-        cpu->created = true;
-    }
+    qemu_for_each_cpu(signal_cpu_creation, NULL);
     qemu_cond_signal(&qemu_cpu_cond);
 
     /* wait for initial kick-off after machine start */
diff --git a/exec.c b/exec.c
index fa1e0c3..19725db 100644
--- a/exec.c
+++ b/exec.c
@@ -265,6 +265,16 @@ CPUState *qemu_get_cpu(int index)
     return env ? cpu : NULL;
 }
 
+void qemu_for_each_cpu(void (*func)(CPUState *cpu, void *data), void *data)
+{
+    CPUArchState *env = first_cpu;
+
+    while (env) {
+        func(ENV_GET_CPU(env), data);
+        env = env->next_cpu;
+    }
+}
+
 void cpu_exec_init(CPUArchState *env)
 {
     CPUState *cpu = ENV_GET_CPU(env);
diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index 639b436..d4a21f4 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -215,6 +215,14 @@ bool cpu_is_stopped(CPUState *cpu);
  */
 void run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data);
 
+/** qemu_for_each_cpu:
+ * @func: The function to be executed.
+ * @data: Data to pass to the function.
+ *
+ * Executes @func on all CPUs
+ */
+void qemu_for_each_cpu(void (*func)(CPUState *cpu, void *data), void *data);
+
 /**
  * qemu_get_cpu:
  * @index: The CPUState@cpu_index value of the CPU to obtain.
-- 
1.7.1

Re: [Qemu-devel] [PATCH 08/21] exec: add qemu_for_each_cpu

[Andreas Färber]

Am 23.04.2013 10:29, schrieb Igor Mammedov:
> wrapper will help to remove open-coded loops
> 
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
> Note:
>   Will be used by ACPI table generation and cpu_exists()
> ---
>  cpus.c            |   13 +++++++------
>  exec.c            |   10 ++++++++++
>  include/qom/cpu.h |    8 ++++++++
>  3 files changed, 25 insertions(+), 6 deletions(-)

Thanks, split in two and applied to qom-cpu (with changes below):
https://github.com/afaerber/qemu-cpu/commits/qom-cpu

> 
> diff --git a/cpus.c b/cpus.c
> index 1d88761..5850151 100644
> --- a/cpus.c
> +++ b/cpus.c
> @@ -812,6 +812,12 @@ static void *qemu_dummy_cpu_thread_fn(void *arg)
>  
>  static void tcg_exec_all(void);
>  
> +static void signal_cpu_creation(CPUState *cpu, void *data)
> +{
> +    cpu->thread_id = qemu_get_thread_id();
> +    cpu->created = true;
> +}
> +
>  static void *qemu_tcg_cpu_thread_fn(void *arg)
>  {
>      CPUState *cpu = arg;
> @@ -820,13 +826,8 @@ static void *qemu_tcg_cpu_thread_fn(void *arg)
>      qemu_tcg_init_cpu_signals();
>      qemu_thread_get_self(cpu->thread);
>  
> -    /* signal CPU creation */
>      qemu_mutex_lock(&qemu_global_mutex);
> -    for (env = first_cpu; env != NULL; env = env->next_cpu) {
> -        cpu = ENV_GET_CPU(env);
> -        cpu->thread_id = qemu_get_thread_id();
> -        cpu->created = true;
> -    }
> +    qemu_for_each_cpu(signal_cpu_creation, NULL);
>      qemu_cond_signal(&qemu_cpu_cond);
>  
>      /* wait for initial kick-off after machine start */

We figured on IRC that TCG was not reflected in the function name, so I
changed as follows:

diff --git a/cpus.c b/cpus.c
index 5850151..2e7bbad 100644
--- a/cpus.c
+++ b/cpus.c
@@ -812,7 +812,7 @@ static void *qemu_dummy_cpu_thread_fn(void *arg)

 static void tcg_exec_all(void);

-static void signal_cpu_creation(CPUState *cpu, void *data)
+static void signal_tcg_cpu_creation(CPUState *cpu, void *data)
 {
     cpu->thread_id = qemu_get_thread_id();
     cpu->created = true;
@@ -827,7 +827,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg)
     qemu_thread_get_self(cpu->thread);

     qemu_mutex_lock(&qemu_global_mutex);
-    qemu_for_each_cpu(signal_cpu_creation, NULL);
+    qemu_for_each_cpu(signal_tcg_cpu_creation, NULL);
     qemu_cond_signal(&qemu_cpu_cond);

     /* wait for initial kick-off after machine start */


> diff --git a/exec.c b/exec.c
> index fa1e0c3..19725db 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -265,6 +265,16 @@ CPUState *qemu_get_cpu(int index)
>      return env ? cpu : NULL;
>  }
>  
> +void qemu_for_each_cpu(void (*func)(CPUState *cpu, void *data), void *data)
> +{
> +    CPUArchState *env = first_cpu;
> +
> +    while (env) {
> +        func(ENV_GET_CPU(env), data);
> +        env = env->next_cpu;
> +    }
> +}
> +
>  void cpu_exec_init(CPUArchState *env)
>  {
>      CPUState *cpu = ENV_GET_CPU(env);
> diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> index 639b436..d4a21f4 100644
> --- a/include/qom/cpu.h
> +++ b/include/qom/cpu.h
> @@ -215,6 +215,14 @@ bool cpu_is_stopped(CPUState *cpu);
>   */
>  void run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data);
>  
> +/** qemu_for_each_cpu:

Moved to next line.

> + * @func: The function to be executed.
> + * @data: Data to pass to the function.
> + *
> + * Executes @func on all CPUs

Changed to "for each CPU" to distinguish from run_on_cpu().

> + */
> +void qemu_for_each_cpu(void (*func)(CPUState *cpu, void *data), void *data);
> +
>  /**
>   * qemu_get_cpu:
>   * @index: The CPUState@cpu_index value of the CPU to obtain.

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

[Qemu-devel] [PATCH 09/21] cpu: add helper cpu_exists(), to check if CPU with specified id exists

[Igor Mammedov]

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v3:
  * use qemu_for_each_cpu() instead of recursion
v2:
  * s/get_firmware_id()/get_arch_id()/ rebase fixup
  * remove check for get_arch_id being NULL, since it's always defined
---
 include/qom/cpu.h |   10 ++++++++++
 qom/cpu.c         |   26 ++++++++++++++++++++++++++
 2 files changed, 36 insertions(+), 0 deletions(-)

diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index d4a21f4..7ba65a5 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -233,6 +233,16 @@ void qemu_for_each_cpu(void (*func)(CPUState *cpu, void *data), void *data);
  */
 CPUState *qemu_get_cpu(int index);
 
+/**
+ * cpu_exists:
+ * @id - guest exposed CPU ID to lookup
+ *
+ * Search for CPU with specified ID.
+ *
+ * Returns: true - CPU is found, false - CPU isn't found
+ */
+bool cpu_exists(int64_t id);
+
 #ifndef CONFIG_USER_ONLY
 
 typedef void (*CPUInterruptHandler)(CPUState *, int);
diff --git a/qom/cpu.c b/qom/cpu.c
index d44f987..fcc34aa 100644
--- a/qom/cpu.c
+++ b/qom/cpu.c
@@ -24,6 +24,32 @@
 #include "qemu/notify.h"
 #include "sysemu/sysemu.h"
 
+typedef struct CPU_exists_args {
+    int64_t id;
+    bool found;
+} CPU_exists_args;
+
+static void cpu_exist_cb(CPUState *cpu, void *data)
+{
+    CPUClass *klass = CPU_GET_CLASS(cpu);
+    CPU_exists_args *arg = data;
+
+    if (klass->get_arch_id(cpu) == arg->id) {
+        arg->found = true;
+    }
+}
+
+bool cpu_exists(int64_t id)
+{
+    CPU_exists_args data = {
+        .id = id,
+        .found = false,
+    };
+
+    qemu_for_each_cpu(cpu_exist_cb, &data);
+    return data.found;
+}
+
 /* CPU hot-plug notifiers */
 static NotifierList cpu_added_notifiers =
     NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
-- 
1.7.1

[Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Igor Mammedov]

* introduce processor status bitmask visible to guest at 0xaf00 addr,
  where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on
  receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v4:
  * added spec for QEMU-Seabios interface
  * added PIIX4_ prefix to PROC_ defines
v3:
  * s/get_firmware_id()/get_arch_id()/ due rebase
  * s/cpu_add_notifier/cpu_added_notifier/
v2:
  * use CPUClass.get_firmware_id() to make code target independent
  * bump up vmstate_acpi version
---
 docs/specs/acpi_cpu_hotplug.txt |   22 +++++++
 hw/acpi/piix4.c                 |  117 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 136 insertions(+), 3 deletions(-)
 create mode 100644 docs/specs/acpi_cpu_hotplug.txt

diff --git a/docs/specs/acpi_cpu_hotplug.txt b/docs/specs/acpi_cpu_hotplug.txt
new file mode 100644
index 0000000..5dec0c5
--- /dev/null
+++ b/docs/specs/acpi_cpu_hotplug.txt
@@ -0,0 +1,22 @@
+QEMU<->ACPI BIOS CPU hotplug interface
+--------------------------------------
+
+QEMU supports CPU hotplug via ACPI. This document
+describes the interface between QEMU and the ACPI BIOS.
+
+ACPI GPE block (IO ports 0xafe0-0xafe3, byte access):
+-----------------------------------------
+
+Generic ACPI GPE block. Bit 2 (GPE.2) used to notify CPU
+hot-add/remove event to ACPI BIOS, via SCI interrupt.
+
+CPU present bitmap (IO port 0xaf00-0xae1f, 1-byte access):
+---------------------------------------------------------------
+One bit per CPU. Bit position reflects corresponding CPU APIC ID.
+Read-only.
+
+CPU hot-add/remove notification:
+-----------------------------------------------------
+QEMU sets/clears corresponding CPU bit on hot-add/remove event.
+CPU present map read by ACPI BIOS GPE.2 handler to notify OS of CPU
+hot-(un)plug events.
diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
index 88386d7..b845123 100644
--- a/hw/acpi/piix4.c
+++ b/hw/acpi/piix4.c
@@ -48,19 +48,28 @@
 #define PCI_EJ_BASE 0xae08
 #define PCI_RMV_BASE 0xae0c
 
+#define PIIX4_PROC_BASE 0xaf00
+#define PIIX4_PROC_LEN 32
+
 #define PIIX4_PCI_HOTPLUG_STATUS 2
+#define PIIX4_CPU_HOTPLUG_STATUS 4
 
 struct pci_status {
     uint32_t up; /* deprecated, maintained for migration compatibility */
     uint32_t down;
 };
 
+struct cpu_status {
+    uint8_t sts[PIIX4_PROC_LEN];
+};
+
 typedef struct PIIX4PMState {
     PCIDevice dev;
 
     MemoryRegion io;
     MemoryRegion io_gpe;
     MemoryRegion io_pci;
+    MemoryRegion io_cpu;
     ACPIREGS ar;
 
     APMState apm;
@@ -82,6 +91,9 @@ typedef struct PIIX4PMState {
     uint8_t disable_s3;
     uint8_t disable_s4;
     uint8_t s4_val;
+
+    struct cpu_status gpe_cpu;
+    Notifier cpu_added_notifier;
 } PIIX4PMState;
 
 static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
@@ -100,8 +112,8 @@ static void pm_update_sci(PIIX4PMState *s)
                    ACPI_BITMASK_POWER_BUTTON_ENABLE |
                    ACPI_BITMASK_GLOBAL_LOCK_ENABLE |
                    ACPI_BITMASK_TIMER_ENABLE)) != 0) ||
-        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0])
-          & PIIX4_PCI_HOTPLUG_STATUS) != 0);
+        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) &
+          (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0);
 
     qemu_set_irq(s->irq, sci_level);
     /* schedule a timer interruption if needed */
@@ -257,6 +269,18 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
     return ret;
 }
 
+#define VMSTATE_CPU_STATUS_ARRAY(_field, _state)                             \
+ {                                                                           \
+     .name       = (stringify(_field)),                                      \
+     .version_id = 0,                                                        \
+     .num        = PIIX4_PROC_LEN,                                           \
+     .info       = &vmstate_info_uint8,                                      \
+     .size       = sizeof(uint8_t),                                          \
+     .flags      = VMS_ARRAY,                                                \
+     .offset     = vmstate_offset_array(_state, _field, uint8_t,             \
+                                        PIIX4_PROC_LEN),                     \
+ }
+
 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
  * and minimum_version_id to 2 below (which breaks migration from
@@ -265,7 +289,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
  */
 static const VMStateDescription vmstate_acpi = {
     .name = "piix4_pm",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = acpi_load_old,
@@ -281,6 +305,7 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
         VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
                        struct pci_status),
+        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),
         VMSTATE_END_OF_LIST()
     }
 };
@@ -585,6 +610,85 @@ static const MemoryRegionOps piix4_pci_ops = {
     },
 };
 
+static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned width)
+{
+    PIIX4PMState *s = opaque;
+    struct cpu_status *cpus = &s->gpe_cpu;
+    uint64_t val = cpus->sts[addr];
+
+    return val;
+}
+
+static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
+                             unsigned int size)
+{
+    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
+}
+
+static const MemoryRegionOps cpu_hotplug_ops = {
+    .read = cpu_status_read,
+    .write = cpu_status_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 1,
+    },
+};
+
+typedef enum {
+    PLUG,
+    UNPLUG,
+} HotplugEventType;
+
+static void piix4_cpu_hotplug_req(PIIX4PMState *s, CPUState *cpu,
+                                  HotplugEventType action)
+{
+    struct cpu_status *g = &s->gpe_cpu;
+    ACPIGPE *gpe = &s->ar.gpe;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t cpu_id;
+
+    assert(s != NULL);
+
+    *gpe->sts = *gpe->sts | PIIX4_CPU_HOTPLUG_STATUS;
+    cpu_id = k->get_arch_id(CPU(cpu));
+    if (action == PLUG) {
+        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+    } else {
+        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+    }
+    pm_update_sci(s);
+}
+
+static void piix4_cpu_added_req(Notifier *n, void *opaque)
+{
+    PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
+
+    piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
+}
+
+static int piix4_init_cpu_status(Object *obj, void *opaque)
+{
+    struct cpu_status *g = (struct cpu_status *)opaque;
+    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
+
+    if (cpu_obj) {
+        struct Error *error = NULL;
+        CPUClass *k = CPU_GET_CLASS(cpu_obj);
+        int64_t id = k->get_arch_id(CPU(cpu_obj));
+
+        if (error) {
+            fprintf(stderr, "failed to initilize CPU status for ACPI: %s\n",
+                    error_get_pretty(error));
+            error_free(error);
+            abort();
+        }
+        g_assert((id / 8) < PIIX4_PROC_LEN);
+        g->sts[id / 8] |= (1 << (id % 8));
+    }
+    return object_child_foreach(obj, piix4_init_cpu_status, opaque);
+}
+
 static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
                                 PCIHotplugState state);
 
@@ -600,6 +704,13 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
     memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
                                 &s->io_pci);
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
+
+    piix4_init_cpu_status(qdev_get_machine(), &s->gpe_cpu);
+    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
+                          PIIX4_PROC_LEN);
+    memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
+    s->cpu_added_notifier.notify = piix4_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
 }
 
 static void enable_device(PIIX4PMState *s, int slot)
-- 
1.7.1

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Juan Quintela]

Igor Mammedov <imammedo@redhat.com> wrote:

>  
> +#define VMSTATE_CPU_STATUS_ARRAY(_field, _state)                             \
> + {                                                                           \
> +     .name       = (stringify(_field)),                                      \
> +     .version_id = 0,
> \

this line should be:
   .version_id = 4, 


> +     .num        = PIIX4_PROC_LEN,                                           \
> +     .info       = &vmstate_info_uint8,                                      \
> +     .size       = sizeof(uint8_t),                                          \
> +     .flags      = VMS_ARRAY,                                                \
> +     .offset     = vmstate_offset_array(_state, _field, uint8_t,             \
> +                                        PIIX4_PROC_LEN),                     \
> + }
> +
>  /* qemu-kvm 1.2 uses version 3 but advertised as 2
>   * To support incoming qemu-kvm 1.2 migration, change version_id
>   * and minimum_version_id to 2 below (which breaks migration from

> @@ -265,7 +289,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
>   */
>  static const VMStateDescription vmstate_acpi = {
>      .name = "piix4_pm",
> -    .version_id = 3,
> +    .version_id = 4,
>      .minimum_version_id = 3,
>      .minimum_version_id_old = 1,
>      .load_state_old = acpi_load_old,
> @@ -281,6 +305,7 @@ static const VMStateDescription vmstate_acpi = {
>          VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
>          VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
>                         struct pci_status),
> +        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),

It is more,  I think that simply:

VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts,  PIIX4PMstate, PIIX4_PROC_LEN,  4);

Should do the trick without the previous blob (it was needed for the old
version because we have a uint32 data but we send a uint16 one).

Could you check?  I don't have an easy way to test that it "receives"
the right value.

Later,  Juan.

>          VMSTATE_END_OF_LIST()
>      }
>  };
> @@ -585,6 +610,85 @@ static const MemoryRegionOps piix4_pci_ops = {

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Igor Mammedov]

On Tue, 23 Apr 2013 13:38:10 +0200
Juan Quintela <quintela@redhat.com> wrote:

> Igor Mammedov <imammedo@redhat.com> wrote:
> 
> >  
> > +#define VMSTATE_CPU_STATUS_ARRAY(_field,
> > _state)                             \
> > +
> > {                                                                           \
> > +     .name       =
> > (stringify(_field)),                                      \
> > +     .version_id = 0,
> > \
> 
> this line should be:
>    .version_id = 4, 
> 
> 
> > +     .num        =
> > PIIX4_PROC_LEN,                                           \
> > +     .info       =
> > &vmstate_info_uint8,                                      \
> > +     .size       =
> > sizeof(uint8_t),                                          \
> > +     .flags      =
> > VMS_ARRAY,                                                \
> > +     .offset     = vmstate_offset_array(_state, _field,
> > uint8_t,             \
> > +
> > PIIX4_PROC_LEN),                     \
> > + }
> > +
> >  /* qemu-kvm 1.2 uses version 3 but advertised as 2
> >   * To support incoming qemu-kvm 1.2 migration, change version_id
> >   * and minimum_version_id to 2 below (which breaks migration from
> 
> > @@ -265,7 +289,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque,
> > int version_id) */
> >  static const VMStateDescription vmstate_acpi = {
> >      .name = "piix4_pm",
> > -    .version_id = 3,
> > +    .version_id = 4,
> >      .minimum_version_id = 3,
> >      .minimum_version_id_old = 1,
> >      .load_state_old = acpi_load_old,
> > @@ -281,6 +305,7 @@ static const VMStateDescription vmstate_acpi = {
> >          VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
> >          VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
> >                         struct pci_status),
> > +        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),
> 
> It is more,  I think that simply:
> 
> VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts,  PIIX4PMstate, PIIX4_PROC_LEN,  4);
> 
> Should do the trick without the previous blob (it was needed for the old
> version because we have a uint32 data but we send a uint16 one).
> 
> Could you check?  I don't have an easy way to test that it "receives"
> the right value.
Just checked, it works with VMSTATE_UINT8_ARRAY_V as well,

but I have a question, why version should be 4, looking at git history
components of vmstate_acpi don't advance their version each time
vmstate_acpi change, they do it only when they themselves change.

> 
> Later,  Juan.
> 
> >          VMSTATE_END_OF_LIST()
> >      }
> >  };
> > @@ -585,6 +610,85 @@ static const MemoryRegionOps piix4_pci_ops = {
> 

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Michael S. Tsirkin]

On Tue, Apr 23, 2013 at 02:54:16PM +0200, Igor Mammedov wrote:
> On Tue, 23 Apr 2013 13:38:10 +0200
> Juan Quintela <quintela@redhat.com> wrote:
> 
> > Igor Mammedov <imammedo@redhat.com> wrote:
> > 
> > >  
> > > +#define VMSTATE_CPU_STATUS_ARRAY(_field,
> > > _state)                             \
> > > +
> > > {                                                                           \
> > > +     .name       =
> > > (stringify(_field)),                                      \
> > > +     .version_id = 0,
> > > \
> > 
> > this line should be:
> >    .version_id = 4, 
> > 
> > 
> > > +     .num        =
> > > PIIX4_PROC_LEN,                                           \
> > > +     .info       =
> > > &vmstate_info_uint8,                                      \
> > > +     .size       =
> > > sizeof(uint8_t),                                          \
> > > +     .flags      =
> > > VMS_ARRAY,                                                \
> > > +     .offset     = vmstate_offset_array(_state, _field,
> > > uint8_t,             \
> > > +
> > > PIIX4_PROC_LEN),                     \
> > > + }
> > > +
> > >  /* qemu-kvm 1.2 uses version 3 but advertised as 2
> > >   * To support incoming qemu-kvm 1.2 migration, change version_id
> > >   * and minimum_version_id to 2 below (which breaks migration from
> > 
> > > @@ -265,7 +289,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque,
> > > int version_id) */
> > >  static const VMStateDescription vmstate_acpi = {
> > >      .name = "piix4_pm",
> > > -    .version_id = 3,
> > > +    .version_id = 4,
> > >      .minimum_version_id = 3,
> > >      .minimum_version_id_old = 1,
> > >      .load_state_old = acpi_load_old,
> > > @@ -281,6 +305,7 @@ static const VMStateDescription vmstate_acpi = {
> > >          VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
> > >          VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
> > >                         struct pci_status),
> > > +        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),
> > 
> > It is more,  I think that simply:
> > 
> > VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts,  PIIX4PMstate, PIIX4_PROC_LEN,  4);
> > 
> > Should do the trick without the previous blob (it was needed for the old
> > version because we have a uint32 data but we send a uint16 one).
> > 
> > Could you check?  I don't have an easy way to test that it "receives"
> > the right value.
> Just checked, it works with VMSTATE_UINT8_ARRAY_V as well,
> 
> but I have a question, why version should be 4, looking at git history
> components of vmstate_acpi don't advance their version each time
> vmstate_acpi change, they do it only when they themselves change.

Generally changing version breaks cross version migration.
So please don't do it for an optional
feature like CPU hotplug.

> > 
> > Later,  Juan.
> > 
> > >          VMSTATE_END_OF_LIST()
> > >      }
> > >  };
> > > @@ -585,6 +610,85 @@ static const MemoryRegionOps piix4_pci_ops = {
> > 

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Igor Mammedov]

On Tue, 23 Apr 2013 16:04:22 +0300
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> On Tue, Apr 23, 2013 at 02:54:16PM +0200, Igor Mammedov wrote:
> > On Tue, 23 Apr 2013 13:38:10 +0200
> > Juan Quintela <quintela@redhat.com> wrote:
> > 
> > > Igor Mammedov <imammedo@redhat.com> wrote:
> > > 
> > > >  
> > > > +#define VMSTATE_CPU_STATUS_ARRAY(_field,
> > > > _state)                             \
> > > > +
> > > > {                                                                           \
> > > > +     .name       =
> > > > (stringify(_field)),                                      \
> > > > +     .version_id = 0,
> > > > \
> > > 
> > > this line should be:
> > >    .version_id = 4, 
> > > 
> > > 
> > > > +     .num        =
> > > > PIIX4_PROC_LEN,                                           \
> > > > +     .info       =
> > > > &vmstate_info_uint8,                                      \
> > > > +     .size       =
> > > > sizeof(uint8_t),                                          \
> > > > +     .flags      =
> > > > VMS_ARRAY,                                                \
> > > > +     .offset     = vmstate_offset_array(_state, _field,
> > > > uint8_t,             \
> > > > +
> > > > PIIX4_PROC_LEN),                     \
> > > > + }
> > > > +
> > > >  /* qemu-kvm 1.2 uses version 3 but advertised as 2
> > > >   * To support incoming qemu-kvm 1.2 migration, change version_id
> > > >   * and minimum_version_id to 2 below (which breaks migration from
> > > 
> > > > @@ -265,7 +289,7 @@ static int acpi_load_old(QEMUFile *f, void
> > > > *opaque, int version_id) */
> > > >  static const VMStateDescription vmstate_acpi = {
> > > >      .name = "piix4_pm",
> > > > -    .version_id = 3,
> > > > +    .version_id = 4,
> > > >      .minimum_version_id = 3,
> > > >      .minimum_version_id_old = 1,
> > > >      .load_state_old = acpi_load_old,
> > > > @@ -281,6 +305,7 @@ static const VMStateDescription vmstate_acpi = {
> > > >          VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe,
> > > > ACPIGPE), VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2,
> > > > vmstate_pci_status, struct pci_status),
> > > > +        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),
> > > 
> > > It is more,  I think that simply:
> > > 
> > > VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts,  PIIX4PMstate, PIIX4_PROC_LEN,  4);
> > > 
> > > Should do the trick without the previous blob (it was needed for the old
> > > version because we have a uint32 data but we send a uint16 one).
> > > 
> > > Could you check?  I don't have an easy way to test that it "receives"
> > > the right value.
> > Just checked, it works with VMSTATE_UINT8_ARRAY_V as well,
> > 
> > but I have a question, why version should be 4, looking at git history
> > components of vmstate_acpi don't advance their version each time
> > vmstate_acpi change, they do it only when they themselves change.
> 
> Generally changing version breaks cross version migration.
> So please don't do it for an optional
> feature like CPU hotplug.
I probably wasn't clear enough, question was about why a new component
VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts, ...) should be version 4 and not 0?

As for advancing vmstate_acpi version, it was requested:
http://lists.gnu.org/archive/html/qemu-devel/2013-03/msg04775.html



Anyway why chatting on IRC with Juan, question arises:
Do we really need to save/restore gpe_cpu.sts field?

Since target has to be started with all CPUs (including hot-plugged), it will
have the same gpe_cpu.sts bitmap.

> > > 
> > > Later,  Juan.
> > > 
> > > >          VMSTATE_END_OF_LIST()
> > > >      }
> > > >  };
> > > > @@ -585,6 +610,85 @@ static const MemoryRegionOps piix4_pci_ops = {
> > > 
> 

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Michael S. Tsirkin]

On Tue, Apr 23, 2013 at 04:51:36PM +0200, Igor Mammedov wrote:
> On Tue, 23 Apr 2013 16:04:22 +0300
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
> 
> > On Tue, Apr 23, 2013 at 02:54:16PM +0200, Igor Mammedov wrote:
> > > On Tue, 23 Apr 2013 13:38:10 +0200
> > > Juan Quintela <quintela@redhat.com> wrote:
> > > 
> > > > Igor Mammedov <imammedo@redhat.com> wrote:
> > > > 
> > > > >  
> > > > > +#define VMSTATE_CPU_STATUS_ARRAY(_field,
> > > > > _state)                             \
> > > > > +
> > > > > {                                                                           \
> > > > > +     .name       =
> > > > > (stringify(_field)),                                      \
> > > > > +     .version_id = 0,
> > > > > \
> > > > 
> > > > this line should be:
> > > >    .version_id = 4, 
> > > > 
> > > > 
> > > > > +     .num        =
> > > > > PIIX4_PROC_LEN,                                           \
> > > > > +     .info       =
> > > > > &vmstate_info_uint8,                                      \
> > > > > +     .size       =
> > > > > sizeof(uint8_t),                                          \
> > > > > +     .flags      =
> > > > > VMS_ARRAY,                                                \
> > > > > +     .offset     = vmstate_offset_array(_state, _field,
> > > > > uint8_t,             \
> > > > > +
> > > > > PIIX4_PROC_LEN),                     \
> > > > > + }
> > > > > +
> > > > >  /* qemu-kvm 1.2 uses version 3 but advertised as 2
> > > > >   * To support incoming qemu-kvm 1.2 migration, change version_id
> > > > >   * and minimum_version_id to 2 below (which breaks migration from
> > > > 
> > > > > @@ -265,7 +289,7 @@ static int acpi_load_old(QEMUFile *f, void
> > > > > *opaque, int version_id) */
> > > > >  static const VMStateDescription vmstate_acpi = {
> > > > >      .name = "piix4_pm",
> > > > > -    .version_id = 3,
> > > > > +    .version_id = 4,
> > > > >      .minimum_version_id = 3,
> > > > >      .minimum_version_id_old = 1,
> > > > >      .load_state_old = acpi_load_old,
> > > > > @@ -281,6 +305,7 @@ static const VMStateDescription vmstate_acpi = {
> > > > >          VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe,
> > > > > ACPIGPE), VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2,
> > > > > vmstate_pci_status, struct pci_status),
> > > > > +        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),
> > > > 
> > > > It is more,  I think that simply:
> > > > 
> > > > VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts,  PIIX4PMstate, PIIX4_PROC_LEN,  4);
> > > > 
> > > > Should do the trick without the previous blob (it was needed for the old
> > > > version because we have a uint32 data but we send a uint16 one).
> > > > 
> > > > Could you check?  I don't have an easy way to test that it "receives"
> > > > the right value.
> > > Just checked, it works with VMSTATE_UINT8_ARRAY_V as well,
> > > 
> > > but I have a question, why version should be 4, looking at git history
> > > components of vmstate_acpi don't advance their version each time
> > > vmstate_acpi change, they do it only when they themselves change.
> > 
> > Generally changing version breaks cross version migration.
> > So please don't do it for an optional
> > feature like CPU hotplug.
> I probably wasn't clear enough, question was about why a new component
> VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts, ...) should be version 4 and not 0?
> 
> As for advancing vmstate_acpi version, it was requested:
> http://lists.gnu.org/archive/html/qemu-devel/2013-03/msg04775.html

It says
	You need to either bump the version, or add a subsection.   The
	subsection is not needed until the first hot-(un)plug action.
please add a subsection, don't bump the version :)

> 
> 
> Anyway why chatting on IRC with Juan, question arises:
> Do we really need to save/restore gpe_cpu.sts field?
> 
> Since target has to be started with all CPUs (including hot-plugged), it will
> have the same gpe_cpu.sts bitmap.
> 
> > > > 
> > > > Later,  Juan.
> > > > 
> > > > >          VMSTATE_END_OF_LIST()
> > > > >      }
> > > > >  };
> > > > > @@ -585,6 +610,85 @@ static const MemoryRegionOps piix4_pci_ops = {
> > > > 
> > 

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Juan Quintela]

Igor Mammedov <imammedo@redhat.com> wrote:
> On Tue, 23 Apr 2013 13:38:10 +0200
> Juan Quintela <quintela@redhat.com> wrote:
>
>> Igor Mammedov <imammedo@redhat.com> wrote:
>> 
>> >  
>> > +#define VMSTATE_CPU_STATUS_ARRAY(_field,
>> > _state)                             \
>> > +
>> > {                                                                           \
>> > +     .name       =
>> > (stringify(_field)),                                      \
>> > +     .version_id = 0,
>> > \
>> 
>> this line should be:
>>    .version_id = 4, 
>> 
>> 
>> > +     .num        =
>> > PIIX4_PROC_LEN,                                           \
>> > +     .info       =
>> > &vmstate_info_uint8,                                      \
>> > +     .size       =
>> > sizeof(uint8_t),                                          \
>> > +     .flags      =
>> > VMS_ARRAY,                                                \
>> > +     .offset     = vmstate_offset_array(_state, _field,
>> > uint8_t,             \
>> > +
>> > PIIX4_PROC_LEN),                     \
>> > + }
>> > +
>> >  /* qemu-kvm 1.2 uses version 3 but advertised as 2
>> >   * To support incoming qemu-kvm 1.2 migration, change version_id
>> >   * and minimum_version_id to 2 below (which breaks migration from
>> 
>> > @@ -265,7 +289,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque,
>> > int version_id) */
>> >  static const VMStateDescription vmstate_acpi = {
>> >      .name = "piix4_pm",
>> > -    .version_id = 3,
>> > +    .version_id = 4,
>> >      .minimum_version_id = 3,
>> >      .minimum_version_id_old = 1,
>> >      .load_state_old = acpi_load_old,
>> > @@ -281,6 +305,7 @@ static const VMStateDescription vmstate_acpi = {
>> >          VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
>> >          VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
>> >                         struct pci_status),
>> > +        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),
>> 
>> It is more,  I think that simply:
>> 
>> VMSTATE_UINT8_ARRAY_V(gpu_cpu.sts,  PIIX4PMstate, PIIX4_PROC_LEN,  4);
>> 
>> Should do the trick without the previous blob (it was needed for the old
>> version because we have a uint32 data but we send a uint16 one).
>> 
>> Could you check?  I don't have an easy way to test that it "receives"
>> the right value.
> Just checked, it works with VMSTATE_UINT8_ARRAY_V as well,
>
> but I have a question, why version should be 4, looking at git history
> components of vmstate_acpi don't advance their version each time
> vmstate_acpi change, they do it only when they themselves change.

I didn't knew this was an optional field.  Is this only needed for cpu
hotplug,  or said otherwise,  when iti s needed?  If so,  I can try to
put an optional section there.

Later,  Juan.


>
>> 
>> Later,  Juan.
>> 
>> >          VMSTATE_END_OF_LIST()
>> >      }
>> >  };
>> > @@ -585,6 +610,85 @@ static const MemoryRegionOps piix4_pci_ops = {
>> 

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Juan Quintela]

Igor Mammedov <imammedo@redhat.com> wrote:
> On Tue, 23 Apr 2013 13:38:10 +0200
> Juan Quintela <quintela@redhat.com> wrote:

Something like this (on top of your series) should work.

It implements an optional subsection and is sent depending if we have
used (or not) cpu hotplug.  I just compiled it,  haven't tested it,  so
it should be perfect (TM).

What do you think?

Thanks,  Juan.


>From f887ce646f65a81ceff1920c7e6e2287b914b3ca Mon Sep 17 00:00:00 2001
From: Juan Quintela <quintela@trasno.org>
Date: Tue, 23 Apr 2013 17:23:29 +0200
Subject: [PATCH] vmstate subsection example


Signed-off-by: Juan Quintela <quintela@trasno.org>
---
 hw/acpi/piix4.c | 42 +++++++++++++++++++++++++++++-------------
 1 file changed, 29 insertions(+), 13 deletions(-)

diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
index b845123..14dba76 100644
--- a/hw/acpi/piix4.c
+++ b/hw/acpi/piix4.c
@@ -60,6 +60,7 @@ struct pci_status {
 };

 struct cpu_status {
+    bool hotplug_happened;
     uint8_t sts[PIIX4_PROC_LEN];
 };

@@ -269,17 +270,23 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
     return ret;
 }

-#define VMSTATE_CPU_STATUS_ARRAY(_field, _state)                             \
- {                                                                           \
-     .name       = (stringify(_field)),                                      \
-     .version_id = 0,                                                        \
-     .num        = PIIX4_PROC_LEN,                                           \
-     .info       = &vmstate_info_uint8,                                      \
-     .size       = sizeof(uint8_t),                                          \
-     .flags      = VMS_ARRAY,                                                \
-     .offset     = vmstate_offset_array(_state, _field, uint8_t,             \
-                                        PIIX4_PROC_LEN),                     \
- }
+static bool vmstate_piix4_hotplug_needed(void *opaque)
+{
+    PIIX4PMState *s = opaque;
+
+    return s->gpe_cpu.hotplug_happened;
+}
+
+static const VMStateDescription vmstate_piix4_hotplug_state = {
+    .name ="piix4_pm/hotplug",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields = (VMStateField []) {
+        VMSTATE_UINT8_ARRAY(gpe_cpu.sts, PIIX4PMState, PIIX4_PROC_LEN),
+        VMSTATE_END_OF_LIST()
+    }
+};

 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
@@ -289,7 +296,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
  */
 static const VMStateDescription vmstate_acpi = {
     .name = "piix4_pm",
-    .version_id = 4,
+    .version_id = 3,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = acpi_load_old,
@@ -305,8 +312,15 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
         VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
                        struct pci_status),
-        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),
         VMSTATE_END_OF_LIST()
+    },
+    .subsections = (VMStateSubsection[]) {
+        {
+            .vmsd = &vmstate_piix4_hotplug_state,
+            .needed = vmstate_piix4_hotplug_needed,
+        }, {
+            /* empty */
+        }
     }
 };

@@ -664,6 +678,7 @@ static void piix4_cpu_added_req(Notifier *n, void *opaque)
 {
     PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);

+    s->gpe_cpu.hotplug_happened = true;
     piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
 }

@@ -706,6 +721,7 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);

     piix4_init_cpu_status(qdev_get_machine(), &s->gpe_cpu);
+    s->gpe_cpu.hotplug_happened = false;
     memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
                           PIIX4_PROC_LEN);
     memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
-- 
1.8.1.4

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Igor Mammedov]

On Tue, 23 Apr 2013 17:25:50 +0200
Juan Quintela <quintela@redhat.com> wrote:

> Igor Mammedov <imammedo@redhat.com> wrote:
> > On Tue, 23 Apr 2013 13:38:10 +0200
> > Juan Quintela <quintela@redhat.com> wrote:
> 
> Something like this (on top of your series) should work.
> 
> It implements an optional subsection and is sent depending if we have
> used (or not) cpu hotplug.  I just compiled it,  haven't tested it,  so
> it should be perfect (TM).
> 
> What do you think?
> 
> Thanks,  Juan.
> 

Just tested. it works.
I'm going to merge it in and remove unused Error* and repost patch.

> From f887ce646f65a81ceff1920c7e6e2287b914b3ca Mon Sep 17 00:00:00 2001
> From: Juan Quintela <quintela@trasno.org>
> Date: Tue, 23 Apr 2013 17:23:29 +0200
> Subject: [PATCH] vmstate subsection example
> 
> 
> Signed-off-by: Juan Quintela <quintela@trasno.org>
> ---
>  hw/acpi/piix4.c | 42 +++++++++++++++++++++++++++++-------------
>  1 file changed, 29 insertions(+), 13 deletions(-)
> 
> diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> index b845123..14dba76 100644
> --- a/hw/acpi/piix4.c
> +++ b/hw/acpi/piix4.c
> @@ -60,6 +60,7 @@ struct pci_status {
>  };
> 
>  struct cpu_status {
> +    bool hotplug_happened;
>      uint8_t sts[PIIX4_PROC_LEN];
>  };
> 
> @@ -269,17 +270,23 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
>      return ret;
>  }
> 
> -#define VMSTATE_CPU_STATUS_ARRAY(_field, _state)                             \
> - {                                                                           \
> -     .name       = (stringify(_field)),                                      \
> -     .version_id = 0,                                                        \
> -     .num        = PIIX4_PROC_LEN,                                           \
> -     .info       = &vmstate_info_uint8,                                      \
> -     .size       = sizeof(uint8_t),                                          \
> -     .flags      = VMS_ARRAY,                                                \
> -     .offset     = vmstate_offset_array(_state, _field, uint8_t,             \
> -                                        PIIX4_PROC_LEN),                     \
> - }
> +static bool vmstate_piix4_hotplug_needed(void *opaque)
> +{
> +    PIIX4PMState *s = opaque;
> +
> +    return s->gpe_cpu.hotplug_happened;
> +}
> +
> +static const VMStateDescription vmstate_piix4_hotplug_state = {
> +    .name ="piix4_pm/hotplug",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .minimum_version_id_old = 1,
> +    .fields = (VMStateField []) {
> +        VMSTATE_UINT8_ARRAY(gpe_cpu.sts, PIIX4PMState, PIIX4_PROC_LEN),
> +        VMSTATE_END_OF_LIST()
> +    }
> +};
> 
>  /* qemu-kvm 1.2 uses version 3 but advertised as 2
>   * To support incoming qemu-kvm 1.2 migration, change version_id
> @@ -289,7 +296,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
>   */
>  static const VMStateDescription vmstate_acpi = {
>      .name = "piix4_pm",
> -    .version_id = 4,
> +    .version_id = 3,
>      .minimum_version_id = 3,
>      .minimum_version_id_old = 1,
>      .load_state_old = acpi_load_old,
> @@ -305,8 +312,15 @@ static const VMStateDescription vmstate_acpi = {
>          VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
>          VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
>                         struct pci_status),
> -        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),
>          VMSTATE_END_OF_LIST()
> +    },
> +    .subsections = (VMStateSubsection[]) {
> +        {
> +            .vmsd = &vmstate_piix4_hotplug_state,
> +            .needed = vmstate_piix4_hotplug_needed,
> +        }, {
> +            /* empty */
> +        }
>      }
>  };
> 
> @@ -664,6 +678,7 @@ static void piix4_cpu_added_req(Notifier *n, void *opaque)
>  {
>      PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
> 
> +    s->gpe_cpu.hotplug_happened = true;
>      piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
>  }
> 
> @@ -706,6 +721,7 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
>      pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
> 
>      piix4_init_cpu_status(qdev_get_machine(), &s->gpe_cpu);
> +    s->gpe_cpu.hotplug_happened = false;
>      memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
>                            PIIX4_PROC_LEN);
>      memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Juan Quintela]

Igor Mammedov <imammedo@redhat.com> wrote:
> * introduce processor status bitmask visible to guest at 0xaf00 addr,
>   where ACPI asl code expects it
> * set bit corresponding to APIC ID in processor status bitmask on
>   receiving CPU hot-plug notification
> * trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event
>
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>

This is wrong (or at least supperfluous)

> +static int piix4_init_cpu_status(Object *obj, void *opaque)
> +{
> +    struct cpu_status *g = (struct cpu_status *)opaque;
> +    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
> +
> +    if (cpu_obj) {
> +        struct Error *error = NULL;

we set error to NULL

> +        CPUClass *k = CPU_GET_CLASS(cpu_obj);
> +        int64_t id = k->get_arch_id(CPU(cpu_obj));
> +
> +        if (error) {

and without touching error we test if it is != NULL.


something is missing here?

> +            fprintf(stderr, "failed to initilize CPU status for ACPI: %s\n",
> +                    error_get_pretty(error));
> +            error_free(error);
> +            abort();

Can't we return an error code at this point?  I guess no,  but asking
will not hurt.

> +        }
> +        g_assert((id / 8) < PIIX4_PROC_LEN);
> +        g->sts[id / 8] |= (1 << (id % 8));
> +    }
> +    return object_child_foreach(obj, piix4_init_cpu_status, opaque);
> +}
> +
>  static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
>                                  PCIHotplugState state);
>  
> @@ -600,6 +704,13 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
>      memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
>                                  &s->io_pci);
>      pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
> +
> +    piix4_init_cpu_status(qdev_get_machine(), &s->gpe_cpu);
> +    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
> +                          PIIX4_PROC_LEN);
> +    memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
> +    s->cpu_added_notifier.notify = piix4_cpu_added_req;
> +    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
>  }
>  
>  static void enable_device(PIIX4PMState *s, int slot)

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Eduardo Habkost]

On Tue, Apr 23, 2013 at 03:43:51PM +0200, Juan Quintela wrote:
> Igor Mammedov <imammedo@redhat.com> wrote:
> > * introduce processor status bitmask visible to guest at 0xaf00 addr,
> >   where ACPI asl code expects it
> > * set bit corresponding to APIC ID in processor status bitmask on
> >   receiving CPU hot-plug notification
> > * trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event
> >
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> 
> This is wrong (or at least supperfluous)
> 
> > +static int piix4_init_cpu_status(Object *obj, void *opaque)
> > +{
> > +    struct cpu_status *g = (struct cpu_status *)opaque;
> > +    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
> > +
> > +    if (cpu_obj) {
> > +        struct Error *error = NULL;
> 
> we set error to NULL
> 
> > +        CPUClass *k = CPU_GET_CLASS(cpu_obj);
> > +        int64_t id = k->get_arch_id(CPU(cpu_obj));
> > +
> > +        if (error) {
> 
> and without touching error we test if it is != NULL.
> 
> 
> something is missing here?

Looks like a leftover from v1, that called object_property_get_int()
(with an error argument) instead of k->get_arch_id().

-- 
Eduardo

Re: [Qemu-devel] [PATCH 10/21] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Igor Mammedov]

On Tue, 23 Apr 2013 15:43:51 +0200
Juan Quintela <quintela@redhat.com> wrote:

> Igor Mammedov <imammedo@redhat.com> wrote:
> > * introduce processor status bitmask visible to guest at 0xaf00 addr,
> >   where ACPI asl code expects it
> > * set bit corresponding to APIC ID in processor status bitmask on
> >   receiving CPU hot-plug notification
> > * trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event
> >
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> 
> This is wrong (or at least supperfluous)
> 
> > +static int piix4_init_cpu_status(Object *obj, void *opaque)
> > +{
> > +    struct cpu_status *g = (struct cpu_status *)opaque;
> > +    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
> > +
> > +    if (cpu_obj) {
> > +        struct Error *error = NULL;
> 
> we set error to NULL
> 
> > +        CPUClass *k = CPU_GET_CLASS(cpu_obj);
> > +        int64_t id = k->get_arch_id(CPU(cpu_obj));
> > +
> > +        if (error) {
> 
> and without touching error we test if it is != NULL.
> 
> 
> something is missing here?

It's leftover from previous versions, all error handing should in this
function should be dropped, leaving only assert().

> 
> > +            fprintf(stderr, "failed to initilize CPU status for ACPI:
> > %s\n",
> > +                    error_get_pretty(error));
> > +            error_free(error);
> > +            abort();
> 
> Can't we return an error code at this point?  I guess no,  but asking
> will not hurt.
> 
> > +        }
> > +        g_assert((id / 8) < PIIX4_PROC_LEN);
> > +        g->sts[id / 8] |= (1 << (id % 8));
> > +    }
> > +    return object_child_foreach(obj, piix4_init_cpu_status, opaque);
> > +}
> > +
> >  static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
> >                                  PCIHotplugState state);
> >  
> > @@ -600,6 +704,13 @@ static void
> > piix4_acpi_system_hot_add_init(MemoryRegion *parent,
> > memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR, &s->io_pci);
> >      pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
> > +
> > +    piix4_init_cpu_status(qdev_get_machine(), &s->gpe_cpu);
> > +    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s,
> > "apci-cpu-hotplug",
> > +                          PIIX4_PROC_LEN);
> > +    memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
> > +    s->cpu_added_notifier.notify = piix4_cpu_added_req;
> > +    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
> >  }
> >  
> >  static void enable_device(PIIX4PMState *s, int slot)
> 

[Qemu-devel] [PATCH 10/21 DISGISED v6] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Igor Mammedov]

* introduce processor status bitmask visible to guest at 0xaf00 addr,
  where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on
  receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Signed-off-by: Juan Quintela <quintela@trasno.org>
---

v5: available at: https://github.com/imammedo/qemu/tree/cpu_set.WIP
  * add optional vmstate subsection if there was CPU hotplug event
  * remove unused Error*
  * use qemu_for_each_cpu() instead of recursion over QOM tree
v4:
  * added spec for QEMU-Seabios interface
  * added PIIX4_ prefix to PROC_ defines
v3:
  * s/get_firmware_id()/get_arch_id()/ due rebase
  * s/cpu_add_notifier/cpu_added_notifier/
v2:
  * use CPUClass.get_firmware_id() to make code target independent
  * bump up vmstate_acpi version
---
 docs/specs/acpi_cpu_hotplug.txt |   22 +++++++
 hw/acpi/piix4.c                 |  119 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 139 insertions(+), 2 deletions(-)
 create mode 100644 docs/specs/acpi_cpu_hotplug.txt

diff --git a/docs/specs/acpi_cpu_hotplug.txt b/docs/specs/acpi_cpu_hotplug.txt
new file mode 100644
index 0000000..5dec0c5
--- /dev/null
+++ b/docs/specs/acpi_cpu_hotplug.txt
@@ -0,0 +1,22 @@
+QEMU<->ACPI BIOS CPU hotplug interface
+--------------------------------------
+
+QEMU supports CPU hotplug via ACPI. This document
+describes the interface between QEMU and the ACPI BIOS.
+
+ACPI GPE block (IO ports 0xafe0-0xafe3, byte access):
+-----------------------------------------
+
+Generic ACPI GPE block. Bit 2 (GPE.2) used to notify CPU
+hot-add/remove event to ACPI BIOS, via SCI interrupt.
+
+CPU present bitmap (IO port 0xaf00-0xae1f, 1-byte access):
+---------------------------------------------------------------
+One bit per CPU. Bit position reflects corresponding CPU APIC ID.
+Read-only.
+
+CPU hot-add/remove notification:
+-----------------------------------------------------
+QEMU sets/clears corresponding CPU bit on hot-add/remove event.
+CPU present map read by ACPI BIOS GPE.2 handler to notify OS of CPU
+hot-(un)plug events.
diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
index 88386d7..c29739e 100644
--- a/hw/acpi/piix4.c
+++ b/hw/acpi/piix4.c
@@ -48,19 +48,29 @@
 #define PCI_EJ_BASE 0xae08
 #define PCI_RMV_BASE 0xae0c
 
+#define PIIX4_PROC_BASE 0xaf00
+#define PIIX4_PROC_LEN 32
+
 #define PIIX4_PCI_HOTPLUG_STATUS 2
+#define PIIX4_CPU_HOTPLUG_STATUS 4
 
 struct pci_status {
     uint32_t up; /* deprecated, maintained for migration compatibility */
     uint32_t down;
 };
 
+struct cpu_status {
+    bool hotplug_happened;
+    uint8_t sts[PIIX4_PROC_LEN];
+};
+
 typedef struct PIIX4PMState {
     PCIDevice dev;
 
     MemoryRegion io;
     MemoryRegion io_gpe;
     MemoryRegion io_pci;
+    MemoryRegion io_cpu;
     ACPIREGS ar;
 
     APMState apm;
@@ -82,6 +92,9 @@ typedef struct PIIX4PMState {
     uint8_t disable_s3;
     uint8_t disable_s4;
     uint8_t s4_val;
+
+    struct cpu_status gpe_cpu;
+    Notifier cpu_added_notifier;
 } PIIX4PMState;
 
 static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
@@ -100,8 +113,8 @@ static void pm_update_sci(PIIX4PMState *s)
                    ACPI_BITMASK_POWER_BUTTON_ENABLE |
                    ACPI_BITMASK_GLOBAL_LOCK_ENABLE |
                    ACPI_BITMASK_TIMER_ENABLE)) != 0) ||
-        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0])
-          & PIIX4_PCI_HOTPLUG_STATUS) != 0);
+        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) &
+          (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0);
 
     qemu_set_irq(s->irq, sci_level);
     /* schedule a timer interruption if needed */
@@ -257,6 +270,24 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
     return ret;
 }
 
+static bool vmstate_piix4_hotplug_needed(void *opaque)
+{
+    PIIX4PMState *s = opaque;
+
+    return s->gpe_cpu.hotplug_happened;
+}
+
+static const VMStateDescription vmstate_piix4_hotplug_state = {
+    .name = "piix4_pm/hotplug",
+    .version_id = 1,
+    .minimum_version_id = 1,
+    .minimum_version_id_old = 1,
+    .fields = (VMStateField[]) {
+        VMSTATE_UINT8_ARRAY(gpe_cpu.sts, PIIX4PMState, PIIX4_PROC_LEN),
+        VMSTATE_END_OF_LIST()
+    }
+};
+
 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
  * and minimum_version_id to 2 below (which breaks migration from
@@ -282,6 +313,14 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
                        struct pci_status),
         VMSTATE_END_OF_LIST()
+    },
+    .subsections = (VMStateSubsection[]) {
+        {
+            .vmsd = &vmstate_piix4_hotplug_state,
+            .needed = vmstate_piix4_hotplug_needed,
+        }, {
+            /* empty */
+        }
     }
 };
 
@@ -585,6 +624,74 @@ static const MemoryRegionOps piix4_pci_ops = {
     },
 };
 
+static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned width)
+{
+    PIIX4PMState *s = opaque;
+    struct cpu_status *cpus = &s->gpe_cpu;
+    uint64_t val = cpus->sts[addr];
+
+    return val;
+}
+
+static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
+                             unsigned int size)
+{
+    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
+}
+
+static const MemoryRegionOps cpu_hotplug_ops = {
+    .read = cpu_status_read,
+    .write = cpu_status_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 1,
+    },
+};
+
+typedef enum {
+    PLUG,
+    UNPLUG,
+} HotplugEventType;
+
+static void piix4_cpu_hotplug_req(PIIX4PMState *s, CPUState *cpu,
+                                  HotplugEventType action)
+{
+    struct cpu_status *g = &s->gpe_cpu;
+    ACPIGPE *gpe = &s->ar.gpe;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t cpu_id;
+
+    assert(s != NULL);
+
+    *gpe->sts = *gpe->sts | PIIX4_CPU_HOTPLUG_STATUS;
+    cpu_id = k->get_arch_id(CPU(cpu));
+    if (action == PLUG) {
+        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+    } else {
+        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+    }
+    pm_update_sci(s);
+}
+
+static void piix4_cpu_added_req(Notifier *n, void *opaque)
+{
+    PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
+
+    s->gpe_cpu.hotplug_happened = true;
+    piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
+}
+
+static void piix4_init_cpu_status(CPUState *cpu, void *data)
+{
+    struct cpu_status *g = (struct cpu_status *)data;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t id = k->get_arch_id(cpu);
+
+    g_assert((id / 8) < PIIX4_PROC_LEN);
+    g->sts[id / 8] |= (1 << (id % 8));
+}
+
 static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
                                 PCIHotplugState state);
 
@@ -600,6 +707,14 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
     memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
                                 &s->io_pci);
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
+
+    qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
+    s->gpe_cpu.hotplug_happened = false;
+    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
+                          PIIX4_PROC_LEN);
+    memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
+    s->cpu_added_notifier.notify = piix4_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
 }
 
 static void enable_device(PIIX4PMState *s, int slot)
-- 
1.7.1

Re: [Qemu-devel] [PATCH 10/21 DISGISED v6] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Igor Mammedov]

On Tue, 23 Apr 2013 18:27:11 +0200
Igor Mammedov <imammedo@redhat.com> wrote:

NACK


  gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during
  migration just doesn't work, since destination QEMU has to be started
  with all present in guest CPUs (including hotplugged).
  i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4
  Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as
  on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
  since it has been started with 3 CPUs on command line.

tested on ping pong migration, It works as expected without gpe_cpu.sts migrated.
I'll post v7 to this thread shortly.

> * introduce processor status bitmask visible to guest at 0xaf00 addr,
>   where ACPI asl code expects it
> * set bit corresponding to APIC ID in processor status bitmask on
>   receiving CPU hot-plug notification
> * trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> Signed-off-by: Juan Quintela <quintela@trasno.org>
> ---
> 
> v5: available at: https://github.com/imammedo/qemu/tree/cpu_set.WIP
>   * add optional vmstate subsection if there was CPU hotplug event
>   * remove unused Error*
>   * use qemu_for_each_cpu() instead of recursion over QOM tree
> v4:
>   * added spec for QEMU-Seabios interface
>   * added PIIX4_ prefix to PROC_ defines
> v3:
>   * s/get_firmware_id()/get_arch_id()/ due rebase
>   * s/cpu_add_notifier/cpu_added_notifier/
> v2:
>   * use CPUClass.get_firmware_id() to make code target independent
>   * bump up vmstate_acpi version
> ---
>  docs/specs/acpi_cpu_hotplug.txt |   22 +++++++
>  hw/acpi/piix4.c                 |  119 ++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 139 insertions(+), 2 deletions(-)
>  create mode 100644 docs/specs/acpi_cpu_hotplug.txt
> 
> diff --git a/docs/specs/acpi_cpu_hotplug.txt b/docs/specs/acpi_cpu_hotplug.txt
> new file mode 100644
> index 0000000..5dec0c5
> --- /dev/null
> +++ b/docs/specs/acpi_cpu_hotplug.txt
> @@ -0,0 +1,22 @@
> +QEMU<->ACPI BIOS CPU hotplug interface
> +--------------------------------------
> +
> +QEMU supports CPU hotplug via ACPI. This document
> +describes the interface between QEMU and the ACPI BIOS.
> +
> +ACPI GPE block (IO ports 0xafe0-0xafe3, byte access):
> +-----------------------------------------
> +
> +Generic ACPI GPE block. Bit 2 (GPE.2) used to notify CPU
> +hot-add/remove event to ACPI BIOS, via SCI interrupt.
> +
> +CPU present bitmap (IO port 0xaf00-0xae1f, 1-byte access):
> +---------------------------------------------------------------
> +One bit per CPU. Bit position reflects corresponding CPU APIC ID.
> +Read-only.
> +
> +CPU hot-add/remove notification:
> +-----------------------------------------------------
> +QEMU sets/clears corresponding CPU bit on hot-add/remove event.
> +CPU present map read by ACPI BIOS GPE.2 handler to notify OS of CPU
> +hot-(un)plug events.
> diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> index 88386d7..c29739e 100644
> --- a/hw/acpi/piix4.c
> +++ b/hw/acpi/piix4.c
> @@ -48,19 +48,29 @@
>  #define PCI_EJ_BASE 0xae08
>  #define PCI_RMV_BASE 0xae0c
>  
> +#define PIIX4_PROC_BASE 0xaf00
> +#define PIIX4_PROC_LEN 32
> +
>  #define PIIX4_PCI_HOTPLUG_STATUS 2
> +#define PIIX4_CPU_HOTPLUG_STATUS 4
>  
>  struct pci_status {
>      uint32_t up; /* deprecated, maintained for migration compatibility */
>      uint32_t down;
>  };
>  
> +struct cpu_status {
> +    bool hotplug_happened;
> +    uint8_t sts[PIIX4_PROC_LEN];
> +};
> +
>  typedef struct PIIX4PMState {
>      PCIDevice dev;
>  
>      MemoryRegion io;
>      MemoryRegion io_gpe;
>      MemoryRegion io_pci;
> +    MemoryRegion io_cpu;
>      ACPIREGS ar;
>  
>      APMState apm;
> @@ -82,6 +92,9 @@ typedef struct PIIX4PMState {
>      uint8_t disable_s3;
>      uint8_t disable_s4;
>      uint8_t s4_val;
> +
> +    struct cpu_status gpe_cpu;
> +    Notifier cpu_added_notifier;
>  } PIIX4PMState;
>  
>  static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
> @@ -100,8 +113,8 @@ static void pm_update_sci(PIIX4PMState *s)
>                     ACPI_BITMASK_POWER_BUTTON_ENABLE |
>                     ACPI_BITMASK_GLOBAL_LOCK_ENABLE |
>                     ACPI_BITMASK_TIMER_ENABLE)) != 0) ||
> -        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0])
> -          & PIIX4_PCI_HOTPLUG_STATUS) != 0);
> +        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) &
> +          (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0);
>  
>      qemu_set_irq(s->irq, sci_level);
>      /* schedule a timer interruption if needed */
> @@ -257,6 +270,24 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
>      return ret;
>  }
>  
> +static bool vmstate_piix4_hotplug_needed(void *opaque)
> +{
> +    PIIX4PMState *s = opaque;
> +
> +    return s->gpe_cpu.hotplug_happened;
> +}
> +
> +static const VMStateDescription vmstate_piix4_hotplug_state = {
> +    .name = "piix4_pm/hotplug",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .minimum_version_id_old = 1,
> +    .fields = (VMStateField[]) {
> +        VMSTATE_UINT8_ARRAY(gpe_cpu.sts, PIIX4PMState, PIIX4_PROC_LEN),
> +        VMSTATE_END_OF_LIST()
> +    }
> +};
> +
>  /* qemu-kvm 1.2 uses version 3 but advertised as 2
>   * To support incoming qemu-kvm 1.2 migration, change version_id
>   * and minimum_version_id to 2 below (which breaks migration from
> @@ -282,6 +313,14 @@ static const VMStateDescription vmstate_acpi = {
>          VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
>                         struct pci_status),
>          VMSTATE_END_OF_LIST()
> +    },
> +    .subsections = (VMStateSubsection[]) {
> +        {
> +            .vmsd = &vmstate_piix4_hotplug_state,
> +            .needed = vmstate_piix4_hotplug_needed,
> +        }, {
> +            /* empty */
> +        }
>      }
>  };
>  
> @@ -585,6 +624,74 @@ static const MemoryRegionOps piix4_pci_ops = {
>      },
>  };
>  
> +static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned width)
> +{
> +    PIIX4PMState *s = opaque;
> +    struct cpu_status *cpus = &s->gpe_cpu;
> +    uint64_t val = cpus->sts[addr];
> +
> +    return val;
> +}
> +
> +static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
> +                             unsigned int size)
> +{
> +    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
> +}
> +
> +static const MemoryRegionOps cpu_hotplug_ops = {
> +    .read = cpu_status_read,
> +    .write = cpu_status_write,
> +    .endianness = DEVICE_LITTLE_ENDIAN,
> +    .valid = {
> +        .min_access_size = 1,
> +        .max_access_size = 1,
> +    },
> +};
> +
> +typedef enum {
> +    PLUG,
> +    UNPLUG,
> +} HotplugEventType;
> +
> +static void piix4_cpu_hotplug_req(PIIX4PMState *s, CPUState *cpu,
> +                                  HotplugEventType action)
> +{
> +    struct cpu_status *g = &s->gpe_cpu;
> +    ACPIGPE *gpe = &s->ar.gpe;
> +    CPUClass *k = CPU_GET_CLASS(cpu);
> +    int64_t cpu_id;
> +
> +    assert(s != NULL);
> +
> +    *gpe->sts = *gpe->sts | PIIX4_CPU_HOTPLUG_STATUS;
> +    cpu_id = k->get_arch_id(CPU(cpu));
> +    if (action == PLUG) {
> +        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
> +    } else {
> +        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
> +    }
> +    pm_update_sci(s);
> +}
> +
> +static void piix4_cpu_added_req(Notifier *n, void *opaque)
> +{
> +    PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
> +
> +    s->gpe_cpu.hotplug_happened = true;
> +    piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
> +}
> +
> +static void piix4_init_cpu_status(CPUState *cpu, void *data)
> +{
> +    struct cpu_status *g = (struct cpu_status *)data;
> +    CPUClass *k = CPU_GET_CLASS(cpu);
> +    int64_t id = k->get_arch_id(cpu);
> +
> +    g_assert((id / 8) < PIIX4_PROC_LEN);
> +    g->sts[id / 8] |= (1 << (id % 8));
> +}
> +
>  static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
>                                  PCIHotplugState state);
>  
> @@ -600,6 +707,14 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
>      memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
>                                  &s->io_pci);
>      pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
> +
> +    qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
> +    s->gpe_cpu.hotplug_happened = false;
> +    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
> +                          PIIX4_PROC_LEN);
> +    memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
> +    s->cpu_added_notifier.notify = piix4_cpu_added_req;
> +    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
>  }
>  
>  static void enable_device(PIIX4PMState *s, int slot)

Re: [Qemu-devel] [PATCH 10/21 DISGISED v6] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Eduardo Habkost]

On Wed, Apr 24, 2013 at 05:56:46PM +0200, Igor Mammedov wrote:
> On Tue, 23 Apr 2013 18:27:11 +0200
> Igor Mammedov <imammedo@redhat.com> wrote:
> 
> NACK
> 
> 
>   gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during
>   migration just doesn't work, since destination QEMU has to be started
>   with all present in guest CPUs (including hotplugged).
>   i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4
>   Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as
>   on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
>   since it has been started with 3 CPUs on command line.
> 
> tested on ping pong migration, It works as expected without gpe_cpu.sts migrated.
> I'll post v7 to this thread shortly.

Are we going to make the cpu-add command fail in case it gets called
during migration?

-- 
Eduardo

Re: [Qemu-devel] [PATCH 10/21 DISGISED v6] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Paolo Bonzini]

Il 24/04/2013 18:03, Eduardo Habkost ha scritto:
>> > tested on ping pong migration, It works as expected without gpe_cpu.sts migrated.
>> > I'll post v7 to this thread shortly.
> Are we going to make the cpu-add command fail in case it gets called
> during migration?

It's the same as device-add.  If the devices do not match for any
reason, migration will fail when an unrecognized state is read from the
source in the destination.

Paolo

Re: [Qemu-devel] [PATCH 10/21 DISGISED v6] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Andreas Färber]

Am 24.04.2013 18:03, schrieb Eduardo Habkost:
> On Wed, Apr 24, 2013 at 05:56:46PM +0200, Igor Mammedov wrote:
>> On Tue, 23 Apr 2013 18:27:11 +0200
>> Igor Mammedov <imammedo@redhat.com> wrote:
>>
>> NACK
>>
>>
>>   gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during
>>   migration just doesn't work, since destination QEMU has to be started
>>   with all present in guest CPUs (including hotplugged).
>>   i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4
>>   Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as
>>   on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
>>   since it has been started with 3 CPUs on command line.
>>
>> tested on ping pong migration, It works as expected without gpe_cpu.sts migrated.
>> I'll post v7 to this thread shortly.
> 
> Are we going to make the cpu-add command fail in case it gets called
> during migration?

I would rather suggest to throw an Error* on realize, if this can be
detected.

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 10/21 DISGISED v6] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Igor Mammedov]

On Wed, 24 Apr 2013 18:09:11 +0200
Andreas Färber <afaerber@suse.de> wrote:

> Am 24.04.2013 18:03, schrieb Eduardo Habkost:
> > On Wed, Apr 24, 2013 at 05:56:46PM +0200, Igor Mammedov wrote:
> >> On Tue, 23 Apr 2013 18:27:11 +0200
> >> Igor Mammedov <imammedo@redhat.com> wrote:
> >>
> >> NACK
> >>
> >>
> >>   gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during
> >>   migration just doesn't work, since destination QEMU has to be started
> >>   with all present in guest CPUs (including hotplugged).
> >>   i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4
> >>   Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as
> >>   on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
> >>   since it has been started with 3 CPUs on command line.
> >>
> >> tested on ping pong migration, It works as expected without gpe_cpu.sts migrated.
> >> I'll post v7 to this thread shortly.
> > 
> > Are we going to make the cpu-add command fail in case it gets called
> > during migration?
> 
> I would rather suggest to throw an Error* on realize, if this can be
> detected.
it would be better not to start/allow hot add at all if migration in
progress and let it complete. it probably could be made generic???

> 
> Andreas
> 
> -- 
> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg


-- 
Regards,
  Igor

[Qemu-devel] [PATCH 08/19 v7] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Igor Mammedov]

* introduce processor status bitmask visible to guest at 0xaf00 addr,
  where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on
  receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
Note:
  gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during
  migration just doesn't work, since destination QEMU has to be started
  with all present in guest CPUs (including hotplugged).
  i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4
  Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as
  on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
  since it has been started with 3 CPUs on command line.

v6:
  * drop gpe_cpu.sts migration hunks
v5:
  * add optional vmstate subsection if there was CPU hotplug event
  * remove unused Error*
  * use qemu_for_each_cpu() instead of recursion over QOM tree
v4:
  * added spec for QEMU-Seabios interface
  * added PIIX4_ prefix to PROC_ defines
v3:
  * s/get_firmware_id()/get_arch_id()/ due rebase
  * s/cpu_add_notifier/cpu_added_notifier/
v2:
  * use CPUClass.get_firmware_id() to make code target independent
  * bump up vmstate_acpi version
---
 docs/specs/acpi_cpu_hotplug.txt |   22 +++++++++
 hw/acpi/piix4.c                 |   90 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 110 insertions(+), 2 deletions(-)
 create mode 100644 docs/specs/acpi_cpu_hotplug.txt

diff --git a/docs/specs/acpi_cpu_hotplug.txt b/docs/specs/acpi_cpu_hotplug.txt
new file mode 100644
index 0000000..5dec0c5
--- /dev/null
+++ b/docs/specs/acpi_cpu_hotplug.txt
@@ -0,0 +1,22 @@
+QEMU<->ACPI BIOS CPU hotplug interface
+--------------------------------------
+
+QEMU supports CPU hotplug via ACPI. This document
+describes the interface between QEMU and the ACPI BIOS.
+
+ACPI GPE block (IO ports 0xafe0-0xafe3, byte access):
+-----------------------------------------
+
+Generic ACPI GPE block. Bit 2 (GPE.2) used to notify CPU
+hot-add/remove event to ACPI BIOS, via SCI interrupt.
+
+CPU present bitmap (IO port 0xaf00-0xae1f, 1-byte access):
+---------------------------------------------------------------
+One bit per CPU. Bit position reflects corresponding CPU APIC ID.
+Read-only.
+
+CPU hot-add/remove notification:
+-----------------------------------------------------
+QEMU sets/clears corresponding CPU bit on hot-add/remove event.
+CPU present map read by ACPI BIOS GPE.2 handler to notify OS of CPU
+hot-(un)plug events.
diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
index 88386d7..b18202c 100644
--- a/hw/acpi/piix4.c
+++ b/hw/acpi/piix4.c
@@ -48,19 +48,28 @@
 #define PCI_EJ_BASE 0xae08
 #define PCI_RMV_BASE 0xae0c
 
+#define PIIX4_PROC_BASE 0xaf00
+#define PIIX4_PROC_LEN 32
+
 #define PIIX4_PCI_HOTPLUG_STATUS 2
+#define PIIX4_CPU_HOTPLUG_STATUS 4
 
 struct pci_status {
     uint32_t up; /* deprecated, maintained for migration compatibility */
     uint32_t down;
 };
 
+struct cpu_status {
+    uint8_t sts[PIIX4_PROC_LEN];
+};
+
 typedef struct PIIX4PMState {
     PCIDevice dev;
 
     MemoryRegion io;
     MemoryRegion io_gpe;
     MemoryRegion io_pci;
+    MemoryRegion io_cpu;
     ACPIREGS ar;
 
     APMState apm;
@@ -82,6 +91,9 @@ typedef struct PIIX4PMState {
     uint8_t disable_s3;
     uint8_t disable_s4;
     uint8_t s4_val;
+
+    struct cpu_status gpe_cpu;
+    Notifier cpu_added_notifier;
 } PIIX4PMState;
 
 static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
@@ -100,8 +112,8 @@ static void pm_update_sci(PIIX4PMState *s)
                    ACPI_BITMASK_POWER_BUTTON_ENABLE |
                    ACPI_BITMASK_GLOBAL_LOCK_ENABLE |
                    ACPI_BITMASK_TIMER_ENABLE)) != 0) ||
-        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0])
-          & PIIX4_PCI_HOTPLUG_STATUS) != 0);
+        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) &
+          (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0);
 
     qemu_set_irq(s->irq, sci_level);
     /* schedule a timer interruption if needed */
@@ -585,6 +597,73 @@ static const MemoryRegionOps piix4_pci_ops = {
     },
 };
 
+static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned width)
+{
+    PIIX4PMState *s = opaque;
+    struct cpu_status *cpus = &s->gpe_cpu;
+    uint64_t val = cpus->sts[addr];
+
+    return val;
+}
+
+static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
+                             unsigned int size)
+{
+    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
+}
+
+static const MemoryRegionOps cpu_hotplug_ops = {
+    .read = cpu_status_read,
+    .write = cpu_status_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 1,
+    },
+};
+
+typedef enum {
+    PLUG,
+    UNPLUG,
+} HotplugEventType;
+
+static void piix4_cpu_hotplug_req(PIIX4PMState *s, CPUState *cpu,
+                                  HotplugEventType action)
+{
+    struct cpu_status *g = &s->gpe_cpu;
+    ACPIGPE *gpe = &s->ar.gpe;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t cpu_id;
+
+    assert(s != NULL);
+
+    *gpe->sts = *gpe->sts | PIIX4_CPU_HOTPLUG_STATUS;
+    cpu_id = k->get_arch_id(CPU(cpu));
+    if (action == PLUG) {
+        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+    } else {
+        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+    }
+    pm_update_sci(s);
+}
+
+static void piix4_cpu_added_req(Notifier *n, void *opaque)
+{
+    PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
+
+    piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
+}
+
+static void piix4_init_cpu_status(CPUState *cpu, void *data)
+{
+    struct cpu_status *g = (struct cpu_status *)data;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t id = k->get_arch_id(cpu);
+
+    g_assert((id / 8) < PIIX4_PROC_LEN);
+    g->sts[id / 8] |= (1 << (id % 8));
+}
+
 static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
                                 PCIHotplugState state);
 
@@ -600,6 +679,13 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
     memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
                                 &s->io_pci);
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
+
+    qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
+    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
+                          PIIX4_PROC_LEN);
+    memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
+    s->cpu_added_notifier.notify = piix4_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
 }
 
 static void enable_device(PIIX4PMState *s, int slot)
-- 
1.7.1

Re: [Qemu-devel] [PATCH 08/19 v7] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Andreas Färber]

Am 24.04.2013 17:58, schrieb Igor Mammedov:
> * introduce processor status bitmask visible to guest at 0xaf00 addr,
>   where ACPI asl code expects it
> * set bit corresponding to APIC ID in processor status bitmask on
>   receiving CPU hot-plug notification
> * trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
> Note:
>   gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during
>   migration just doesn't work, since destination QEMU has to be started
>   with all present in guest CPUs (including hotplugged).
>   i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4
>   Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as
>   on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
>   since it has been started with 3 CPUs on command line.
> 
> v6:
>   * drop gpe_cpu.sts migration hunks
> v5:
>   * add optional vmstate subsection if there was CPU hotplug event
>   * remove unused Error*
>   * use qemu_for_each_cpu() instead of recursion over QOM tree
> v4:
>   * added spec for QEMU-Seabios interface
>   * added PIIX4_ prefix to PROC_ defines
> v3:
>   * s/get_firmware_id()/get_arch_id()/ due rebase
>   * s/cpu_add_notifier/cpu_added_notifier/
> v2:
>   * use CPUClass.get_firmware_id() to make code target independent
>   * bump up vmstate_acpi version
> ---
>  docs/specs/acpi_cpu_hotplug.txt |   22 +++++++++
>  hw/acpi/piix4.c                 |   90 ++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 110 insertions(+), 2 deletions(-)
>  create mode 100644 docs/specs/acpi_cpu_hotplug.txt
> 
[...]
> diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> index 88386d7..b18202c 100644
> --- a/hw/acpi/piix4.c
> +++ b/hw/acpi/piix4.c
> @@ -48,19 +48,28 @@
>  #define PCI_EJ_BASE 0xae08
>  #define PCI_RMV_BASE 0xae0c
>  
> +#define PIIX4_PROC_BASE 0xaf00
> +#define PIIX4_PROC_LEN 32
> +
>  #define PIIX4_PCI_HOTPLUG_STATUS 2
> +#define PIIX4_CPU_HOTPLUG_STATUS 4
>  
>  struct pci_status {
>      uint32_t up; /* deprecated, maintained for migration compatibility */
>      uint32_t down;
>  };
>  
> +struct cpu_status {

I see that you're copying pci_status above, but Coding Style asks for
CamelCase and typedef, so could we change this to CPUStatus or better
PIIX4CPUStatus? (I counted 5 occurrences)

Andreas

> +    uint8_t sts[PIIX4_PROC_LEN];
> +};
> +
>  typedef struct PIIX4PMState {
>      PCIDevice dev;
>  
>      MemoryRegion io;
>      MemoryRegion io_gpe;
>      MemoryRegion io_pci;
> +    MemoryRegion io_cpu;
>      ACPIREGS ar;
>  
>      APMState apm;
> @@ -82,6 +91,9 @@ typedef struct PIIX4PMState {
>      uint8_t disable_s3;
>      uint8_t disable_s4;
>      uint8_t s4_val;
> +
> +    struct cpu_status gpe_cpu;
> +    Notifier cpu_added_notifier;
>  } PIIX4PMState;
>  
>  static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
> @@ -100,8 +112,8 @@ static void pm_update_sci(PIIX4PMState *s)
>                     ACPI_BITMASK_POWER_BUTTON_ENABLE |
>                     ACPI_BITMASK_GLOBAL_LOCK_ENABLE |
>                     ACPI_BITMASK_TIMER_ENABLE)) != 0) ||
> -        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0])
> -          & PIIX4_PCI_HOTPLUG_STATUS) != 0);
> +        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) &
> +          (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0);
>  
>      qemu_set_irq(s->irq, sci_level);
>      /* schedule a timer interruption if needed */
> @@ -585,6 +597,73 @@ static const MemoryRegionOps piix4_pci_ops = {
>      },
>  };
>  
> +static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned width)
> +{
> +    PIIX4PMState *s = opaque;
> +    struct cpu_status *cpus = &s->gpe_cpu;
> +    uint64_t val = cpus->sts[addr];
> +
> +    return val;
> +}
> +
> +static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
> +                             unsigned int size)
> +{
> +    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
> +}
> +
> +static const MemoryRegionOps cpu_hotplug_ops = {
> +    .read = cpu_status_read,
> +    .write = cpu_status_write,
> +    .endianness = DEVICE_LITTLE_ENDIAN,
> +    .valid = {
> +        .min_access_size = 1,
> +        .max_access_size = 1,
> +    },
> +};
> +
> +typedef enum {
> +    PLUG,
> +    UNPLUG,
> +} HotplugEventType;
> +
> +static void piix4_cpu_hotplug_req(PIIX4PMState *s, CPUState *cpu,
> +                                  HotplugEventType action)
> +{
> +    struct cpu_status *g = &s->gpe_cpu;
> +    ACPIGPE *gpe = &s->ar.gpe;
> +    CPUClass *k = CPU_GET_CLASS(cpu);
> +    int64_t cpu_id;
> +
> +    assert(s != NULL);
> +
> +    *gpe->sts = *gpe->sts | PIIX4_CPU_HOTPLUG_STATUS;
> +    cpu_id = k->get_arch_id(CPU(cpu));
> +    if (action == PLUG) {
> +        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
> +    } else {
> +        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
> +    }
> +    pm_update_sci(s);
> +}
> +
> +static void piix4_cpu_added_req(Notifier *n, void *opaque)
> +{
> +    PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
> +
> +    piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
> +}
> +
> +static void piix4_init_cpu_status(CPUState *cpu, void *data)
> +{
> +    struct cpu_status *g = (struct cpu_status *)data;
> +    CPUClass *k = CPU_GET_CLASS(cpu);
> +    int64_t id = k->get_arch_id(cpu);
> +
> +    g_assert((id / 8) < PIIX4_PROC_LEN);
> +    g->sts[id / 8] |= (1 << (id % 8));
> +}
> +
>  static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
>                                  PCIHotplugState state);
>  
> @@ -600,6 +679,13 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
>      memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
>                                  &s->io_pci);
>      pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
> +
> +    qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
> +    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
> +                          PIIX4_PROC_LEN);
> +    memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
> +    s->cpu_added_notifier.notify = piix4_cpu_added_req;
> +    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
>  }
>  
>  static void enable_device(PIIX4PMState *s, int slot)
> 


-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 08/19 v7] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Igor Mammedov]

On Wed, 24 Apr 2013 18:06:51 +0200
Andreas Färber <afaerber@suse.de> wrote:

> Am 24.04.2013 17:58, schrieb Igor Mammedov:
> > * introduce processor status bitmask visible to guest at 0xaf00 addr,
> >   where ACPI asl code expects it
> > * set bit corresponding to APIC ID in processor status bitmask on
> >   receiving CPU hot-plug notification
> > * trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > ---
> > Note:
> >   gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during
> >   migration just doesn't work, since destination QEMU has to be started
> >   with all present in guest CPUs (including hotplugged).
> >   i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4
> >   Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as
> >   on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
> >   since it has been started with 3 CPUs on command line.
> > 
> > v6:
> >   * drop gpe_cpu.sts migration hunks
> > v5:
> >   * add optional vmstate subsection if there was CPU hotplug event
> >   * remove unused Error*
> >   * use qemu_for_each_cpu() instead of recursion over QOM tree
> > v4:
> >   * added spec for QEMU-Seabios interface
> >   * added PIIX4_ prefix to PROC_ defines
> > v3:
> >   * s/get_firmware_id()/get_arch_id()/ due rebase
> >   * s/cpu_add_notifier/cpu_added_notifier/
> > v2:
> >   * use CPUClass.get_firmware_id() to make code target independent
> >   * bump up vmstate_acpi version
> > ---
> >  docs/specs/acpi_cpu_hotplug.txt |   22 +++++++++
> >  hw/acpi/piix4.c                 |   90 ++++++++++++++++++++++++++++++++++++++-
> >  2 files changed, 110 insertions(+), 2 deletions(-)
> >  create mode 100644 docs/specs/acpi_cpu_hotplug.txt
> > 
> [...]
> > diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> > index 88386d7..b18202c 100644
> > --- a/hw/acpi/piix4.c
> > +++ b/hw/acpi/piix4.c
> > @@ -48,19 +48,28 @@
> >  #define PCI_EJ_BASE 0xae08
> >  #define PCI_RMV_BASE 0xae0c
> >  
> > +#define PIIX4_PROC_BASE 0xaf00
> > +#define PIIX4_PROC_LEN 32
> > +
> >  #define PIIX4_PCI_HOTPLUG_STATUS 2
> > +#define PIIX4_CPU_HOTPLUG_STATUS 4
> >  
> >  struct pci_status {
> >      uint32_t up; /* deprecated, maintained for migration compatibility */
> >      uint32_t down;
> >  };
> >  
> > +struct cpu_status {
> 
> I see that you're copying pci_status above, but Coding Style asks for
> CamelCase and typedef, so could we change this to CPUStatus or better
> PIIX4CPUStatus? (I counted 5 occurrences)
sure, I'll resend  it with CPUStatus variant.

> Andreas
> 
> > +    uint8_t sts[PIIX4_PROC_LEN];
> > +};
> > +
> >  typedef struct PIIX4PMState {
> >      PCIDevice dev;
> >  
> >      MemoryRegion io;
> >      MemoryRegion io_gpe;
> >      MemoryRegion io_pci;
> > +    MemoryRegion io_cpu;
> >      ACPIREGS ar;
> >  
> >      APMState apm;
> > @@ -82,6 +91,9 @@ typedef struct PIIX4PMState {
> >      uint8_t disable_s3;
> >      uint8_t disable_s4;
> >      uint8_t s4_val;
> > +
> > +    struct cpu_status gpe_cpu;
> > +    Notifier cpu_added_notifier;
> >  } PIIX4PMState;
> >  
> >  static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
> > @@ -100,8 +112,8 @@ static void pm_update_sci(PIIX4PMState *s)
> >                     ACPI_BITMASK_POWER_BUTTON_ENABLE |
> >                     ACPI_BITMASK_GLOBAL_LOCK_ENABLE |
> >                     ACPI_BITMASK_TIMER_ENABLE)) != 0) ||
> > -        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0])
> > -          & PIIX4_PCI_HOTPLUG_STATUS) != 0);
> > +        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) &
> > +          (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0);
> >  
> >      qemu_set_irq(s->irq, sci_level);
> >      /* schedule a timer interruption if needed */
> > @@ -585,6 +597,73 @@ static const MemoryRegionOps piix4_pci_ops = {
> >      },
> >  };
> >  
> > +static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned width)
> > +{
> > +    PIIX4PMState *s = opaque;
> > +    struct cpu_status *cpus = &s->gpe_cpu;
> > +    uint64_t val = cpus->sts[addr];
> > +
> > +    return val;
> > +}
> > +
> > +static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
> > +                             unsigned int size)
> > +{
> > +    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
> > +}
> > +
> > +static const MemoryRegionOps cpu_hotplug_ops = {
> > +    .read = cpu_status_read,
> > +    .write = cpu_status_write,
> > +    .endianness = DEVICE_LITTLE_ENDIAN,
> > +    .valid = {
> > +        .min_access_size = 1,
> > +        .max_access_size = 1,
> > +    },
> > +};
> > +
> > +typedef enum {
> > +    PLUG,
> > +    UNPLUG,
> > +} HotplugEventType;
> > +
> > +static void piix4_cpu_hotplug_req(PIIX4PMState *s, CPUState *cpu,
> > +                                  HotplugEventType action)
> > +{
> > +    struct cpu_status *g = &s->gpe_cpu;
> > +    ACPIGPE *gpe = &s->ar.gpe;
> > +    CPUClass *k = CPU_GET_CLASS(cpu);
> > +    int64_t cpu_id;
> > +
> > +    assert(s != NULL);
> > +
> > +    *gpe->sts = *gpe->sts | PIIX4_CPU_HOTPLUG_STATUS;
> > +    cpu_id = k->get_arch_id(CPU(cpu));
> > +    if (action == PLUG) {
> > +        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
> > +    } else {
> > +        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
> > +    }
> > +    pm_update_sci(s);
> > +}
> > +
> > +static void piix4_cpu_added_req(Notifier *n, void *opaque)
> > +{
> > +    PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
> > +
> > +    piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
> > +}
> > +
> > +static void piix4_init_cpu_status(CPUState *cpu, void *data)
> > +{
> > +    struct cpu_status *g = (struct cpu_status *)data;
> > +    CPUClass *k = CPU_GET_CLASS(cpu);
> > +    int64_t id = k->get_arch_id(cpu);
> > +
> > +    g_assert((id / 8) < PIIX4_PROC_LEN);
> > +    g->sts[id / 8] |= (1 << (id % 8));
> > +}
> > +
> >  static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
> >                                  PCIHotplugState state);
> >  
> > @@ -600,6 +679,13 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
> >      memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
> >                                  &s->io_pci);
> >      pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
> > +
> > +    qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
> > +    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
> > +                          PIIX4_PROC_LEN);
> > +    memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
> > +    s->cpu_added_notifier.notify = piix4_cpu_added_req;
> > +    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
> >  }
> >  
> >  static void enable_device(PIIX4PMState *s, int slot)
> > 
> 
> 
> -- 
> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
> 


-- 
Regards,
  Igor

[Qemu-devel] [PATCH 10/21 v8] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Igor Mammedov]

* introduce processor status bitmask visible to guest at 0xaf00 addr,
  where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on
  receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
Note:
  gpe_cpu.sts isn't need to be migrated, since CPU hotpluging during
  migration just doesn't work, since destination QEMU has to be started
  with all present in guest CPUs (including hotplugged).
  i.e. src-qemu -smp 2,max-cpus=4; cpu-add id=2; dst-qemu -smp 3,max-cpus=4
  Destination QEMU will recreate the same gpe_cpu.sts=t'111' bitmap as
  on source by calling qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
  since it has been started with 3 CPUs on command line.

v7:
  * s/struct cpu_status/struct CPUStatus/
v6:
  * drop gpe_cpu.sts migration hunks
v5:
  * add optional vmstate subsection if there was CPU hotplug event
  * remove unused Error*
  * use qemu_for_each_cpu() instead of recursion over QOM tree
v4:
  * added spec for QEMU-Seabios interface
  * added PIIX4_ prefix to PROC_ defines
v3:
  * s/get_firmware_id()/get_arch_id()/ due rebase
  * s/cpu_add_notifier/cpu_added_notifier/
v2:
  * use CPUClass.get_firmware_id() to make code target independent
  * bump up vmstate_acpi version
---
 docs/specs/acpi_cpu_hotplug.txt | 22 ++++++++++
 hw/acpi/piix4.c                 | 90 ++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 110 insertions(+), 2 deletions(-)
 create mode 100644 docs/specs/acpi_cpu_hotplug.txt

diff --git a/docs/specs/acpi_cpu_hotplug.txt b/docs/specs/acpi_cpu_hotplug.txt
new file mode 100644
index 0000000..5dec0c5
--- /dev/null
+++ b/docs/specs/acpi_cpu_hotplug.txt
@@ -0,0 +1,22 @@
+QEMU<->ACPI BIOS CPU hotplug interface
+--------------------------------------
+
+QEMU supports CPU hotplug via ACPI. This document
+describes the interface between QEMU and the ACPI BIOS.
+
+ACPI GPE block (IO ports 0xafe0-0xafe3, byte access):
+-----------------------------------------
+
+Generic ACPI GPE block. Bit 2 (GPE.2) used to notify CPU
+hot-add/remove event to ACPI BIOS, via SCI interrupt.
+
+CPU present bitmap (IO port 0xaf00-0xae1f, 1-byte access):
+---------------------------------------------------------------
+One bit per CPU. Bit position reflects corresponding CPU APIC ID.
+Read-only.
+
+CPU hot-add/remove notification:
+-----------------------------------------------------
+QEMU sets/clears corresponding CPU bit on hot-add/remove event.
+CPU present map read by ACPI BIOS GPE.2 handler to notify OS of CPU
+hot-(un)plug events.
diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
index 88386d7..ebd1af9 100644
--- a/hw/acpi/piix4.c
+++ b/hw/acpi/piix4.c
@@ -48,19 +48,28 @@
 #define PCI_EJ_BASE 0xae08
 #define PCI_RMV_BASE 0xae0c
 
+#define PIIX4_PROC_BASE 0xaf00
+#define PIIX4_PROC_LEN 32
+
 #define PIIX4_PCI_HOTPLUG_STATUS 2
+#define PIIX4_CPU_HOTPLUG_STATUS 4
 
 struct pci_status {
     uint32_t up; /* deprecated, maintained for migration compatibility */
     uint32_t down;
 };
 
+struct CPUStatus {
+    uint8_t sts[PIIX4_PROC_LEN];
+};
+
 typedef struct PIIX4PMState {
     PCIDevice dev;
 
     MemoryRegion io;
     MemoryRegion io_gpe;
     MemoryRegion io_pci;
+    MemoryRegion io_cpu;
     ACPIREGS ar;
 
     APMState apm;
@@ -82,6 +91,9 @@ typedef struct PIIX4PMState {
     uint8_t disable_s3;
     uint8_t disable_s4;
     uint8_t s4_val;
+
+    struct CPUStatus gpe_cpu;
+    Notifier cpu_added_notifier;
 } PIIX4PMState;
 
 static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
@@ -100,8 +112,8 @@ static void pm_update_sci(PIIX4PMState *s)
                    ACPI_BITMASK_POWER_BUTTON_ENABLE |
                    ACPI_BITMASK_GLOBAL_LOCK_ENABLE |
                    ACPI_BITMASK_TIMER_ENABLE)) != 0) ||
-        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0])
-          & PIIX4_PCI_HOTPLUG_STATUS) != 0);
+        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) &
+          (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0);
 
     qemu_set_irq(s->irq, sci_level);
     /* schedule a timer interruption if needed */
@@ -585,6 +597,73 @@ static const MemoryRegionOps piix4_pci_ops = {
     },
 };
 
+static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned width)
+{
+    PIIX4PMState *s = opaque;
+    struct CPUStatus *cpus = &s->gpe_cpu;
+    uint64_t val = cpus->sts[addr];
+
+    return val;
+}
+
+static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
+                             unsigned int size)
+{
+    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
+}
+
+static const MemoryRegionOps cpu_hotplug_ops = {
+    .read = cpu_status_read,
+    .write = cpu_status_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 1,
+    },
+};
+
+typedef enum {
+    PLUG,
+    UNPLUG,
+} HotplugEventType;
+
+static void piix4_cpu_hotplug_req(PIIX4PMState *s, CPUState *cpu,
+                                  HotplugEventType action)
+{
+    struct CPUStatus *g = &s->gpe_cpu;
+    ACPIGPE *gpe = &s->ar.gpe;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t cpu_id;
+
+    assert(s != NULL);
+
+    *gpe->sts = *gpe->sts | PIIX4_CPU_HOTPLUG_STATUS;
+    cpu_id = k->get_arch_id(CPU(cpu));
+    if (action == PLUG) {
+        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+    } else {
+        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+    }
+    pm_update_sci(s);
+}
+
+static void piix4_cpu_added_req(Notifier *n, void *opaque)
+{
+    PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
+
+    piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
+}
+
+static void piix4_init_cpu_status(CPUState *cpu, void *data)
+{
+    struct CPUStatus *g = (struct CPUStatus *)data;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t id = k->get_arch_id(cpu);
+
+    g_assert((id / 8) < PIIX4_PROC_LEN);
+    g->sts[id / 8] |= (1 << (id % 8));
+}
+
 static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
                                 PCIHotplugState state);
 
@@ -600,6 +679,13 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
     memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
                                 &s->io_pci);
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
+
+    qemu_for_each_cpu(piix4_init_cpu_status, &s->gpe_cpu);
+    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
+                          PIIX4_PROC_LEN);
+    memory_region_add_subregion(parent, PIIX4_PROC_BASE, &s->io_cpu);
+    s->cpu_added_notifier.notify = piix4_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
 }
 
 static void enable_device(PIIX4PMState *s, int slot)
-- 
1.8.2

[Qemu-devel] [PATCH 11/21] target-i386: introduce apic-id property

[Igor Mammedov]

The property is used from board level to set APIC ID for CPUs it
creates. Do so in a new pc_new_cpu() helper, to be reused for hot-plug.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
Note:
  * pc_new_cpu() function will be reused later in CPU hot-plug hook.

v4:
  * after switching to qemu_for_each_cpu() in cpu_exists(), first CPU
    becomes visible to cpu_exists() early and setting property fails,
    skip APIC ID check if value to be set is the same as the current.
  * use error_propagate() in pc_new_cpu()
  * return CPU from pc_new_cpu(). Moved from "move APIC to ICC bus"
    to reduce its size.
v3:
  * user error_propagate() in property setter
v2:
  * use generic cpu_exists() instead of custom one
  * make apic-id dynamic property, so it won't be possible to use it
    as global property, since it's instance specific
---
 hw/i386/pc.c      |   29 ++++++++++++++++++++++++++++-
 target-i386/cpu.c |   42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 70 insertions(+), 1 deletions(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index bbc3bea..30ec338 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -890,9 +890,33 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
+static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+{
+    X86CPU *cpu;
+    Error *local_err = NULL;
+
+    cpu = cpu_x86_create(cpu_model, errp);
+    if (!cpu) {
+        return cpu;
+    }
+
+    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", &local_err);
+    object_property_set_bool(OBJECT(cpu), true, "realized", &local_err);
+
+    if (local_err) {
+        if (cpu != NULL) {
+            object_unref(OBJECT(cpu));
+            cpu = NULL;
+        }
+        error_propagate(errp, local_err);
+    }
+    return cpu;
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    Error *error = NULL;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -904,7 +928,10 @@ void pc_cpus_init(const char *cpu_model)
     }
 
     for (i = 0; i < smp_cpus; i++) {
-        if (!cpu_x86_init(cpu_model)) {
+        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        if (error) {
+            fprintf(stderr, "%s\n", error_get_pretty(error));
+            error_free(error);
             exit(1);
         }
     }
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index a415fa3..296ba58 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -1271,6 +1271,45 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque,
     cpu->env.tsc_khz = value / 1000;
 }
 
+static void x86_cpuid_get_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    int64_t value = cpu->env.cpuid_apic_id;
+
+    visit_type_int(v, &value, name, errp);
+}
+
+static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    const int64_t min = 0;
+    const int64_t max = UINT32_MAX;
+    Error *error = NULL;
+    int64_t value;
+
+    visit_type_int(v, &value, name, &error);
+    if (error) {
+        error_propagate(errp, error);
+        return;
+    }
+    if (value < min || value > max) {
+        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
+                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
+                   object_get_typename(obj), name, value, min, max);
+        error_propagate(errp, error);
+        return;
+    }
+
+    if ((value != cpu->env.cpuid_apic_id) && cpu_exists(value)) {
+        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
+        error_propagate(errp, error);
+        return;
+    }
+    cpu->env.cpuid_apic_id = value;
+}
+
 static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, const char *name)
 {
     x86_def_t *def;
@@ -2259,6 +2298,9 @@ static void x86_cpu_initfn(Object *obj)
     object_property_add(obj, "tsc-frequency", "int",
                         x86_cpuid_get_tsc_freq,
                         x86_cpuid_set_tsc_freq, NULL, NULL, NULL);
+    object_property_add(obj, "apic-id", "int",
+                        x86_cpuid_get_apic_id,
+                        x86_cpuid_set_apic_id, NULL, NULL, NULL);
 
     env->cpuid_apic_id = x86_cpu_apic_id_from_index(cs->cpu_index);
 
-- 
1.7.1

[Qemu-devel] [PATCH 12/21] target-i386: introduce ICC bus/device/bridge

[Igor Mammedov]

Provides hotplug-able bus for APIC and CPU.

* icc-bridge will serve as a parent for icc-bus and provide
  mmio mapping services to child icc-devices.
* icc-device will replace SysBusDevice as a parent of APIC
  and IOAPIC devices.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v3:
  * mv include/hw/i386/icc_bus.h into include/hw/cpu/
  * various style fixes
  * embed icc-bus inside icc-bridge and use qbus_create_inplace()
  * update MAINTAINERS with new files
v2:
  * Rebase on top the last HW reorganization series.
  * Move hw/icc_bus.c into hw/cpu/ and hw/icc_bus.h include/hw/i386/
---
 MAINTAINERS                        |    6 ++
 default-configs/i386-softmmu.mak   |    1 +
 default-configs/x86_64-softmmu.mak |    1 +
 hw/cpu/Makefile.objs               |    1 +
 hw/cpu/icc_bus.c                   |  104 ++++++++++++++++++++++++++++++++++++
 hw/i386/pc_piix.c                  |    7 +++
 hw/i386/pc_q35.c                   |    7 +++
 include/hw/cpu/icc_bus.h           |   58 ++++++++++++++++++++
 8 files changed, 185 insertions(+), 0 deletions(-)
 create mode 100644 hw/cpu/icc_bus.c
 create mode 100644 include/hw/cpu/icc_bus.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 4dfd8bf..be02724 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -644,6 +644,12 @@ F: qom/cpu.c
 F: include/qemu/cpu.h
 F: target-i386/cpu.c
 
+ICC Bus
+M: Igor Mammedov <imammedo@redhat.com>
+S: Supported
+F: include/hw/cpu/icc_bus.h
+F: hw/cpu/icc_bus.c
+
 Device Tree
 M: Peter Crosthwaite <peter.crosthwaite@petalogix.com>
 M: Alexander Graf <agraf@suse.de>
diff --git a/default-configs/i386-softmmu.mak b/default-configs/i386-softmmu.mak
index 368a776..833e722 100644
--- a/default-configs/i386-softmmu.mak
+++ b/default-configs/i386-softmmu.mak
@@ -44,3 +44,4 @@ CONFIG_LPC_ICH9=y
 CONFIG_PCI_Q35=y
 CONFIG_APIC=y
 CONFIG_IOAPIC=y
+CONFIG_ICC_BUS=y
diff --git a/default-configs/x86_64-softmmu.mak b/default-configs/x86_64-softmmu.mak
index 2711b83..b749eb7 100644
--- a/default-configs/x86_64-softmmu.mak
+++ b/default-configs/x86_64-softmmu.mak
@@ -44,3 +44,4 @@ CONFIG_LPC_ICH9=y
 CONFIG_PCI_Q35=y
 CONFIG_APIC=y
 CONFIG_IOAPIC=y
+CONFIG_ICC_BUS=y
diff --git a/hw/cpu/Makefile.objs b/hw/cpu/Makefile.objs
index a49ca04..4461ece 100644
--- a/hw/cpu/Makefile.objs
+++ b/hw/cpu/Makefile.objs
@@ -1,4 +1,5 @@
 obj-$(CONFIG_ARM11MPCORE) += arm11mpcore.o
 obj-$(CONFIG_ARM9MPCORE) += a9mpcore.o
 obj-$(CONFIG_ARM15MPCORE) += a15mpcore.o
+obj-$(CONFIG_ICC_BUS) += icc_bus.o
 
diff --git a/hw/cpu/icc_bus.c b/hw/cpu/icc_bus.c
new file mode 100644
index 0000000..f6091b9
--- /dev/null
+++ b/hw/cpu/icc_bus.c
@@ -0,0 +1,104 @@
+/* icc_bus.c
+ * emulate x86 ICC(Interrupt Controller Communications) bus
+ *
+ * Copyright (c) 2013 Red Hat, Inc
+ *
+ * Authors:
+ *     Igor Mammedov <imammedo@redhat.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>
+ */
+#include "hw/cpu/icc_bus.h"
+#include "hw/sysbus.h"
+
+/* icc-bridge implementation */
+static void icc_bus_init(Object *obj)
+{
+    BusState *b = BUS(obj);
+
+    b->allow_hotplug = true;
+}
+
+static const TypeInfo icc_bus_info = {
+    .name = TYPE_ICC_BUS,
+    .parent = TYPE_BUS,
+    .instance_size = sizeof(ICCBus),
+    .instance_init = icc_bus_init,
+};
+
+/* icc-device implementation */
+static void icc_device_realizefn(DeviceState *dev, Error **err)
+{
+    ICCDevice *id = ICC_DEVICE(dev);
+    ICCDeviceClass *k = ICC_DEVICE_GET_CLASS(id);
+    Error *local_err = NULL;
+
+    if (k->init) {
+        if (k->init(id) < 0) {
+            error_setg(&local_err, "%s initialization failed.",
+                       object_get_typename(OBJECT(dev)));
+            error_propagate(err, local_err);
+        }
+    }
+}
+
+static void icc_device_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *k = DEVICE_CLASS(klass);
+
+    k->realize = icc_device_realizefn;
+    k->bus_type = TYPE_ICC_BUS;
+}
+
+static const TypeInfo icc_device_info = {
+    .name = TYPE_ICC_DEVICE,
+    .parent = TYPE_DEVICE,
+    .abstract = true,
+    .instance_size = sizeof(ICCDevice),
+    .class_size = sizeof(ICCDeviceClass),
+    .class_init = icc_device_class_init,
+};
+
+/*  icc-bridge implementation */
+typedef struct ICCBridgeState {
+    /*< private >*/
+    SysBusDevice parent_obj;
+    ICCBus icc_bus;
+} ICCBridgeState;
+
+#define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
+
+static void icc_bridge_init(Object *obj)
+{
+    ICCBridgeState *s = ICC_BRIGDE(obj);
+
+    qbus_create_inplace(&s->icc_bus, TYPE_ICC_BUS, DEVICE(s), "icc-bus");
+}
+
+static const TypeInfo icc_bridge_info = {
+    .name  = TYPE_ICC_BRIDGE,
+    .parent = TYPE_SYS_BUS_DEVICE,
+    .instance_init  = icc_bridge_init,
+    .instance_size  = sizeof(ICCBridgeState),
+};
+
+
+static void icc_bus_register_types(void)
+{
+    type_register_static(&icc_bus_info);
+    type_register_static(&icc_device_info);
+    type_register_static(&icc_bridge_info);
+}
+
+type_init(icc_bus_register_types)
diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index 943758a..9a6f9fe 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -37,6 +37,7 @@
 #include "hw/kvm/clock.h"
 #include "sysemu/sysemu.h"
 #include "hw/sysbus.h"
+#include "hw/cpu/icc_bus.h"
 #include "sysemu/arch_init.h"
 #include "sysemu/blockdev.h"
 #include "hw/i2c/smbus.h"
@@ -85,8 +86,13 @@ static void pc_init1(MemoryRegion *system_memory,
     MemoryRegion *ram_memory;
     MemoryRegion *pci_memory;
     MemoryRegion *rom_memory;
+    DeviceState *icc_bridge;
     void *fw_cfg = NULL;
 
+    icc_bridge = qdev_create(NULL, TYPE_ICC_BRIDGE);
+    object_property_add_child(qdev_get_machine(), "icc-bridge",
+                              OBJECT(icc_bridge), NULL);
+
     pc_cpus_init(cpu_model);
     pc_acpi_init("acpi-dsdt.aml");
 
@@ -161,6 +167,7 @@ static void pc_init1(MemoryRegion *system_memory,
     if (pci_enabled) {
         ioapic_init_gsi(gsi_state, "i440fx");
     }
+    qdev_init_nofail(icc_bridge);
 
     pc_register_ferr_irq(gsi[13]);
 
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 6ac1a89..64c5a6c 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -41,6 +41,7 @@
 #include "hw/ide/pci.h"
 #include "hw/ide/ahci.h"
 #include "hw/usb.h"
+#include "hw/cpu/icc_bus.h"
 
 /* ICH9 AHCI has 6 ports */
 #define MAX_SATA_PORTS     6
@@ -85,6 +86,11 @@ static void pc_q35_init(QEMUMachineInitArgs *args)
     ICH9LPCState *ich9_lpc;
     PCIDevice *ahci;
     qemu_irq *cmos_s3;
+    DeviceState *icc_bridge;
+
+    icc_bridge = qdev_create(NULL, TYPE_ICC_BRIDGE);
+    object_property_add_child(qdev_get_machine(), "icc-bridge",
+                              OBJECT(icc_bridge), NULL);
 
     pc_cpus_init(cpu_model);
     pc_acpi_init("q35-acpi-dsdt.aml");
@@ -168,6 +174,7 @@ static void pc_q35_init(QEMUMachineInitArgs *args)
     if (pci_enabled) {
         ioapic_init_gsi(gsi_state, NULL);
     }
+    qdev_init_nofail(icc_bridge);
 
     pc_register_ferr_irq(gsi[13]);
 
diff --git a/include/hw/cpu/icc_bus.h b/include/hw/cpu/icc_bus.h
new file mode 100644
index 0000000..a0abc21
--- /dev/null
+++ b/include/hw/cpu/icc_bus.h
@@ -0,0 +1,58 @@
+/* icc_bus.h
+ * emulate x86 ICC(Interrupt Controller Communications) bus
+ *
+ * Copyright (c) 2013 Red Hat, Inc
+ *
+ * Authors:
+ *     Igor Mammedov <imammedo@redhat.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>
+ */
+#ifndef ICC_BUS_H
+#define ICC_BUS_H
+
+#include "hw/qdev-core.h"
+
+#define TYPE_ICC_BUS "icc-bus"
+
+#ifndef CONFIG_USER_ONLY
+
+typedef struct ICCBus {
+    /*< private >*/
+    BusState parent_obj;
+} ICCBus;
+
+#define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
+
+typedef struct ICCDevice {
+    /*< public >*/
+    DeviceState qdev;
+} ICCDevice;
+
+typedef struct ICCDeviceClass {
+    DeviceClass parent_class;
+    int (*init)(ICCDevice *dev);
+} ICCDeviceClass;
+
+#define TYPE_ICC_DEVICE "icc-device"
+#define ICC_DEVICE(obj) OBJECT_CHECK(ICCDevice, (obj), TYPE_ICC_DEVICE)
+#define ICC_DEVICE_CLASS(klass) \
+     OBJECT_CLASS_CHECK(ICCDeviceClass, (klass), TYPE_ICC_DEVICE)
+#define ICC_DEVICE_GET_CLASS(obj) \
+     OBJECT_GET_CLASS(ICCDeviceClass, (obj), TYPE_ICC_DEVICE)
+
+#define TYPE_ICC_BRIDGE "icc-bridge"
+
+#endif /* CONFIG_USER_ONLY */
+#endif
-- 
1.7.1

[Qemu-devel] [PATCH 13/21] target-i386: cpu: attach ICC bus to CPU on its creation

[Igor Mammedov]

X86CPU should have parent bus so it would be possible to unplug
it later. Set bus_type to TYPE_ICC_BUS for X86CPU type to make
device_add attach hotplugged CPU to ICC bus.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v2:
  * make sure that missing or ambiguous icc-bus will cause error
    on softmmu target.
---
 target-i386/cpu.c |   15 +++++++++++++++
 1 files changed, 15 insertions(+), 0 deletions(-)

diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 296ba58..8d3ad00 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -41,6 +41,7 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/cpu/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
 #include "hw/sysbus.h"
@@ -1619,6 +1620,19 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
     features = model_pieces[1];
 
     cpu = X86_CPU(object_new(TYPE_X86_CPU));
+#ifndef CONFIG_USER_ONLY
+    do {
+        bool ambiguous = false;
+        Object *icc_bus = object_resolve_path_type("icc-bus", TYPE_ICC_BUS,
+                                                   &ambiguous);
+        if ((icc_bus == NULL) || ambiguous) {
+            error_setg(&error, "Invalid icc-bus value");
+            goto out;
+        }
+        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
+        object_unref(OBJECT(cpu));
+   } while (0);
+#endif
     env = &cpu->env;
     env->cpu_model_str = cpu_model;
 
@@ -2330,6 +2344,7 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     xcc->parent_realize = dc->realize;
     dc->realize = x86_cpu_realizefn;
+    dc->bus_type = TYPE_ICC_BUS;
 
     xcc->parent_reset = cc->reset;
     cc->reset = x86_cpu_reset;
-- 
1.7.1

[Qemu-devel] [PATCH 14/21] target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE

[Igor Mammedov]

... and put APIC_SPACE_SIZE in public header so that it could be
reused later elsewhere.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Andreas Färber <afaerber@suse.de>
---
 hw/i386/kvm/apic.c              |    2 +-
 hw/intc/apic.c                  |    2 +-
 hw/xen/xen_apic.c               |    2 +-
 include/hw/i386/apic_internal.h |    2 --
 target-i386/cpu.h               |    1 +
 5 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/hw/i386/kvm/apic.c b/hw/i386/kvm/apic.c
index c6ff982..8f80425 100644
--- a/hw/i386/kvm/apic.c
+++ b/hw/i386/kvm/apic.c
@@ -174,7 +174,7 @@ static const MemoryRegionOps kvm_apic_io_ops = {
 static void kvm_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &kvm_apic_io_ops, s, "kvm-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     if (kvm_has_gsi_routing()) {
         msi_supported = true;
diff --git a/hw/intc/apic.c b/hw/intc/apic.c
index 2d79a9e..756dff0 100644
--- a/hw/intc/apic.c
+++ b/hw/intc/apic.c
@@ -874,7 +874,7 @@ static const MemoryRegionOps apic_io_ops = {
 static void apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &apic_io_ops, s, "apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     s->timer = qemu_new_timer_ns(vm_clock, apic_timer, s);
     local_apics[s->idx] = s;
diff --git a/hw/xen/xen_apic.c b/hw/xen/xen_apic.c
index a2eb8a1..a958782 100644
--- a/hw/xen/xen_apic.c
+++ b/hw/xen/xen_apic.c
@@ -39,7 +39,7 @@ static const MemoryRegionOps xen_apic_io_ops = {
 static void xen_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &xen_apic_io_ops, s, "xen-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
 #if defined(CONFIG_XEN_CTRL_INTERFACE_VERSION) \
     && CONFIG_XEN_CTRL_INTERFACE_VERSION >= 420
diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index 578241f..aac6290 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -66,8 +66,6 @@
 
 #define MAX_APICS 255
 
-#define MSI_SPACE_SIZE                  0x100000
-
 typedef struct APICCommonState APICCommonState;
 
 #define TYPE_APIC_COMMON "apic-common"
diff --git a/target-i386/cpu.h b/target-i386/cpu.h
index a1614e8..ab151d5 100644
--- a/target-i386/cpu.h
+++ b/target-i386/cpu.h
@@ -1270,5 +1270,6 @@ uint32_t x86_cpu_apic_id_from_index(unsigned int cpu_index);
 void enable_compat_apic_id_mode(void);
 
 #define APIC_DEFAULT_ADDRESS 0xfee00000
+#define APIC_SPACE_SIZE      0x100000
 
 #endif /* CPU_I386_H */
-- 
1.7.1

[Qemu-devel] [PATCH 15/21] target-i386: kvmvapic: make expilict dependency on sysbus.h

[Igor Mammedov]

Allows kvmvapic to compile if sysbus.h is removed from apic_internal.h,
from which it is indirectly included.
sysbus.h will be removed from apic_internal.h after converting
APICs to ICCDevice.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
Note:
  split it in separate patch from "move APIC to ICC bus" patch to
  simplify review. Feel free to squash it back.
---
 hw/i386/kvmvapic.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/hw/i386/kvmvapic.c b/hw/i386/kvmvapic.c
index 3a10c07..5b558aa 100644
--- a/hw/i386/kvmvapic.c
+++ b/hw/i386/kvmvapic.c
@@ -12,6 +12,7 @@
 #include "sysemu/cpus.h"
 #include "sysemu/kvm.h"
 #include "hw/i386/apic_internal.h"
+#include "hw/sysbus.h"
 
 #define VAPIC_IO_PORT           0x7e
 
-- 
1.7.1

[Qemu-devel] [PATCH 16/21] target-i386: move APIC to ICC bus

[Igor Mammedov]

It allows APIC to be hotplugged

 * map APIC's mmio at board level if it is present
 * do not register mmio region for each APIC, since
   only one is used/mapped

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v2:
  * various style fixes
  * split out sysbus.h inclusion in kvmvapic.c to simplify review
  * squash pc_new_cpu() hunk into "introduce apic-id property" patch
    to simplify review
  * use qdev_get_parent_bus() to get icc-bus for apic
---
 hw/cpu/icc_bus.c                |   10 ++++++++++
 hw/i386/pc.c                    |   14 +++++++++++++-
 hw/intc/apic_common.c           |   18 ++++++++++++------
 include/hw/cpu/icc_bus.h        |    4 ++++
 include/hw/i386/apic_internal.h |    6 +++---
 target-i386/cpu.c               |   16 +++-------------
 6 files changed, 45 insertions(+), 23 deletions(-)

diff --git a/hw/cpu/icc_bus.c b/hw/cpu/icc_bus.c
index f6091b9..9715e07 100644
--- a/hw/cpu/icc_bus.c
+++ b/hw/cpu/icc_bus.c
@@ -75,6 +75,7 @@ typedef struct ICCBridgeState {
     /*< private >*/
     SysBusDevice parent_obj;
     ICCBus icc_bus;
+    MemoryRegion apic_container;
 } ICCBridgeState;
 
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
@@ -82,8 +83,17 @@ typedef struct ICCBridgeState {
 static void icc_bridge_init(Object *obj)
 {
     ICCBridgeState *s = ICC_BRIGDE(obj);
+    SysBusDevice *sb = SYS_BUS_DEVICE(obj);
 
     qbus_create_inplace(&s->icc_bus, TYPE_ICC_BUS, DEVICE(s), "icc-bus");
+
+    /* Do not change order of registering regions,
+     * APIC must be first registered region, board maps it by 0 index
+     */
+    memory_region_init(&s->apic_container, "icc-apic-container",
+                       APIC_SPACE_SIZE);
+    sysbus_init_mmio(sb, &s->apic_container);
+    s->icc_bus.apic_address_space = &s->apic_container;
 }
 
 static const TypeInfo icc_bridge_info = {
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 30ec338..a09b7ee 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -53,6 +53,7 @@
 #include "qemu/bitmap.h"
 #include "qemu/config-file.h"
 #include "hw/acpi/acpi.h"
+#include "hw/cpu/icc_bus.h"
 
 /* debug PC/ISA interrupts */
 //#define DEBUG_IRQ
@@ -916,7 +917,9 @@ static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    X86CPU *cpu = NULL;
     Error *error = NULL;
+    SysBusDevice *icc_bridge;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -927,14 +930,23 @@ void pc_cpus_init(const char *cpu_model)
 #endif
     }
 
+    icc_bridge = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge",
+                                                 TYPE_ICC_BRIDGE, NULL));
+
     for (i = 0; i < smp_cpus; i++) {
-        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
         if (error) {
             fprintf(stderr, "%s\n", error_get_pretty(error));
             error_free(error);
             exit(1);
         }
     }
+
+    /* map APIC MMIO area if CPU has APIC */
+    if (cpu && cpu->env.apic_state) {
+        /* XXX: what if the base changes? */
+        sysbus_mmio_map_overlap(icc_bridge, 0, APIC_DEFAULT_ADDRESS, 0x1000);
+    }
 }
 
 void pc_acpi_init(const char *default_dsdt)
diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
index e0ae07a..b03e904 100644
--- a/hw/intc/apic_common.c
+++ b/hw/intc/apic_common.c
@@ -21,6 +21,8 @@
 #include "hw/i386/apic_internal.h"
 #include "trace.h"
 #include "sysemu/kvm.h"
+#include "hw/qdev.h"
+#include "hw/sysbus.h"
 
 static int apic_irq_delivered;
 bool apic_report_tpr_access;
@@ -282,12 +284,13 @@ static int apic_load_old(QEMUFile *f, void *opaque, int version_id)
     return 0;
 }
 
-static int apic_init_common(SysBusDevice *dev)
+static int apic_init_common(ICCDevice *dev)
 {
     APICCommonState *s = APIC_COMMON(dev);
     APICCommonClass *info;
     static DeviceState *vapic;
     static int apic_no;
+    static bool mmio_registered;
 
     if (apic_no >= MAX_APICS) {
         return -1;
@@ -296,8 +299,11 @@ static int apic_init_common(SysBusDevice *dev)
 
     info = APIC_COMMON_GET_CLASS(s);
     info->init(s);
-
-    sysbus_init_mmio(dev, &s->io_memory);
+    if (!mmio_registered) {
+        ICCBus *b = ICC_BUS(qdev_get_parent_bus(DEVICE(dev)));
+        memory_region_add_subregion(b->apic_address_space, 0, &s->io_memory);
+        mmio_registered = true;
+    }
 
     /* Note: We need at least 1M to map the VAPIC option ROM */
     if (!vapic && s->vapic_control & VAPIC_ENABLE_MASK &&
@@ -375,19 +381,19 @@ static Property apic_properties_common[] = {
 
 static void apic_common_class_init(ObjectClass *klass, void *data)
 {
-    SysBusDeviceClass *sc = SYS_BUS_DEVICE_CLASS(klass);
+    ICCDeviceClass *idc = ICC_DEVICE_CLASS(klass);
     DeviceClass *dc = DEVICE_CLASS(klass);
 
     dc->vmsd = &vmstate_apic_common;
     dc->reset = apic_reset_common;
     dc->no_user = 1;
     dc->props = apic_properties_common;
-    sc->init = apic_init_common;
+    idc->init = apic_init_common;
 }
 
 static const TypeInfo apic_common_type = {
     .name = TYPE_APIC_COMMON,
-    .parent = TYPE_SYS_BUS_DEVICE,
+    .parent = TYPE_ICC_DEVICE,
     .instance_size = sizeof(APICCommonState),
     .class_size = sizeof(APICCommonClass),
     .class_init = apic_common_class_init,
diff --git a/include/hw/cpu/icc_bus.h b/include/hw/cpu/icc_bus.h
index a0abc21..53ef223 100644
--- a/include/hw/cpu/icc_bus.h
+++ b/include/hw/cpu/icc_bus.h
@@ -22,6 +22,7 @@
 #ifndef ICC_BUS_H
 #define ICC_BUS_H
 
+#include "exec/memory.h"
 #include "hw/qdev-core.h"
 
 #define TYPE_ICC_BUS "icc-bus"
@@ -31,6 +32,9 @@
 typedef struct ICCBus {
     /*< private >*/
     BusState parent_obj;
+
+    /*< public >*/
+    MemoryRegion *apic_address_space;
 } ICCBus;
 
 #define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index aac6290..1b0a7fb 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -21,7 +21,7 @@
 #define QEMU_APIC_INTERNAL_H
 
 #include "exec/memory.h"
-#include "hw/sysbus.h"
+#include "hw/cpu/icc_bus.h"
 #include "qemu/timer.h"
 
 /* APIC Local Vector Table */
@@ -78,7 +78,7 @@ typedef struct APICCommonState APICCommonState;
 
 typedef struct APICCommonClass
 {
-    SysBusDeviceClass parent_class;
+    ICCDeviceClass parent_class;
 
     void (*init)(APICCommonState *s);
     void (*set_base)(APICCommonState *s, uint64_t val);
@@ -92,7 +92,7 @@ typedef struct APICCommonClass
 } APICCommonClass;
 
 struct APICCommonState {
-    SysBusDevice busdev;
+    ICCDevice busdev;
 
     MemoryRegion io_memory;
     X86CPU *cpu;
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 8d3ad00..c648ea7 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -41,10 +41,10 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/qdev-properties.h"
 #include "hw/cpu/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
-#include "hw/sysbus.h"
 #include "hw/i386/apic_internal.h"
 #endif
 
@@ -2119,6 +2119,7 @@ static void mce_init(X86CPU *cpu)
 static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
 {
     CPUX86State *env = &cpu->env;
+    DeviceState *dev = DEVICE(cpu);
     APICCommonState *apic;
     const char *apic_type = "apic";
 
@@ -2128,7 +2129,7 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
         apic_type = "xen-apic";
     }
 
-    env->apic_state = qdev_try_create(NULL, apic_type);
+    env->apic_state = qdev_try_create(qdev_get_parent_bus(dev), apic_type);
     if (env->apic_state == NULL) {
         error_setg(errp, "APIC device '%s' could not be created", apic_type);
         return;
@@ -2145,7 +2146,6 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
 static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
 {
     CPUX86State *env = &cpu->env;
-    static int apic_mapped;
 
     if (env->apic_state == NULL) {
         return;
@@ -2156,16 +2156,6 @@ static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
                    object_get_typename(OBJECT(env->apic_state)));
         return;
     }
-
-    /* XXX: mapping more APICs at the same memory location */
-    if (apic_mapped == 0) {
-        /* NOTE: the APIC is directly connected to the CPU - it is not
-           on the global memory bus. */
-        /* XXX: what if the base changes? */
-        sysbus_mmio_map_overlap(SYS_BUS_DEVICE(env->apic_state), 0,
-                                APIC_DEFAULT_ADDRESS, 0x1000);
-        apic_mapped = 1;
-    }
 }
 #else
 static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
-- 
1.7.1

[Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Igor Mammedov]

kvm/ioapic is relying on the fact that SysBus device
maps mmio regions with offset counted from start of system memory.
But if ioapic's region is moved to another sub-region which doesn't
start at the beginning of system memory then using offset isn't correct.

Fix kvm/ioapic by providing and using helper function that returns
absolute region address in respective address space.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
Note:
  next patch "move IOAPIC to ICC bus" converts IOAPICs to ICCDevice
  and breaks SysBus device assumption used by kvm/ioapic.
---
 hw/i386/kvm/ioapic.c  |    2 +-
 include/exec/memory.h |   10 ++++++++++
 memory.c              |   11 +++++++++++
 3 files changed, 22 insertions(+), 1 deletions(-)

diff --git a/hw/i386/kvm/ioapic.c b/hw/i386/kvm/ioapic.c
index a3bd519..b80d41a 100644
--- a/hw/i386/kvm/ioapic.c
+++ b/hw/i386/kvm/ioapic.c
@@ -96,7 +96,7 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
 
     kioapic->id = s->id;
     kioapic->ioregsel = s->ioregsel;
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = memory_region_get_address(&s->io_memory);
     kioapic->irr = s->irr;
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         kioapic->redirtbl[i].bits = s->ioredtbl[i];
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 9e88320..954f353 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -706,6 +706,16 @@ void memory_region_set_enabled(MemoryRegion *mr, bool enabled);
 void memory_region_set_address(MemoryRegion *mr, hwaddr addr);
 
 /*
+ * memory_region_get_address: get current the address of a region
+ *
+ * Returns the absolute address of a region.
+ * May be used on regions that are currently part of a memory hierarchy.
+ *
+ * @mr: the region being queried
+ */
+hwaddr memory_region_get_address(MemoryRegion *mr);
+
+/*
  * memory_region_set_alias_offset: dynamically update a memory alias's offset
  *
  * Dynamically updates the offset into the target region that an alias points
diff --git a/memory.c b/memory.c
index 75ca281..0651050 100644
--- a/memory.c
+++ b/memory.c
@@ -1413,6 +1413,17 @@ void memory_region_set_address(MemoryRegion *mr, hwaddr addr)
     memory_region_transaction_commit();
 }
 
+hwaddr memory_region_get_address(MemoryRegion *mr)
+{
+    hwaddr addr = mr->addr;
+
+    while (mr->parent) {
+        mr = mr->parent;
+        addr += mr->addr;
+    }
+    return addr;
+}
+
 void memory_region_set_alias_offset(MemoryRegion *mr, hwaddr offset)
 {
     assert(mr->alias);
-- 
1.7.1

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Paolo Bonzini]

Il 23/04/2013 10:29, Igor Mammedov ha scritto:
> kvm/ioapic is relying on the fact that SysBus device
> maps mmio regions with offset counted from start of system memory.
> But if ioapic's region is moved to another sub-region which doesn't
> start at the beginning of system memory then using offset isn't correct.
> 
> Fix kvm/ioapic by providing and using helper function that returns
> absolute region address in respective address space.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>

Looks good.

Paolo

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Peter Maydell]

On 23 April 2013 09:29, Igor Mammedov <imammedo@redhat.com> wrote:
>  /*
> + * memory_region_get_address: get current the address of a region
> + *
> + * Returns the absolute address of a region.
> + * May be used on regions that are currently part of a memory hierarchy.
> + *
> + * @mr: the region being queried
> + */
> +hwaddr memory_region_get_address(MemoryRegion *mr);

This doesn't make sense as a memory API in my opinion. There's
no such thing as the "absolute address of a MemoryRegion".

-- PMM

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Paolo Bonzini]

Il 23/04/2013 19:06, Peter Maydell ha scritto:
>> >  /*
>> > + * memory_region_get_address: get current the address of a region
>> > + *
>> > + * Returns the absolute address of a region.
>> > + * May be used on regions that are currently part of a memory hierarchy.
>> > + *
>> > + * @mr: the region being queried
>> > + */
>> > +hwaddr memory_region_get_address(MemoryRegion *mr);
> This doesn't make sense as a memory API in my opinion. There's
> no such thing as the "absolute address of a MemoryRegion".

What about also passing an AddressSpace and stopping at the address
space's root?  Would that make more sense?

Paolo

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Peter Maydell]

On 23 April 2013 18:14, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 23/04/2013 19:06, Peter Maydell ha scritto:
>>> >  /*
>>> > + * memory_region_get_address: get current the address of a region
>>> > + *
>>> > + * Returns the absolute address of a region.
>>> > + * May be used on regions that are currently part of a memory hierarchy.
>>> > + *
>>> > + * @mr: the region being queried
>>> > + */
>>> > +hwaddr memory_region_get_address(MemoryRegion *mr);
>> This doesn't make sense as a memory API in my opinion. There's
>> no such thing as the "absolute address of a MemoryRegion".
>
> What about also passing an AddressSpace and stopping at the address
> space's root?  Would that make more sense?

That would be an improvement, but it still requires this
device to know what address space it's mapped into, which
is a bit ugly.

-- PMM

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Jan Kiszka]

On 2013-04-23 19:26, Peter Maydell wrote:
> On 23 April 2013 18:14, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> Il 23/04/2013 19:06, Peter Maydell ha scritto:
>>>>>  /*
>>>>> + * memory_region_get_address: get current the address of a region
>>>>> + *
>>>>> + * Returns the absolute address of a region.
>>>>> + * May be used on regions that are currently part of a memory hierarchy.
>>>>> + *
>>>>> + * @mr: the region being queried
>>>>> + */
>>>>> +hwaddr memory_region_get_address(MemoryRegion *mr);
>>> This doesn't make sense as a memory API in my opinion. There's
>>> no such thing as the "absolute address of a MemoryRegion".
>>
>> What about also passing an AddressSpace and stopping at the address
>> space's root?  Would that make more sense?
> 
> That would be an improvement, but it still requires this
> device to know what address space it's mapped into, which
> is a bit ugly.

Unfortunately the KVM API mandates this knowledge (IOAPIC state contains
the absolute mapping address). So we need some channel to explore where
this device is mapped.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Peter Maydell]

On 23 April 2013 18:39, Jan Kiszka <jan.kiszka@siemens.com> wrote:
> On 2013-04-23 19:26, Peter Maydell wrote:
>> That would be an improvement, but it still requires this
>> device to know what address space it's mapped into, which
>> is a bit ugly.
>
> Unfortunately the KVM API mandates this knowledge (IOAPIC state contains
> the absolute mapping address). So we need some channel to explore where
> this device is mapped.

For ARM we defer the mechanics of "tell the kernel where this memory
region is" from the device implementation (hw/intc/arm_gic_kvm.c) to
generic code in target-arm/kvm.c, which I personally think is a
neat way to solve this problem. It looks like the x86 ABI has
tied up one-time config stuff (like the base address) with
generic state save/load though, so that doesn't quite work for
the x86 case.

Anyway, I think I've previously argued on this list against having
a "return address of this MR in this AddressSpace" function in the
API, but I've changed my mind on that: it's kind of equivalent to
the listener API functionality except it's a one-off query.

NB: I don't think we should call it memory_region_get_address()
though -- that implies a non-existent parallel to the current
memory_region_set_address().

thanks
-- PMM

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Paolo Bonzini]

Il 23/04/2013 20:00, Peter Maydell ha scritto:
> For ARM we defer the mechanics of "tell the kernel where this memory
> region is" from the device implementation (hw/intc/arm_gic_kvm.c) to
> generic code in target-arm/kvm.c, which I personally think is a
> neat way to solve this problem.

Yes, it is.

> NB: I don't think we should call it memory_region_get_address()
> though -- that implies a non-existent parallel to the current
> memory_region_set_address().

Hmm, good point.  address_space_get_region_addr()?

Paolo

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Peter Maydell]

On 23 April 2013 22:02, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 23/04/2013 20:00, Peter Maydell ha scritto:
>> NB: I don't think we should call it memory_region_get_address()
>> though -- that implies a non-existent parallel to the current
>> memory_region_set_address().
>
> Hmm, good point.  address_space_get_region_addr()?

Maybe we should make this parallel-ish to memory_region_find()
(which finds an MR within an AS by addr/size)?

Actually, I think I'd favour cleaning up some of the existing
API which has parameters named "address_space" which are
MemoryRegion*s. I think these probably predate the AddressSpace
type and should be updated to use it (and be renamed to fit too).

So how about:
 MemoryRegionSection memory_region_find(MemoryRegion *address_space,
                                       hwaddr addr, uint64_t size);
becomes
 MemoryRegionSection address_space_find_region_by_addr(
                                       AddressSpace *address_space,
                                       hwaddr addr, uint64_t size);
(bit of a mouthful, but never mind)

 void memory_global_sync_dirty_bitmap(MemoryRegion *address_space);
becomes
 void address_space_sync_dirty_bitmap(AddressSpace *address_space);

(in both cases the first thing the implementation does is call
memory_region_to_address_space() to get the AddressSpace* anyway.)

and our new function is
 MemoryRegionSection address_space_find_region(AddressSpace *as,
                                               MemoryRegion *mr);

?

(we don't have to do the tidyup first, but the new function
signature makes more sense viewed in the light of the others)

thanks
-- PMM

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Paolo Bonzini]

Il 23/04/2013 23:39, Peter Maydell ha scritto:
>> >
>> > Hmm, good point.  address_space_get_region_addr()?
> Maybe we should make this parallel-ish to memory_region_find()
> (which finds an MR within an AS by addr/size)?
> 
> Actually, I think I'd favour cleaning up some of the existing
> API which has parameters named "address_space" which are
> MemoryRegion*s. I think these probably predate the AddressSpace
> type and should be updated to use it (and be renamed to fit too).
> 
> So how about:
>  MemoryRegionSection memory_region_find(MemoryRegion *address_space,
>                                        hwaddr addr, uint64_t size);
> becomes
>  MemoryRegionSection address_space_find_region_by_addr(
>                                        AddressSpace *address_space,
>                                        hwaddr addr, uint64_t size);
> (bit of a mouthful, but never mind)
> 
>  void memory_global_sync_dirty_bitmap(MemoryRegion *address_space);
> becomes
>  void address_space_sync_dirty_bitmap(AddressSpace *address_space);
> 
> (in both cases the first thing the implementation does is call
> memory_region_to_address_space() to get the AddressSpace* anyway.)
> 
> and our new function is
>  MemoryRegionSection address_space_find_region(AddressSpace *as,
>                                                MemoryRegion *mr);
> 
> ?
> 
> (we don't have to do the tidyup first, but the new function
> signature makes more sense viewed in the light of the others)

I can look at it for 1.6, since I've already forward ported Avi's IOMMU
patches and added s/DMAContext/AddressSpace/ on top.

What about 1.5 though?

Paolo

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Peter Maydell]

On 23 April 2013 22:46, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 23/04/2013 23:39, Peter Maydell ha scritto:

>> and our new function is
>>  MemoryRegionSection address_space_find_region(AddressSpace *as,
>>                                                MemoryRegion *mr);
>>
>> ?
>>
>> (we don't have to do the tidyup first, but the new function
>> signature makes more sense viewed in the light of the others)
>
> I can look at it for 1.6, since I've already forward ported Avi's IOMMU
> patches and added s/DMAContext/AddressSpace/ on top.
>
> What about 1.5 though?

Ah, I was going to ask if this was a 1.5 series. If so, maybe
just add and use that new function? Anyway, I don't want to
bikeshed the function too much, as long as it takes an
AddressSpace* to search and lets the caller distinguish
"couldn't find the region" from valid actual addresses.

thanks
-- PMM

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Paolo Bonzini]

> So how about:
>  MemoryRegionSection memory_region_find(MemoryRegion *address_space,
>                                        hwaddr addr, uint64_t size);
> becomes
>  MemoryRegionSection address_space_find_region_by_addr(
>                                        AddressSpace *address_space,
>                                        hwaddr addr, uint64_t size);
> (bit of a mouthful, but never mind)
> 
>  void memory_global_sync_dirty_bitmap(MemoryRegion *address_space);
> becomes
>  void address_space_sync_dirty_bitmap(AddressSpace *address_space);

I think the latter makes definite sense, I am not quite as sure about the
former.

Looking at framebuffer.c's use of memory_region_find, here you really want
to go from the "local" view to a global one in order to look at global
data structures such as the dirty bitmap.  So it is right to have
memory_region_find as a MemoryRegion operation, even though right now
it is always passed an AddressSpace.

Communicating the absolute address to KVM is another example of
this local->global translation.  Hence, my suggestion is to remove
memory_region_find's limitation on the first argument, and make it work
on nested regions too.  With this change, memory_region_find() nicely
fits Igor's use case.

Not coincidentially, the additional code in memory_region_find() is
very similar to Igor's memory_region_get_address().

See the attached patch, which I tested on master.  Igor, can you try it
with iccbus?

Paolo

-------------- 8< --------------------
>From 953460fa8ee9f9e7243ea34eb57a901102be9307 Mon Sep 17 00:00:00 2001
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Tue, 23 Apr 2013 10:29:51 +0200
Subject: [RFC PATCH 17/21] extend memory_region_find() and use it in kvm/ioapic

kvm/ioapic is relying on the fact that SysBus device
maps mmio regions with offset counted from start of system memory.
But if ioapic's region is moved to another sub-region which doesn't
start at the beginning of system memory then using offset isn't correct.

To fix kvm/ioapic, extend memory_region_find() so that it can help
retrieving the absolute region address and the respective address space.

The patch is a no-op in case mr is parentless, i.e. mr->addr == 0
and mr->parent == NULL.

Based on a patch by Igor Mammedov. <imammedo@redhat.com>

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 hw/i386/kvm/ioapic.c  |    9 ++++++++-
 include/exec/memory.h |   13 +++++++------
 memory.c              |   19 ++++++++++++++-----
 3 files changed, 29 insertions(+), 12 deletions(-)

diff --git a/hw/i386/kvm/ioapic.c b/hw/i386/kvm/ioapic.c
index a3bd519..7564d07 100644
--- a/hw/i386/kvm/ioapic.c
+++ b/hw/i386/kvm/ioapic.c
@@ -89,14 +89,21 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
 {
     struct kvm_irqchip chip;
     struct kvm_ioapic_state *kioapic;
+    MemoryRegionSection mrs;
     int ret, i;
 
+    mrs = memory_region_find(&s->io_memory, 0, 0x1000);
+    if (mrs.mr != &s->io_memory || mrs.offset_within_region != 0) {
+        fprintf(stderr, "cannot find IOAPIC base\n");
+        abort();
+    }
+
     chip.chip_id = KVM_IRQCHIP_IOAPIC;
     kioapic = &chip.chip.ioapic;
 
     kioapic->id = s->id;
     kioapic->ioregsel = s->ioregsel;
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = mrs.offset_within_address_space;
     kioapic->irr = s->irr;
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         kioapic->redirtbl[i].bits = s->ioredtbl[i];
diff --git a/include/exec/memory.h b/include/exec/memory.h
index eb9e659..5854d19 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -725,17 +725,18 @@ void memory_region_set_alias_offset(MemoryRegion *mr,
  *
  * Returns a #MemoryRegionSection that describes a contiguous overlap.
  * It will have the following characteristics:
- *    .@offset_within_address_space >= @addr
- *    .@offset_within_address_space + .@size <= @addr + @size
  *    .@size = 0 iff no overlap was found
  *    .@mr is non-%NULL iff an overlap was found
  *
- * @address_space: a top-level (i.e. parentless) region that contains
- *       the region to be found
- * @addr: start of the area within @address_space to be searched
+ * If @mr is parent-less,
+ *    .@offset_within_address_space >= @addr
+ *    .@offset_within_address_space + .@size <= @addr + @size
+ *
+ * @mr: a (possibly indirect) parent that contains the region to be found
+ * @addr: start of the area within @as to be searched
  * @size: size of the area to be searched
  */
-MemoryRegionSection memory_region_find(MemoryRegion *address_space,
+MemoryRegionSection memory_region_find(MemoryRegion *mr,
                                        hwaddr addr, uint64_t size);
 
 /**
diff --git a/memory.c b/memory.c
index c82bd12..dba0a4b 100644
--- a/memory.c
+++ b/memory.c
@@ -1451,15 +1451,24 @@ static FlatRange *address_space_lookup(AddressSpace *as, AddrRange addr)
                    sizeof(FlatRange), cmp_flatrange_addr);
 }
 
-MemoryRegionSection memory_region_find(MemoryRegion *address_space,
+MemoryRegionSection memory_region_find(MemoryRegion *mr,
                                        hwaddr addr, uint64_t size)
 {
-    AddressSpace *as = memory_region_to_address_space(address_space);
-    AddrRange range = addrrange_make(int128_make64(addr),
-                                     int128_make64(size));
-    FlatRange *fr = address_space_lookup(as, range);
     MemoryRegionSection ret = { .mr = NULL, .size = 0 };
+    MemoryRegion *root;
+    AddressSpace *as;
+    AddrRange range;
+    FlatRange *fr;
+
+    addr += mr->addr;
+    for (root = mr; root->parent; ) {
+        root = root->parent;
+        addr += root->addr;
+    }
 
+    as = memory_region_to_address_space(root);
+    range = addrrange_make(int128_make64(addr), int128_make64(size));
+    fr = address_space_lookup(as, range);
     if (!fr) {
         return ret;
     }
-- 
1.7.1

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Paolo Bonzini]

Il 24/04/2013 12:22, Paolo Bonzini ha scritto:
> diff --git a/memory.c b/memory.c
> index c82bd12..dba0a4b 100644
> --- a/memory.c
> +++ b/memory.c
> @@ -1451,15 +1451,24 @@ static FlatRange *address_space_lookup(AddressSpace *as, AddrRange addr)
>                     sizeof(FlatRange), cmp_flatrange_addr);
>  }
>  
> -MemoryRegionSection memory_region_find(MemoryRegion *address_space,
> +MemoryRegionSection memory_region_find(MemoryRegion *mr,
>                                         hwaddr addr, uint64_t size)
>  {
> -    AddressSpace *as = memory_region_to_address_space(address_space);
> -    AddrRange range = addrrange_make(int128_make64(addr),
> -                                     int128_make64(size));
> -    FlatRange *fr = address_space_lookup(as, range);
>      MemoryRegionSection ret = { .mr = NULL, .size = 0 };
> +    MemoryRegion *root;
> +    AddressSpace *as;
> +    AddrRange range;
> +    FlatRange *fr;
> +
> +    addr += mr->addr;
> +    for (root = mr; root->parent; ) {
> +        root = root->parent;
> +        addr += root->addr;
> +    }
>  
> +    as = memory_region_to_address_space(root);
> +    range = addrrange_make(int128_make64(addr), int128_make64(size));
> +    fr = address_space_lookup(as, range);
>      if (!fr) {
>          return ret;
>      }
> 

Looking at the code again, mrs.address_space is not filled in.  This should
be squashed in too for completeness.

diff --git a/memory.c b/memory.c
index dba0a4b..1916937 100644
--- a/memory.c
+++ b/memory.c
@@ -1479,6 +1479,7 @@ MemoryRegionSection memory_region_find(MemoryRegion *mr,
     }
 
     ret.mr = fr->mr;
+    ret.address_space = as;
     range = addrrange_intersection(range, fr->addr);
     ret.offset_within_region = fr->offset_in_region;
     ret.offset_within_region += int128_get64(int128_sub(range.start,

Paolo

[Qemu-devel] [PATCH 15/19 v2] extend memory_region_find() and use it in kvm/ioapic

[Igor Mammedov]

From: Paolo Bonzini <pbonzini@redhat.com>

kvm/ioapic is relying on the fact that SysBus device
maps mmio regions with offset counted from start of system memory.
But if ioapic's region is moved to another sub-region which doesn't
start at the beginning of system memory then using offset isn't correct.

To fix kvm/ioapic, extend memory_region_find() so that it can help
retrieving the absolute region address and the respective address space.

The patch is a no-op in case mr is parentless, i.e. mr->addr == 0
and mr->parent == NULL.

In addition fill in MemoryRegionSection.as wich was missing in original
memory_region_find().

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 hw/i386/kvm/ioapic.c  |    9 ++++++++-
 include/exec/memory.h |   13 +++++++------
 memory.c              |   20 +++++++++++++++-----
 3 files changed, 30 insertions(+), 12 deletions(-)

diff --git a/hw/i386/kvm/ioapic.c b/hw/i386/kvm/ioapic.c
index a3bd519..dc6ddab 100644
--- a/hw/i386/kvm/ioapic.c
+++ b/hw/i386/kvm/ioapic.c
@@ -89,14 +89,21 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
 {
     struct kvm_irqchip chip;
     struct kvm_ioapic_state *kioapic;
+    MemoryRegionSection mrs;
     int ret, i;
 
+    mrs = memory_region_find(&s->io_memory, 0, 0x1000);
+    if (mrs.mr != &s->io_memory || mrs.offset_within_region != 0) {
+        fprintf(stderr, "cannot find IOAPIC base\n");
+        abort();
+    }
+
     chip.chip_id = KVM_IRQCHIP_IOAPIC;
     kioapic = &chip.chip.ioapic;
 
     kioapic->id = s->id;
     kioapic->ioregsel = s->ioregsel;
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = mrs.offset_within_address_space;
     kioapic->irr = s->irr;
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         kioapic->redirtbl[i].bits = s->ioredtbl[i];
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 9e88320..efe210b 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -725,17 +725,18 @@ void memory_region_set_alias_offset(MemoryRegion *mr,
  *
  * Returns a #MemoryRegionSection that describes a contiguous overlap.
  * It will have the following characteristics:
- *    .@offset_within_address_space >= @addr
- *    .@offset_within_address_space + .@size <= @addr + @size
  *    .@size = 0 iff no overlap was found
  *    .@mr is non-%NULL iff an overlap was found
  *
- * @address_space: a top-level (i.e. parentless) region that contains
- *       the region to be found
- * @addr: start of the area within @address_space to be searched
+ * If @mr is parent-less,
+ *    .@offset_within_address_space >= @addr
+ *    .@offset_within_address_space + .@size <= @addr + @size
+ *
+ * @mr: a (possibly indirect) parent that contains the region to be found
+ * @addr: start of the area within @as to be searched
  * @size: size of the area to be searched
  */
-MemoryRegionSection memory_region_find(MemoryRegion *address_space,
+MemoryRegionSection memory_region_find(MemoryRegion *mr,
                                        hwaddr addr, uint64_t size);
 
 /**
diff --git a/memory.c b/memory.c
index 75ca281..34bfb13 100644
--- a/memory.c
+++ b/memory.c
@@ -1451,15 +1451,24 @@ static FlatRange *address_space_lookup(AddressSpace *as, AddrRange addr)
                    sizeof(FlatRange), cmp_flatrange_addr);
 }
 
-MemoryRegionSection memory_region_find(MemoryRegion *address_space,
+MemoryRegionSection memory_region_find(MemoryRegion *mr,
                                        hwaddr addr, uint64_t size)
 {
-    AddressSpace *as = memory_region_to_address_space(address_space);
-    AddrRange range = addrrange_make(int128_make64(addr),
-                                     int128_make64(size));
-    FlatRange *fr = address_space_lookup(as, range);
     MemoryRegionSection ret = { .mr = NULL, .size = 0 };
+    MemoryRegion *root;
+    AddressSpace *as;
+    AddrRange range;
+    FlatRange *fr;
+
+    addr += mr->addr;
+    for (root = mr; root->parent; ) {
+        root = root->parent;
+        addr += root->addr;
+    }
 
+    as = memory_region_to_address_space(root);
+    range = addrrange_make(int128_make64(addr), int128_make64(size));
+    fr = address_space_lookup(as, range);
     if (!fr) {
         return ret;
     }
@@ -1470,6 +1479,7 @@ MemoryRegionSection memory_region_find(MemoryRegion *address_space,
     }
 
     ret.mr = fr->mr;
+    ret.address_space = as;
     range = addrrange_intersection(range, fr->addr);
     ret.offset_within_region = fr->offset_in_region;
     ret.offset_within_region += int128_get64(int128_sub(range.start,
-- 
1.7.1

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Blue Swirl]

On Tue, Apr 23, 2013 at 8:29 AM, Igor Mammedov <imammedo@redhat.com> wrote:
> kvm/ioapic is relying on the fact that SysBus device
> maps mmio regions with offset counted from start of system memory.
> But if ioapic's region is moved to another sub-region which doesn't
> start at the beginning of system memory then using offset isn't correct.

But base_address in only initialized once, never changed. Does this
patch matter now?

The correct solution would be to track changes to APICBASE register at
PIIX3 chipset level and adjust mapping and KVM base also from there.

>
> Fix kvm/ioapic by providing and using helper function that returns
> absolute region address in respective address space.
>
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
> Note:
>   next patch "move IOAPIC to ICC bus" converts IOAPICs to ICCDevice
>   and breaks SysBus device assumption used by kvm/ioapic.
> ---
>  hw/i386/kvm/ioapic.c  |    2 +-
>  include/exec/memory.h |   10 ++++++++++
>  memory.c              |   11 +++++++++++
>  3 files changed, 22 insertions(+), 1 deletions(-)
>
> diff --git a/hw/i386/kvm/ioapic.c b/hw/i386/kvm/ioapic.c
> index a3bd519..b80d41a 100644
> --- a/hw/i386/kvm/ioapic.c
> +++ b/hw/i386/kvm/ioapic.c
> @@ -96,7 +96,7 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
>
>      kioapic->id = s->id;
>      kioapic->ioregsel = s->ioregsel;
> -    kioapic->base_address = s->busdev.mmio[0].addr;
> +    kioapic->base_address = memory_region_get_address(&s->io_memory);
>      kioapic->irr = s->irr;
>      for (i = 0; i < IOAPIC_NUM_PINS; i++) {
>          kioapic->redirtbl[i].bits = s->ioredtbl[i];
> diff --git a/include/exec/memory.h b/include/exec/memory.h
> index 9e88320..954f353 100644
> --- a/include/exec/memory.h
> +++ b/include/exec/memory.h
> @@ -706,6 +706,16 @@ void memory_region_set_enabled(MemoryRegion *mr, bool enabled);
>  void memory_region_set_address(MemoryRegion *mr, hwaddr addr);
>
>  /*
> + * memory_region_get_address: get current the address of a region
> + *
> + * Returns the absolute address of a region.
> + * May be used on regions that are currently part of a memory hierarchy.
> + *
> + * @mr: the region being queried
> + */
> +hwaddr memory_region_get_address(MemoryRegion *mr);
> +
> +/*
>   * memory_region_set_alias_offset: dynamically update a memory alias's offset
>   *
>   * Dynamically updates the offset into the target region that an alias points
> diff --git a/memory.c b/memory.c
> index 75ca281..0651050 100644
> --- a/memory.c
> +++ b/memory.c
> @@ -1413,6 +1413,17 @@ void memory_region_set_address(MemoryRegion *mr, hwaddr addr)
>      memory_region_transaction_commit();
>  }
>
> +hwaddr memory_region_get_address(MemoryRegion *mr)
> +{
> +    hwaddr addr = mr->addr;
> +
> +    while (mr->parent) {
> +        mr = mr->parent;
> +        addr += mr->addr;
> +    }
> +    return addr;
> +}
> +
>  void memory_region_set_alias_offset(MemoryRegion *mr, hwaddr offset)
>  {
>      assert(mr->alias);
> --
> 1.7.1
>

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Igor Mammedov]

On Thu, 25 Apr 2013 18:37:19 +0000
Blue Swirl <blauwirbel@gmail.com> wrote:

> On Tue, Apr 23, 2013 at 8:29 AM, Igor Mammedov <imammedo@redhat.com> wrote:
> > kvm/ioapic is relying on the fact that SysBus device
> > maps mmio regions with offset counted from start of system memory.
> > But if ioapic's region is moved to another sub-region which doesn't
> > start at the beginning of system memory then using offset isn't correct.
> 
> But base_address in only initialized once, never changed. Does this
> patch matter now?
this code path is used on only at machine start-up but also on resets and
post migration to initialize IO-APIC. And unfortunately KVM API takes not
only base address but a bunch of other parameters in that ioctl, so splitting
it doesn't look feasible for 1.5. Patch is useful in aspect that it hides
direct access to parent's internals and without dangling SysBusDevice
internals it allows to convert kvm/ioapic to ICCDevice [next patch in series],
leaving code a bit cleaner.

BTW:
there is an improved patch:
http://permalink.gmane.org/gmane.comp.emulators.qemu/208512

that instead of introducing a new function improves current
memory_region_find() API.

> The correct solution would be to track changes to APICBASE register at
> PIIX3 chipset level and adjust mapping and KVM base also from there.
That we would probably need a change in KVM API first to allow set
IO-APIC's base separately.
 
> 
> >
> > Fix kvm/ioapic by providing and using helper function that returns
> > absolute region address in respective address space.
> >
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > ---
> > Note:
> >   next patch "move IOAPIC to ICC bus" converts IOAPICs to ICCDevice
> >   and breaks SysBus device assumption used by kvm/ioapic.
> > ---
> >  hw/i386/kvm/ioapic.c  |    2 +-
> >  include/exec/memory.h |   10 ++++++++++
> >  memory.c              |   11 +++++++++++
> >  3 files changed, 22 insertions(+), 1 deletions(-)
> >
> > diff --git a/hw/i386/kvm/ioapic.c b/hw/i386/kvm/ioapic.c
> > index a3bd519..b80d41a 100644
> > --- a/hw/i386/kvm/ioapic.c
> > +++ b/hw/i386/kvm/ioapic.c
> > @@ -96,7 +96,7 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
> >
> >      kioapic->id = s->id;
> >      kioapic->ioregsel = s->ioregsel;
> > -    kioapic->base_address = s->busdev.mmio[0].addr;
> > +    kioapic->base_address = memory_region_get_address(&s->io_memory);
> >      kioapic->irr = s->irr;
> >      for (i = 0; i < IOAPIC_NUM_PINS; i++) {
> >          kioapic->redirtbl[i].bits = s->ioredtbl[i];
> > diff --git a/include/exec/memory.h b/include/exec/memory.h
> > index 9e88320..954f353 100644
> > --- a/include/exec/memory.h
> > +++ b/include/exec/memory.h
> > @@ -706,6 +706,16 @@ void memory_region_set_enabled(MemoryRegion *mr, bool enabled);
> >  void memory_region_set_address(MemoryRegion *mr, hwaddr addr);
> >
> >  /*
> > + * memory_region_get_address: get current the address of a region
> > + *
> > + * Returns the absolute address of a region.
> > + * May be used on regions that are currently part of a memory hierarchy.
> > + *
> > + * @mr: the region being queried
> > + */
> > +hwaddr memory_region_get_address(MemoryRegion *mr);
> > +
> > +/*
> >   * memory_region_set_alias_offset: dynamically update a memory alias's offset
> >   *
> >   * Dynamically updates the offset into the target region that an alias points
> > diff --git a/memory.c b/memory.c
> > index 75ca281..0651050 100644
> > --- a/memory.c
> > +++ b/memory.c
> > @@ -1413,6 +1413,17 @@ void memory_region_set_address(MemoryRegion *mr, hwaddr addr)
> >      memory_region_transaction_commit();
> >  }
> >
> > +hwaddr memory_region_get_address(MemoryRegion *mr)
> > +{
> > +    hwaddr addr = mr->addr;
> > +
> > +    while (mr->parent) {
> > +        mr = mr->parent;
> > +        addr += mr->addr;
> > +    }
> > +    return addr;
> > +}
> > +
> >  void memory_region_set_alias_offset(MemoryRegion *mr, hwaddr offset)
> >  {
> >      assert(mr->alias);
> > --
> > 1.7.1
> >


-- 
Regards,
  Igor

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Blue Swirl]

On Fri, Apr 26, 2013 at 2:17 PM, Igor Mammedov <imammedo@redhat.com> wrote:
> On Thu, 25 Apr 2013 18:37:19 +0000
> Blue Swirl <blauwirbel@gmail.com> wrote:
>
>> On Tue, Apr 23, 2013 at 8:29 AM, Igor Mammedov <imammedo@redhat.com> wrote:
>> > kvm/ioapic is relying on the fact that SysBus device
>> > maps mmio regions with offset counted from start of system memory.
>> > But if ioapic's region is moved to another sub-region which doesn't
>> > start at the beginning of system memory then using offset isn't correct.
>>
>> But base_address in only initialized once, never changed. Does this
>> patch matter now?
> this code path is used on only at machine start-up but also on resets and
> post migration to initialize IO-APIC. And unfortunately KVM API takes not
> only base address but a bunch of other parameters in that ioctl, so splitting
> it doesn't look feasible for 1.5. Patch is useful in aspect that it hides
> direct access to parent's internals and without dangling SysBusDevice
> internals it allows to convert kvm/ioapic to ICCDevice [next patch in series],
> leaving code a bit cleaner.

But as the address can't be changed (yet), the entire patch could be simply:
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = IO_APIC_DEFAULT_ADDRESS;

Later, when it's possible to change the address via PIIX3 registers,
we can adjust the base and pass that properly to kioapic and on to
KVM.

Resolving the base address every time when kvm_ioapic_put() is called
is also less efficient, assuming of course that the base address
changes less often than the KVM ioctl is used.

>
> BTW:
> there is an improved patch:
> http://permalink.gmane.org/gmane.comp.emulators.qemu/208512
>
> that instead of introducing a new function improves current
> memory_region_find() API.
>
>> The correct solution would be to track changes to APICBASE register at
>> PIIX3 chipset level and adjust mapping and KVM base also from there.
> That we would probably need a change in KVM API first to allow set
> IO-APIC's base separately.
>
>>
>> >
>> > Fix kvm/ioapic by providing and using helper function that returns
>> > absolute region address in respective address space.
>> >
>> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
>> > ---
>> > Note:
>> >   next patch "move IOAPIC to ICC bus" converts IOAPICs to ICCDevice
>> >   and breaks SysBus device assumption used by kvm/ioapic.
>> > ---
>> >  hw/i386/kvm/ioapic.c  |    2 +-
>> >  include/exec/memory.h |   10 ++++++++++
>> >  memory.c              |   11 +++++++++++
>> >  3 files changed, 22 insertions(+), 1 deletions(-)
>> >
>> > diff --git a/hw/i386/kvm/ioapic.c b/hw/i386/kvm/ioapic.c
>> > index a3bd519..b80d41a 100644
>> > --- a/hw/i386/kvm/ioapic.c
>> > +++ b/hw/i386/kvm/ioapic.c
>> > @@ -96,7 +96,7 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
>> >
>> >      kioapic->id = s->id;
>> >      kioapic->ioregsel = s->ioregsel;
>> > -    kioapic->base_address = s->busdev.mmio[0].addr;
>> > +    kioapic->base_address = memory_region_get_address(&s->io_memory);
>> >      kioapic->irr = s->irr;
>> >      for (i = 0; i < IOAPIC_NUM_PINS; i++) {
>> >          kioapic->redirtbl[i].bits = s->ioredtbl[i];
>> > diff --git a/include/exec/memory.h b/include/exec/memory.h
>> > index 9e88320..954f353 100644
>> > --- a/include/exec/memory.h
>> > +++ b/include/exec/memory.h
>> > @@ -706,6 +706,16 @@ void memory_region_set_enabled(MemoryRegion *mr, bool enabled);
>> >  void memory_region_set_address(MemoryRegion *mr, hwaddr addr);
>> >
>> >  /*
>> > + * memory_region_get_address: get current the address of a region
>> > + *
>> > + * Returns the absolute address of a region.
>> > + * May be used on regions that are currently part of a memory hierarchy.
>> > + *
>> > + * @mr: the region being queried
>> > + */
>> > +hwaddr memory_region_get_address(MemoryRegion *mr);
>> > +
>> > +/*
>> >   * memory_region_set_alias_offset: dynamically update a memory alias's offset
>> >   *
>> >   * Dynamically updates the offset into the target region that an alias points
>> > diff --git a/memory.c b/memory.c
>> > index 75ca281..0651050 100644
>> > --- a/memory.c
>> > +++ b/memory.c
>> > @@ -1413,6 +1413,17 @@ void memory_region_set_address(MemoryRegion *mr, hwaddr addr)
>> >      memory_region_transaction_commit();
>> >  }
>> >
>> > +hwaddr memory_region_get_address(MemoryRegion *mr)
>> > +{
>> > +    hwaddr addr = mr->addr;
>> > +
>> > +    while (mr->parent) {
>> > +        mr = mr->parent;
>> > +        addr += mr->addr;
>> > +    }
>> > +    return addr;
>> > +}
>> > +
>> >  void memory_region_set_alias_offset(MemoryRegion *mr, hwaddr offset)
>> >  {
>> >      assert(mr->alias);
>> > --
>> > 1.7.1
>> >
>
>
> --
> Regards,
>   Igor

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Igor Mammedov]

On Fri, 26 Apr 2013 17:35:41 +0000
Blue Swirl <blauwirbel@gmail.com> wrote:

> On Fri, Apr 26, 2013 at 2:17 PM, Igor Mammedov <imammedo@redhat.com> wrote:
> > On Thu, 25 Apr 2013 18:37:19 +0000
> > Blue Swirl <blauwirbel@gmail.com> wrote:
> >
> >> On Tue, Apr 23, 2013 at 8:29 AM, Igor Mammedov <imammedo@redhat.com> wrote:
> >> > kvm/ioapic is relying on the fact that SysBus device
> >> > maps mmio regions with offset counted from start of system memory.
> >> > But if ioapic's region is moved to another sub-region which doesn't
> >> > start at the beginning of system memory then using offset isn't correct.
> >>
> >> But base_address in only initialized once, never changed. Does this
> >> patch matter now?
> > this code path is used on only at machine start-up but also on resets and
> > post migration to initialize IO-APIC. And unfortunately KVM API takes not
> > only base address but a bunch of other parameters in that ioctl, so splitting
> > it doesn't look feasible for 1.5. Patch is useful in aspect that it hides
> > direct access to parent's internals and without dangling SysBusDevice
> > internals it allows to convert kvm/ioapic to ICCDevice [next patch in series],
> > leaving code a bit cleaner.
> 
> But as the address can't be changed (yet), the entire patch could be simply:
> -    kioapic->base_address = s->busdev.mmio[0].addr;
> +    kioapic->base_address = IO_APIC_DEFAULT_ADDRESS;
It's a bit fragile, but that for sure simpler and can work.

Jan, Paolo,
Are you ok with this approach? 

> 
> Later, when it's possible to change the address via PIIX3 registers,
> we can adjust the base and pass that properly to kioapic and on to
> KVM.
> 
> Resolving the base address every time when kvm_ioapic_put() is called
> is also less efficient, assuming of course that the base address
> changes less often than the KVM ioctl is used.
> 
> >
> > BTW:
> > there is an improved patch:
> > http://permalink.gmane.org/gmane.comp.emulators.qemu/208512
> >
> > that instead of introducing a new function improves current
> > memory_region_find() API.
> >
> >> The correct solution would be to track changes to APICBASE register at
> >> PIIX3 chipset level and adjust mapping and KVM base also from there.
> > That we would probably need a change in KVM API first to allow set
> > IO-APIC's base separately.
> >



-- 
Regards,
  Igor

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Paolo Bonzini]

Il 26/04/2013 19:46, Igor Mammedov ha scritto:
>> > But as the address can't be changed (yet), the entire patch could be simply:
>> > -    kioapic->base_address = s->busdev.mmio[0].addr;
>> > +    kioapic->base_address = IO_APIC_DEFAULT_ADDRESS;
> It's a bit fragile, but that for sure simpler and can work.
> 
> Jan, Paolo,
> Are you ok with this approach? 
> 

I think extending memory_region_find is a good idea anyway, and at this
point I don't see a reason to do the above change...

Paolo

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Blue Swirl]

On Fri, Apr 26, 2013 at 10:13 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 26/04/2013 19:46, Igor Mammedov ha scritto:
>>> > But as the address can't be changed (yet), the entire patch could be simply:
>>> > -    kioapic->base_address = s->busdev.mmio[0].addr;
>>> > +    kioapic->base_address = IO_APIC_DEFAULT_ADDRESS;
>> It's a bit fragile, but that for sure simpler and can work.
>>
>> Jan, Paolo,
>> Are you ok with this approach?
>>
>
> I think extending memory_region_find is a good idea anyway, and at this
> point I don't see a reason to do the above change...

The reasoning was in the part that Igor cut off:

"Later, when it's possible to change the address via PIIX3 registers,
we can adjust the base and pass that properly to kioapic and on to
KVM.

Resolving the base address every time when kvm_ioapic_put() is called
is also less efficient, assuming of course that the base address
changes less often than the KVM ioctl is used."

I think the patch is a bit flawed. If the guest maps something else on
top of IOAPIC, like LAPIC (which should be in CPU specific address
spaces, but for now it lives in the global system memory space), the
guest could trigger the abort() by resetting the system.

>
> Paolo

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Paolo Bonzini]

Il 27/04/2013 12:09, Blue Swirl ha scritto:
> On Fri, Apr 26, 2013 at 10:13 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> Il 26/04/2013 19:46, Igor Mammedov ha scritto:
>>>>> But as the address can't be changed (yet), the entire patch could be simply:
>>>>> -    kioapic->base_address = s->busdev.mmio[0].addr;
>>>>> +    kioapic->base_address = IO_APIC_DEFAULT_ADDRESS;
>>> It's a bit fragile, but that for sure simpler and can work.
>>>
>>> Jan, Paolo,
>>> Are you ok with this approach?
>>>
>>
>> I think extending memory_region_find is a good idea anyway, and at this
>> point I don't see a reason to do the above change...
> 
> The reasoning was in the part that Igor cut off:
> 
> "Later, when it's possible to change the address via PIIX3 registers,
> we can adjust the base and pass that properly to kioapic and on to
> KVM.
> 
> Resolving the base address every time when kvm_ioapic_put() is called
> is also less efficient, assuming of course that the base address
> changes less often than the KVM ioctl is used."
> 
> I think the patch is a bit flawed. If the guest maps something else on
> top of IOAPIC, like LAPIC (which should be in CPU specific address
> spaces, but for now it lives in the global system memory space), the
> guest could trigger the abort() by resetting the system.

The questions are, in order of importance:

(1) what privileges would this require in the guest?  Answer: a lot.

(2) is this likely to happen by chance?  Answer: no, not at all.

(3) is there a workaround?  Answer: yes, disable in-kernel irqchip.

Simply setting IO_APIC_DEFAULT_ADDRESS is also flawed in my opinion.
I'm not sure the in-kernel irqchip handles correctly an overlap between
the IOAPIC and LAPIC regions, maybe an abort is predictable after all.

Paolo

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Blue Swirl]

On Sat, Apr 27, 2013 at 12:12 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 27/04/2013 12:09, Blue Swirl ha scritto:
>> On Fri, Apr 26, 2013 at 10:13 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>> Il 26/04/2013 19:46, Igor Mammedov ha scritto:
>>>>>> But as the address can't be changed (yet), the entire patch could be simply:
>>>>>> -    kioapic->base_address = s->busdev.mmio[0].addr;
>>>>>> +    kioapic->base_address = IO_APIC_DEFAULT_ADDRESS;
>>>> It's a bit fragile, but that for sure simpler and can work.
>>>>
>>>> Jan, Paolo,
>>>> Are you ok with this approach?
>>>>
>>>
>>> I think extending memory_region_find is a good idea anyway, and at this
>>> point I don't see a reason to do the above change...
>>
>> The reasoning was in the part that Igor cut off:
>>
>> "Later, when it's possible to change the address via PIIX3 registers,
>> we can adjust the base and pass that properly to kioapic and on to
>> KVM.
>>
>> Resolving the base address every time when kvm_ioapic_put() is called
>> is also less efficient, assuming of course that the base address
>> changes less often than the KVM ioctl is used."
>>
>> I think the patch is a bit flawed. If the guest maps something else on
>> top of IOAPIC, like LAPIC (which should be in CPU specific address
>> spaces, but for now it lives in the global system memory space), the
>> guest could trigger the abort() by resetting the system.
>
> The questions are, in order of importance:
>
> (1) what privileges would this require in the guest?  Answer: a lot.
>
> (2) is this likely to happen by chance?  Answer: no, not at all.
>
> (3) is there a workaround?  Answer: yes, disable in-kernel irqchip.

These questions ask if there is a risk of benevolent guests performing
these activities and I agree that the chances are close to zero.

But the interesting question is to ask if a malevolent guest can bring
down a VM uncontrollably this way and I think it only needs a few
elevated privileges in a guest to do this.

The fix is to avoid abort(), which is a separate issue to whether the
address base should be resolved for each KVM ioctl or not.

>
> Simply setting IO_APIC_DEFAULT_ADDRESS is also flawed in my opinion.
> I'm not sure the in-kernel irqchip handles correctly an overlap between
> the IOAPIC and LAPIC regions, maybe an abort is predictable after all.

At least the guest needs to be stopped. Perhaps we should have a
common function which does this and logs the guest error so we can
start replacing calls to abort() with it.

>
> Paolo

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Paolo Bonzini]

Il 27/04/2013 22:57, Blue Swirl ha scritto:
>> The questions are, in order of importance:
>>
>> (1) what privileges would this require in the guest?  Answer: a lot.
>>
>> (2) is this likely to happen by chance?  Answer: no, not at all.
>>
>> (3) is there a workaround?  Answer: yes, disable in-kernel irqchip.
> 
> These questions ask if there is a risk of benevolent guests performing
> these activities and I agree that the chances are close to zero.
> 
> But the interesting question is to ask if a malevolent guest can bring
> down a VM uncontrollably this way and I think it only needs a few
> elevated privileges in a guest to do this.

If you have them, isn't it simpler to just turn off the VM (using APM or
ACPI)?  Also, killing your guest is not a very interesting thing to do
once you've gotten elevated privileges. ;)

>> Simply setting IO_APIC_DEFAULT_ADDRESS is also flawed in my opinion.
>> I'm not sure the in-kernel irqchip handles correctly an overlap between
>> the IOAPIC and LAPIC regions, maybe an abort is predictable after all.
> 
> At least the guest needs to be stopped. Perhaps we should have a
> common function which does this and logs the guest error so we can
> start replacing calls to abort() with it.

Yes, that's a good idea.  We can reuse the internal error runstate for that.

Paolo

Re: [Qemu-devel] [PATCH 17/21] introduce memory_region_get_address() and use it in kvm/ioapic

[Igor Mammedov]

On Sat, 27 Apr 2013 20:57:26 +0000
Blue Swirl <blauwirbel@gmail.com> wrote:

> On Sat, Apr 27, 2013 at 12:12 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> > Il 27/04/2013 12:09, Blue Swirl ha scritto:
> >> On Fri, Apr 26, 2013 at 10:13 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> >>> Il 26/04/2013 19:46, Igor Mammedov ha scritto:
> >>>>>> But as the address can't be changed (yet), the entire patch could be simply:
> >>>>>> -    kioapic->base_address = s->busdev.mmio[0].addr;
> >>>>>> +    kioapic->base_address = IO_APIC_DEFAULT_ADDRESS;
> >>>> It's a bit fragile, but that for sure simpler and can work.
> >>>>
> >>>> Jan, Paolo,
> >>>> Are you ok with this approach?
> >>>>
> >>>
> >>> I think extending memory_region_find is a good idea anyway, and at this
> >>> point I don't see a reason to do the above change...
> >>
> >> The reasoning was in the part that Igor cut off:
> >>
> >> "Later, when it's possible to change the address via PIIX3 registers,
> >> we can adjust the base and pass that properly to kioapic and on to
> >> KVM.
> >>
> >> Resolving the base address every time when kvm_ioapic_put() is called
> >> is also less efficient, assuming of course that the base address
> >> changes less often than the KVM ioctl is used."
> >>
> >> I think the patch is a bit flawed. If the guest maps something else on
> >> top of IOAPIC, like LAPIC (which should be in CPU specific address
> >> spaces, but for now it lives in the global system memory space), the
> >> guest could trigger the abort() by resetting the system.
> >
> > The questions are, in order of importance:
> >
> > (1) what privileges would this require in the guest?  Answer: a lot.
> >
> > (2) is this likely to happen by chance?  Answer: no, not at all.
> >
> > (3) is there a workaround?  Answer: yes, disable in-kernel irqchip.
> 
> These questions ask if there is a risk of benevolent guests performing
> these activities and I agree that the chances are close to zero.
> 
> But the interesting question is to ask if a malevolent guest can bring
> down a VM uncontrollably this way and I think it only needs a few
> elevated privileges in a guest to do this.
> 
> The fix is to avoid abort(), which is a separate issue to whether the
> address base should be resolved for each KVM ioctl or not.
> 
> >
> > Simply setting IO_APIC_DEFAULT_ADDRESS is also flawed in my opinion.
> > I'm not sure the in-kernel irqchip handles correctly an overlap between
> > the IOAPIC and LAPIC regions, maybe an abort is predictable after all.
> 
> At least the guest needs to be stopped. Perhaps we should have a
> common function which does this and logs the guest error so we can
> start replacing calls to abort() with it.
> 
> >
> > Paolo
> 

It looks like discussion got deviated from what patch does to another issue.

this patch doesn't address/change the way how/when base_address should be
set/updated but it has it's benefits as well:
 - removes/cleanups access to private field of parent, which allows to convert
   it to non SysBusDevice
 - extended memory_region_find() opens venue for cleaning-up/re-factoring
   devices that use framebuffer which are forced currently to access
   system_address_space directly.
 

-- 
Regards,
  Igor

[Qemu-devel] [PATCH 18/21] target-i386: move IOAPIC to ICC bus

[Igor Mammedov]

 * inherit IOAPICs from ICCDevice and attach them to ICC bus
 * map IOAPIC's mmio at board level via indirect icc-bridge
   mmio region that provides address space to IOAPIC via
   icc-bus

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v3:
 * split out introduction of memory_region_get_address() helper
 * use qdev_get_parent_bus() to get parent bus
v2:
 * do not create IOAPIC in icc-bridge and do not make it QOM child
   of icc-bridge, just attach it to icc-bus.
---
 hw/cpu/icc_bus.c                  |    6 ++++++
 hw/i386/pc.c                      |   10 ++++++----
 hw/intc/ioapic_common.c           |   14 ++++++++++----
 include/hw/cpu/icc_bus.h          |    1 +
 include/hw/i386/ioapic_internal.h |    6 +++---
 5 files changed, 26 insertions(+), 11 deletions(-)

diff --git a/hw/cpu/icc_bus.c b/hw/cpu/icc_bus.c
index 9715e07..b9e4f23 100644
--- a/hw/cpu/icc_bus.c
+++ b/hw/cpu/icc_bus.c
@@ -76,6 +76,7 @@ typedef struct ICCBridgeState {
     SysBusDevice parent_obj;
     ICCBus icc_bus;
     MemoryRegion apic_container;
+    MemoryRegion ioapic_container;
 } ICCBridgeState;
 
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
@@ -94,6 +95,11 @@ static void icc_bridge_init(Object *obj)
                        APIC_SPACE_SIZE);
     sysbus_init_mmio(sb, &s->apic_container);
     s->icc_bus.apic_address_space = &s->apic_container;
+
+    /* must be second registered region, board maps it by 1 index */
+    memory_region_init(&s->ioapic_container, "icc-ioapic-container", 0x1000);
+    sysbus_init_mmio(sb, &s->ioapic_container);
+    s->icc_bus.ioapic_address_space = &s->ioapic_container;
 }
 
 static const TypeInfo icc_bridge_info = {
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index a09b7ee..5e50127 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1220,20 +1220,22 @@ void ioapic_init_gsi(GSIState *gsi_state, const char *parent_name)
     DeviceState *dev;
     SysBusDevice *d;
     unsigned int i;
+    BusState *b = BUS(object_resolve_path_type("icc-bus", TYPE_ICC_BUS, NULL));
 
     if (kvm_irqchip_in_kernel()) {
-        dev = qdev_create(NULL, "kvm-ioapic");
+        dev = qdev_create(b, "kvm-ioapic");
     } else {
-        dev = qdev_create(NULL, "ioapic");
+        dev = qdev_create(b, "ioapic");
     }
     if (parent_name) {
         object_property_add_child(object_resolve_path(parent_name, NULL),
                                   "ioapic", OBJECT(dev), NULL);
     }
     qdev_init_nofail(dev);
-    d = SYS_BUS_DEVICE(dev);
-    sysbus_mmio_map(d, 0, IO_APIC_DEFAULT_ADDRESS);
 
+    d = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge", TYPE_ICC_BRIDGE,
+                                                NULL));
+    sysbus_mmio_map(d, 1, IO_APIC_DEFAULT_ADDRESS);
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         gsi_state->ioapic_irq[i] = qdev_get_gpio_in(dev, i);
     }
diff --git a/hw/intc/ioapic_common.c b/hw/intc/ioapic_common.c
index 5c5bb3c..a70d8a2 100644
--- a/hw/intc/ioapic_common.c
+++ b/hw/intc/ioapic_common.c
@@ -57,11 +57,12 @@ static int ioapic_dispatch_post_load(void *opaque, int version_id)
     return 0;
 }
 
-static int ioapic_init_common(SysBusDevice *dev)
+static int ioapic_init_common(ICCDevice *dev)
 {
     IOAPICCommonState *s = IOAPIC_COMMON(dev);
     IOAPICCommonClass *info;
     static int ioapic_no;
+    static bool mmio_registered;
 
     if (ioapic_no >= MAX_IOAPICS) {
         return -1;
@@ -70,7 +71,12 @@ static int ioapic_init_common(SysBusDevice *dev)
     info = IOAPIC_COMMON_GET_CLASS(s);
     info->init(s, ioapic_no);
 
-    sysbus_init_mmio(&s->busdev, &s->io_memory);
+    if (!mmio_registered) {
+        ICCBus *b = ICC_BUS(qdev_get_parent_bus(DEVICE(dev)));
+        memory_region_add_subregion(b->ioapic_address_space, 0, &s->io_memory);
+        mmio_registered = true;
+    }
+
     ioapic_no++;
 
     return 0;
@@ -95,7 +101,7 @@ static const VMStateDescription vmstate_ioapic_common = {
 
 static void ioapic_common_class_init(ObjectClass *klass, void *data)
 {
-    SysBusDeviceClass *sc = SYS_BUS_DEVICE_CLASS(klass);
+    ICCDeviceClass *sc = ICC_DEVICE_CLASS(klass);
     DeviceClass *dc = DEVICE_CLASS(klass);
 
     sc->init = ioapic_init_common;
@@ -105,7 +111,7 @@ static void ioapic_common_class_init(ObjectClass *klass, void *data)
 
 static const TypeInfo ioapic_common_type = {
     .name = TYPE_IOAPIC_COMMON,
-    .parent = TYPE_SYS_BUS_DEVICE,
+    .parent = TYPE_ICC_DEVICE,
     .instance_size = sizeof(IOAPICCommonState),
     .class_size = sizeof(IOAPICCommonClass),
     .class_init = ioapic_common_class_init,
diff --git a/include/hw/cpu/icc_bus.h b/include/hw/cpu/icc_bus.h
index 53ef223..947125d 100644
--- a/include/hw/cpu/icc_bus.h
+++ b/include/hw/cpu/icc_bus.h
@@ -35,6 +35,7 @@ typedef struct ICCBus {
 
     /*< public >*/
     MemoryRegion *apic_address_space;
+    MemoryRegion *ioapic_address_space;
 } ICCBus;
 
 #define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
diff --git a/include/hw/i386/ioapic_internal.h b/include/hw/i386/ioapic_internal.h
index 25576c8..8d5fe3d 100644
--- a/include/hw/i386/ioapic_internal.h
+++ b/include/hw/i386/ioapic_internal.h
@@ -24,7 +24,7 @@
 
 #include "hw/hw.h"
 #include "exec/memory.h"
-#include "hw/sysbus.h"
+#include "hw/cpu/icc_bus.h"
 
 #define MAX_IOAPICS                     1
 
@@ -82,14 +82,14 @@ typedef struct IOAPICCommonState IOAPICCommonState;
      OBJECT_GET_CLASS(IOAPICCommonClass, (obj), TYPE_IOAPIC_COMMON)
 
 typedef struct IOAPICCommonClass {
-    SysBusDeviceClass parent_class;
+    ICCDeviceClass parent_class;
     void (*init)(IOAPICCommonState *s, int instance_no);
     void (*pre_save)(IOAPICCommonState *s);
     void (*post_load)(IOAPICCommonState *s);
 } IOAPICCommonClass;
 
 struct IOAPICCommonState {
-    SysBusDevice busdev;
+    ICCDevice busdev;
     MemoryRegion io_memory;
     uint8_t id;
     uint8_t ioregsel;
-- 
1.7.1

[Qemu-devel] [PATCH 19/21] add hot_add_cpu hook to QEMUMachine and export machine_args

[Igor Mammedov]

hot_add_cpu hook should be overriden by target that implements
cpu hot-add via cpu-add QMP command.

Make machine_args available to machine init code, it allows
to centralize cpu_model starage instead of adding target
specific globals to keep it.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 include/hw/boards.h |    3 +++
 vl.c                |    6 +++++-
 2 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/include/hw/boards.h b/include/hw/boards.h
index 425bdc7..de8f92a 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -18,6 +18,8 @@ typedef struct QEMUMachineInitArgs {
     const char *cpu_model;
 } QEMUMachineInitArgs;
 
+extern QEMUMachineInitArgs *machine_args;
+
 typedef void QEMUMachineInitFunc(QEMUMachineInitArgs *args);
 
 typedef void QEMUMachineResetFunc(void);
@@ -43,6 +45,7 @@ typedef struct QEMUMachine {
     GlobalProperty *compat_props;
     struct QEMUMachine *next;
     const char *hw_version;
+    void (*hot_add_cpu)(const int64_t id, Error **errp);
 } QEMUMachine;
 
 int qemu_register_machine(QEMUMachine *m);
diff --git a/vl.c b/vl.c
index 5612c33..6a612e6 100644
--- a/vl.c
+++ b/vl.c
@@ -179,6 +179,8 @@ int main(int argc, char **argv)
 #define MAX_VIRTIO_CONSOLES 1
 #define MAX_SCLP_CONSOLES 1
 
+QEMUMachineInitArgs *machine_args;
+
 static const char *data_dir[16];
 static int data_dir_idx;
 const char *bios_name = NULL;
@@ -4295,13 +4297,15 @@ int main(int argc, char **argv, char **envp)
                                  .kernel_cmdline = kernel_cmdline,
                                  .initrd_filename = initrd_filename,
                                  .cpu_model = cpu_model };
+    machine_args = &args;
+    current_machine = machine;
+
     machine->init(&args);
 
     cpu_synchronize_all_post_init();
 
     set_numa_modes();
 
-    current_machine = machine;
 
     /* init USB devices */
     if (usb_enabled(false)) {
-- 
1.7.1

Re: [Qemu-devel] [PATCH 19/21] add hot_add_cpu hook to QEMUMachine and export machine_args

[Andreas Färber]

Am 23.04.2013 10:29, schrieb Igor Mammedov:
> hot_add_cpu hook should be overriden by target that implements
> cpu hot-add via cpu-add QMP command.
> 
> Make machine_args available to machine init code, it allows
> to centralize cpu_model starage instead of adding target
> specific globals to keep it.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
>  include/hw/boards.h |    3 +++
>  vl.c                |    6 +++++-
>  2 files changed, 8 insertions(+), 1 deletions(-)
> 
> diff --git a/include/hw/boards.h b/include/hw/boards.h
> index 425bdc7..de8f92a 100644
> --- a/include/hw/boards.h
> +++ b/include/hw/boards.h
> @@ -18,6 +18,8 @@ typedef struct QEMUMachineInitArgs {
>      const char *cpu_model;
>  } QEMUMachineInitArgs;
>  
> +extern QEMUMachineInitArgs *machine_args;
> +
>  typedef void QEMUMachineInitFunc(QEMUMachineInitArgs *args);
>  
>  typedef void QEMUMachineResetFunc(void);
> @@ -43,6 +45,7 @@ typedef struct QEMUMachine {
>      GlobalProperty *compat_props;
>      struct QEMUMachine *next;
>      const char *hw_version;
> +    void (*hot_add_cpu)(const int64_t id, Error **errp);
>  } QEMUMachine;
>  
>  int qemu_register_machine(QEMUMachine *m);
> diff --git a/vl.c b/vl.c
> index 5612c33..6a612e6 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -179,6 +179,8 @@ int main(int argc, char **argv)
>  #define MAX_VIRTIO_CONSOLES 1
>  #define MAX_SCLP_CONSOLES 1
>  
> +QEMUMachineInitArgs *machine_args;
> +
>  static const char *data_dir[16];
>  static int data_dir_idx;
>  const char *bios_name = NULL;
> @@ -4295,13 +4297,15 @@ int main(int argc, char **argv, char **envp)
>                                   .kernel_cmdline = kernel_cmdline,
>                                   .initrd_filename = initrd_filename,
>                                   .cpu_model = cpu_model };
> +    machine_args = &args;

args is a stack variable, so making it global does not strike me as a
good idea... (I did have a patch to split main() in two at one time)

> +    current_machine = machine;
> +
>      machine->init(&args);
>  
>      cpu_synchronize_all_post_init();
>  
>      set_numa_modes();
>  
> -    current_machine = machine;

Did you look up why it was placed here? Eduardo?
I'm not strongly opinionated but I wonder whether we may want to leave
it after machine->init?

Andreas

>  
>      /* init USB devices */
>      if (usb_enabled(false)) {
> 


-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 19/21] add hot_add_cpu hook to QEMUMachine and export machine_args

[Igor Mammedov]

On Wed, 24 Apr 2013 19:25:17 +0200
Andreas Färber <afaerber@suse.de> wrote:

> Am 23.04.2013 10:29, schrieb Igor Mammedov:
> > hot_add_cpu hook should be overriden by target that implements
> > cpu hot-add via cpu-add QMP command.
> > 
> > Make machine_args available to machine init code, it allows
> > to centralize cpu_model starage instead of adding target
> > specific globals to keep it.
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > ---
> >  include/hw/boards.h |    3 +++
> >  vl.c                |    6 +++++-
> >  2 files changed, 8 insertions(+), 1 deletions(-)
> > 
> > diff --git a/include/hw/boards.h b/include/hw/boards.h
> > index 425bdc7..de8f92a 100644
> > --- a/include/hw/boards.h
> > +++ b/include/hw/boards.h
> > @@ -18,6 +18,8 @@ typedef struct QEMUMachineInitArgs {
> >      const char *cpu_model;
> >  } QEMUMachineInitArgs;
> >  
> > +extern QEMUMachineInitArgs *machine_args;
> > +
> >  typedef void QEMUMachineInitFunc(QEMUMachineInitArgs *args);
> >  
> >  typedef void QEMUMachineResetFunc(void);
> > @@ -43,6 +45,7 @@ typedef struct QEMUMachine {
> >      GlobalProperty *compat_props;
> >      struct QEMUMachine *next;
> >      const char *hw_version;
> > +    void (*hot_add_cpu)(const int64_t id, Error **errp);
> >  } QEMUMachine;
> >  
> >  int qemu_register_machine(QEMUMachine *m);
> > diff --git a/vl.c b/vl.c
> > index 5612c33..6a612e6 100644
> > --- a/vl.c
> > +++ b/vl.c
> > @@ -179,6 +179,8 @@ int main(int argc, char **argv)
> >  #define MAX_VIRTIO_CONSOLES 1
> >  #define MAX_SCLP_CONSOLES 1
> >  
> > +QEMUMachineInitArgs *machine_args;
> > +
> >  static const char *data_dir[16];
> >  static int data_dir_idx;
> >  const char *bios_name = NULL;
> > @@ -4295,13 +4297,15 @@ int main(int argc, char **argv, char **envp)
> >                                   .kernel_cmdline = kernel_cmdline,
> >                                   .initrd_filename = initrd_filename,
> >                                   .cpu_model = cpu_model };
> > +    machine_args = &args;
> 
> args is a stack variable, so making it global does not strike me as a
> good idea... (I did have a patch to split main() in two at one time)

True, but it lives till main() exits, and by that time nothing accesses it.
I can make it global though it will produce more code churn.

> 
> > +    current_machine = machine;
> > +
> >      machine->init(&args);
> >  
> >      cpu_synchronize_all_post_init();
> >  
> >      set_numa_modes();
> >  
> > -    current_machine = machine;
> 
> Did you look up why it was placed here? Eduardo?
> I'm not strongly opinionated but I wonder whether we may want to leave
> it after machine->init?
it was added by 6f338c34 for the sake of device hotplug. Moving it earlier
allows to use it during machine->init() as well, it's improves variable
utilization. next patch sets hot_add_cpu hook from machine->init() for x86
target.

> 
> Andreas
> 
> >  
> >      /* init USB devices */
> >      if (usb_enabled(false)) {
> > 
> 
> 
> -- 
> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
> 


-- 
Regards,
  Igor

Re: [Qemu-devel] [PATCH 19/21] add hot_add_cpu hook to QEMUMachine and export machine_args

[Eduardo Habkost]

On Wed, Apr 24, 2013 at 07:25:17PM +0200, Andreas Färber wrote:
> Am 23.04.2013 10:29, schrieb Igor Mammedov:
> > hot_add_cpu hook should be overriden by target that implements
> > cpu hot-add via cpu-add QMP command.
> > 
> > Make machine_args available to machine init code, it allows
> > to centralize cpu_model starage instead of adding target
> > specific globals to keep it.
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > ---
> >  include/hw/boards.h |    3 +++
> >  vl.c                |    6 +++++-
> >  2 files changed, 8 insertions(+), 1 deletions(-)
> > 
> > diff --git a/include/hw/boards.h b/include/hw/boards.h
> > index 425bdc7..de8f92a 100644
> > --- a/include/hw/boards.h
> > +++ b/include/hw/boards.h
> > @@ -18,6 +18,8 @@ typedef struct QEMUMachineInitArgs {
> >      const char *cpu_model;
> >  } QEMUMachineInitArgs;
> >  
> > +extern QEMUMachineInitArgs *machine_args;
> > +
> >  typedef void QEMUMachineInitFunc(QEMUMachineInitArgs *args);
> >  
> >  typedef void QEMUMachineResetFunc(void);
> > @@ -43,6 +45,7 @@ typedef struct QEMUMachine {
> >      GlobalProperty *compat_props;
> >      struct QEMUMachine *next;
> >      const char *hw_version;
> > +    void (*hot_add_cpu)(const int64_t id, Error **errp);
> >  } QEMUMachine;
> >  
> >  int qemu_register_machine(QEMUMachine *m);
> > diff --git a/vl.c b/vl.c
> > index 5612c33..6a612e6 100644
> > --- a/vl.c
> > +++ b/vl.c
> > @@ -179,6 +179,8 @@ int main(int argc, char **argv)
> >  #define MAX_VIRTIO_CONSOLES 1
> >  #define MAX_SCLP_CONSOLES 1
> >  
> > +QEMUMachineInitArgs *machine_args;
> > +
> >  static const char *data_dir[16];
> >  static int data_dir_idx;
> >  const char *bios_name = NULL;
> > @@ -4295,13 +4297,15 @@ int main(int argc, char **argv, char **envp)
> >                                   .kernel_cmdline = kernel_cmdline,
> >                                   .initrd_filename = initrd_filename,
> >                                   .cpu_model = cpu_model };
> > +    machine_args = &args;
> 
> args is a stack variable, so making it global does not strike me as a
> good idea... (I did have a patch to split main() in two at one time)
> 
> > +    current_machine = machine;
> > +
> >      machine->init(&args);
> >  
> >      cpu_synchronize_all_post_init();
> >  
> >      set_numa_modes();
> >  
> > -    current_machine = machine;
> 
> Did you look up why it was placed here? Eduardo?
> I'm not strongly opinionated but I wonder whether we may want to leave
> it after machine->init?

It was always there since the first time I looked at that code. But I
believe it makes sense to set it earlier so initialization code can use
the machine object.

On the other hand, I would really like to reduce usage of global
variables during initialization, and instead pass a QEMUMachineInitArgs
argument to init functions that need it. But I understand that sometimes
we need to be pragmatic and a global variable is a simpler approach (if
considered just a temporary solution).

-- 
Eduardo

[Qemu-devel] [PATCH 20/21] target-i386: implement machine->hot_add_cpu hook

[Igor Mammedov]

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 hw/i386/pc.c |   22 ++++++++++++++++++++++
 1 files changed, 22 insertions(+), 0 deletions(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 5e50127..b649ed5 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -54,6 +54,7 @@
 #include "qemu/config-file.h"
 #include "hw/acpi/acpi.h"
 #include "hw/cpu/icc_bus.h"
+#include "hw/boards.h"
 
 /* debug PC/ISA interrupts */
 //#define DEBUG_IRQ
@@ -914,6 +915,25 @@ static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
     return cpu;
 }
 
+static void do_cpu_hot_add(const int64_t id, Error **errp)
+{
+    int64_t apic_id = x86_cpu_apic_id_from_index(id);
+
+    if (cpu_exists(apic_id)) {
+        error_setg(errp, "Unable to add CPU: %" PRIi64
+                   ", it already exists", id);
+        return;
+    }
+
+    if (id >= max_cpus) {
+        error_setg(errp, "Unable to add CPU: %" PRIi64
+                   ", max allowed: %d", id, max_cpus - 1);
+        return;
+    }
+
+    pc_new_cpu(machine_args->cpu_model, apic_id, errp);
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
@@ -928,7 +948,9 @@ void pc_cpus_init(const char *cpu_model)
 #else
         cpu_model = "qemu32";
 #endif
+        machine_args->cpu_model = cpu_model;
     }
+    current_machine->hot_add_cpu = do_cpu_hot_add;
 
     icc_bridge = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge",
                                                  TYPE_ICC_BRIDGE, NULL));
-- 
1.7.1

Re: [Qemu-devel] [PATCH 20/21] target-i386: implement machine->hot_add_cpu hook

[Andreas Färber]

Am 23.04.2013 10:29, schrieb Igor Mammedov:
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
>  hw/i386/pc.c |   22 ++++++++++++++++++++++
>  1 files changed, 22 insertions(+), 0 deletions(-)
> 
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 5e50127..b649ed5 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -54,6 +54,7 @@
>  #include "qemu/config-file.h"
>  #include "hw/acpi/acpi.h"
>  #include "hw/cpu/icc_bus.h"
> +#include "hw/boards.h"
>  
>  /* debug PC/ISA interrupts */
>  //#define DEBUG_IRQ
> @@ -914,6 +915,25 @@ static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
>      return cpu;
>  }
>  
> +static void do_cpu_hot_add(const int64_t id, Error **errp)
> +{
> +    int64_t apic_id = x86_cpu_apic_id_from_index(id);
> +
> +    if (cpu_exists(apic_id)) {
> +        error_setg(errp, "Unable to add CPU: %" PRIi64
> +                   ", it already exists", id);
> +        return;
> +    }
> +
> +    if (id >= max_cpus) {
> +        error_setg(errp, "Unable to add CPU: %" PRIi64
> +                   ", max allowed: %d", id, max_cpus - 1);

Why -1?

> +        return;
> +    }
> +
> +    pc_new_cpu(machine_args->cpu_model, apic_id, errp);
> +}
> +
>  void pc_cpus_init(const char *cpu_model)
>  {
>      int i;
> @@ -928,7 +948,9 @@ void pc_cpus_init(const char *cpu_model)
>  #else
>          cpu_model = "qemu32";
>  #endif
> +        machine_args->cpu_model = cpu_model;

This could be avoided by changing argument to const char **. :)
Caller will have access to QEMUMachineArgs*.

>      }
> +    current_machine->hot_add_cpu = do_cpu_hot_add;

That kind of answers my previous question, but why can't we have this
statically in the QEMUMachine(s)?

Andreas

>  
>      icc_bridge = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge",
>                                                   TYPE_ICC_BRIDGE, NULL));
> 


-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 20/21] target-i386: implement machine->hot_add_cpu hook

[Eduardo Habkost]

On Wed, Apr 24, 2013 at 07:31:44PM +0200, Andreas Färber wrote:
> > +static void do_cpu_hot_add(const int64_t id, Error **errp)
> > +{
> > +    int64_t apic_id = x86_cpu_apic_id_from_index(id);
> > +
> > +    if (cpu_exists(apic_id)) {
> > +        error_setg(errp, "Unable to add CPU: %" PRIi64
> > +                   ", it already exists", id);
> > +        return;
> > +    }
> > +
> > +    if (id >= max_cpus) {
> > +        error_setg(errp, "Unable to add CPU: %" PRIi64
> > +                   ", max allowed: %d", id, max_cpus - 1);
> 
> Why -1?

max_cpus-1 makes sense to me, as the message is about "maximum ID
allowed", not "maximum number of CPUs".

> 
> > +        return;
> > +    }
> > +
> > +    pc_new_cpu(machine_args->cpu_model, apic_id, errp);
> > +}
> > +
> >  void pc_cpus_init(const char *cpu_model)
> >  {
> >      int i;
> > @@ -928,7 +948,9 @@ void pc_cpus_init(const char *cpu_model)
> >  #else
> >          cpu_model = "qemu32";
> >  #endif
> > +        machine_args->cpu_model = cpu_model;
> 
> This could be avoided by changing argument to const char **. :)
> Caller will have access to QEMUMachineArgs*.

I would prefer to simply pass a pointer to QEMUMachineArgs to
pc_cpus_init(), instead of const char **.

-- 
Eduardo

[Qemu-devel] [PATCH 21/21] QMP: add cpu-add command

[Igor Mammedov]

Adds "cpu-add id=xxx" QMP command.

cpu-add's "id" argument is a CPU number in a range [0..max-cpus)

Example QMP command:
 -> { "execute": "cpu-add", "arguments": { "id": 2 } }
 <- { "return": {} }

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v6:
  * added valid values description to qapi schema
  * split out cpu_hot_add hooks introduction into separate patch
  * split out implementation of cpu_hot_add for target-i386
v5:
  * accept id=[0..max_cpus) range in cpu-add command
v4:
  * merge "qmp: add cpu-add qmp command" & "target-i386: implement CPU hot-add" patches
  * move notifier call to CPUCLass.realize()
  * add hook cpu_hot_add to QEMUMachine
  * make QEMUMachineInitArgs global and keep default cpu_model there

v3:
  * it appears that 'online/offline' in cpu-set are confusing people
    with what command actually does and users might have to distinguish
    if 'offline' is not implemented by parsing error message. To simplify
    things replace cpu-set with cpu-add command to show more clear what
    command does and just add cpu-del when CPU remove is implemented.

v2:
  * s/cpu_set/cpu-set/
  * qmp doc style fix
  * use bool type instead of opencodding online/offline string
     suggested-by: Eric Blake <eblake@redhat.com>
---
 qapi-schema.json |   11 +++++++++++
 qmp-commands.hx  |   23 +++++++++++++++++++++++
 qmp.c            |   10 ++++++++++
 3 files changed, 44 insertions(+), 0 deletions(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index 751d3c2..11a5c82 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1387,6 +1387,17 @@
 { 'command': 'cpu', 'data': {'index': 'int'} }
 
 ##
+# @cpu-add
+#
+# Adds CPU with specified ID
+#
+# @id: ID of CPU to be created, valid values [0..max_cpus)
+#
+# Returns: Nothing on success
+##
+{ 'command': 'cpu-add', 'data': {'id': 'int'} }
+
+##
 # @memsave:
 #
 # Save a portion of guest memory to a file.
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 4d65422..1e5d299 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -385,6 +385,29 @@ Note: CPUs' indexes are obtained with the 'query-cpus' command.
 EQMP
 
     {
+        .name       = "cpu-add",
+        .args_type  = "id:i",
+        .mhandler.cmd_new = qmp_marshal_input_cpu_add,
+    },
+
+SQMP
+cpu-add
+-------
+
+Adds virtual cpu
+
+Arguments:
+
+- "id": cpu id (json-int)
+
+Example:
+
+-> { "execute": "cpu-add", "arguments": { "id": 2 } }
+<- { "return": {} }
+
+EQMP
+
+    {
         .name       = "memsave",
         .args_type  = "val:l,size:i,filename:s,cpu:i?",
         .mhandler.cmd_new = qmp_marshal_input_memsave,
diff --git a/qmp.c b/qmp.c
index ed6c7ef..dd34be6 100644
--- a/qmp.c
+++ b/qmp.c
@@ -24,6 +24,7 @@
 #include "hw/qdev.h"
 #include "sysemu/blockdev.h"
 #include "qom/qom-qobject.h"
+#include "hw/boards.h"
 
 NameInfo *qmp_query_name(Error **errp)
 {
@@ -108,6 +109,15 @@ void qmp_cpu(int64_t index, Error **errp)
     /* Just do nothing */
 }
 
+void qmp_cpu_add(int64_t id, Error **errp)
+{
+    if (current_machine->hot_add_cpu) {
+        current_machine->hot_add_cpu(id, errp);
+    } else {
+        error_setg(errp, "Not supported");
+    }
+}
+
 #ifndef CONFIG_VNC
 /* If VNC support is enabled, the "true" query-vnc command is
    defined in the VNC subsystem */
-- 
1.7.1

Re: [Qemu-devel] [PATCH 21/21] QMP: add cpu-add command

[Luiz Capitulino]

On Tue, 23 Apr 2013 10:29:55 +0200
Igor Mammedov <imammedo@redhat.com> wrote:

> Adds "cpu-add id=xxx" QMP command.
> 
> cpu-add's "id" argument is a CPU number in a range [0..max-cpus)
> 
> Example QMP command:
>  -> { "execute": "cpu-add", "arguments": { "id": 2 } }
>  <- { "return": {} }
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>

QMP parts (this and the previous patch) is trivial:

Acked-by: Luiz Capitulino <lcapitulino@redhat.com>

Just wondering, no HMP command?

> ---
> v6:
>   * added valid values description to qapi schema
>   * split out cpu_hot_add hooks introduction into separate patch
>   * split out implementation of cpu_hot_add for target-i386
> v5:
>   * accept id=[0..max_cpus) range in cpu-add command
> v4:
>   * merge "qmp: add cpu-add qmp command" & "target-i386: implement CPU hot-add" patches
>   * move notifier call to CPUCLass.realize()
>   * add hook cpu_hot_add to QEMUMachine
>   * make QEMUMachineInitArgs global and keep default cpu_model there
> 
> v3:
>   * it appears that 'online/offline' in cpu-set are confusing people
>     with what command actually does and users might have to distinguish
>     if 'offline' is not implemented by parsing error message. To simplify
>     things replace cpu-set with cpu-add command to show more clear what
>     command does and just add cpu-del when CPU remove is implemented.
> 
> v2:
>   * s/cpu_set/cpu-set/
>   * qmp doc style fix
>   * use bool type instead of opencodding online/offline string
>      suggested-by: Eric Blake <eblake@redhat.com>
> ---
>  qapi-schema.json |   11 +++++++++++
>  qmp-commands.hx  |   23 +++++++++++++++++++++++
>  qmp.c            |   10 ++++++++++
>  3 files changed, 44 insertions(+), 0 deletions(-)
> 
> diff --git a/qapi-schema.json b/qapi-schema.json
> index 751d3c2..11a5c82 100644
> --- a/qapi-schema.json
> +++ b/qapi-schema.json
> @@ -1387,6 +1387,17 @@
>  { 'command': 'cpu', 'data': {'index': 'int'} }
>  
>  ##
> +# @cpu-add
> +#
> +# Adds CPU with specified ID
> +#
> +# @id: ID of CPU to be created, valid values [0..max_cpus)
> +#
> +# Returns: Nothing on success
> +##
> +{ 'command': 'cpu-add', 'data': {'id': 'int'} }
> +
> +##
>  # @memsave:
>  #
>  # Save a portion of guest memory to a file.
> diff --git a/qmp-commands.hx b/qmp-commands.hx
> index 4d65422..1e5d299 100644
> --- a/qmp-commands.hx
> +++ b/qmp-commands.hx
> @@ -385,6 +385,29 @@ Note: CPUs' indexes are obtained with the 'query-cpus' command.
>  EQMP
>  
>      {
> +        .name       = "cpu-add",
> +        .args_type  = "id:i",
> +        .mhandler.cmd_new = qmp_marshal_input_cpu_add,
> +    },
> +
> +SQMP
> +cpu-add
> +-------
> +
> +Adds virtual cpu
> +
> +Arguments:
> +
> +- "id": cpu id (json-int)
> +
> +Example:
> +
> +-> { "execute": "cpu-add", "arguments": { "id": 2 } }
> +<- { "return": {} }
> +
> +EQMP
> +
> +    {
>          .name       = "memsave",
>          .args_type  = "val:l,size:i,filename:s,cpu:i?",
>          .mhandler.cmd_new = qmp_marshal_input_memsave,
> diff --git a/qmp.c b/qmp.c
> index ed6c7ef..dd34be6 100644
> --- a/qmp.c
> +++ b/qmp.c
> @@ -24,6 +24,7 @@
>  #include "hw/qdev.h"
>  #include "sysemu/blockdev.h"
>  #include "qom/qom-qobject.h"
> +#include "hw/boards.h"
>  
>  NameInfo *qmp_query_name(Error **errp)
>  {
> @@ -108,6 +109,15 @@ void qmp_cpu(int64_t index, Error **errp)
>      /* Just do nothing */
>  }
>  
> +void qmp_cpu_add(int64_t id, Error **errp)
> +{
> +    if (current_machine->hot_add_cpu) {
> +        current_machine->hot_add_cpu(id, errp);
> +    } else {
> +        error_setg(errp, "Not supported");
> +    }
> +}
> +
>  #ifndef CONFIG_VNC
>  /* If VNC support is enabled, the "true" query-vnc command is
>     defined in the VNC subsystem */

Re: [Qemu-devel] [PATCH 21/21] QMP: add cpu-add command

[Igor Mammedov]

On Tue, 23 Apr 2013 09:26:22 -0400
Luiz Capitulino <lcapitulino@redhat.com> wrote:

> On Tue, 23 Apr 2013 10:29:55 +0200
> Igor Mammedov <imammedo@redhat.com> wrote:
> 
> > Adds "cpu-add id=xxx" QMP command.
> > 
> > cpu-add's "id" argument is a CPU number in a range [0..max-cpus)
> > 
> > Example QMP command:
> >  -> { "execute": "cpu-add", "arguments": { "id": 2 } }
> >  <- { "return": {} }
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> 
> QMP parts (this and the previous patch) is trivial:
> 
> Acked-by: Luiz Capitulino <lcapitulino@redhat.com>
> 
> Just wondering, no HMP command?
Nope, libvirt does prefers QMP and humans could use QMP shell to the same
effect. In shining future humans could use device_add when implemented from
monitor, provided it's not deprecated by then.

> 
> > ---
> > v6:
> >   * added valid values description to qapi schema
> >   * split out cpu_hot_add hooks introduction into separate patch
> >   * split out implementation of cpu_hot_add for target-i386
> > v5:
> >   * accept id=[0..max_cpus) range in cpu-add command
> > v4:
> >   * merge "qmp: add cpu-add qmp command" & "target-i386: implement CPU
> > hot-add" patches
> >   * move notifier call to CPUCLass.realize()
> >   * add hook cpu_hot_add to QEMUMachine
> >   * make QEMUMachineInitArgs global and keep default cpu_model there
> > 
> > v3:
> >   * it appears that 'online/offline' in cpu-set are confusing people
> >     with what command actually does and users might have to distinguish
> >     if 'offline' is not implemented by parsing error message. To simplify
> >     things replace cpu-set with cpu-add command to show more clear what
> >     command does and just add cpu-del when CPU remove is implemented.
> > 
> > v2:
> >   * s/cpu_set/cpu-set/
> >   * qmp doc style fix
> >   * use bool type instead of opencodding online/offline string
> >      suggested-by: Eric Blake <eblake@redhat.com>
> > ---
> >  qapi-schema.json |   11 +++++++++++
> >  qmp-commands.hx  |   23 +++++++++++++++++++++++
> >  qmp.c            |   10 ++++++++++
> >  3 files changed, 44 insertions(+), 0 deletions(-)
> > 
> > diff --git a/qapi-schema.json b/qapi-schema.json
> > index 751d3c2..11a5c82 100644
> > --- a/qapi-schema.json
> > +++ b/qapi-schema.json
> > @@ -1387,6 +1387,17 @@
> >  { 'command': 'cpu', 'data': {'index': 'int'} }
> >  
> >  ##
> > +# @cpu-add
> > +#
> > +# Adds CPU with specified ID
> > +#
> > +# @id: ID of CPU to be created, valid values [0..max_cpus)
> > +#
> > +# Returns: Nothing on success
> > +##
> > +{ 'command': 'cpu-add', 'data': {'id': 'int'} }
> > +
> > +##
> >  # @memsave:
> >  #
> >  # Save a portion of guest memory to a file.
> > diff --git a/qmp-commands.hx b/qmp-commands.hx
> > index 4d65422..1e5d299 100644
> > --- a/qmp-commands.hx
> > +++ b/qmp-commands.hx
> > @@ -385,6 +385,29 @@ Note: CPUs' indexes are obtained with the
> > 'query-cpus' command. EQMP
> >  
> >      {
> > +        .name       = "cpu-add",
> > +        .args_type  = "id:i",
> > +        .mhandler.cmd_new = qmp_marshal_input_cpu_add,
> > +    },
> > +
> > +SQMP
> > +cpu-add
> > +-------
> > +
> > +Adds virtual cpu
> > +
> > +Arguments:
> > +
> > +- "id": cpu id (json-int)
> > +
> > +Example:
> > +
> > +-> { "execute": "cpu-add", "arguments": { "id": 2 } }
> > +<- { "return": {} }
> > +
> > +EQMP
> > +
> > +    {
> >          .name       = "memsave",
> >          .args_type  = "val:l,size:i,filename:s,cpu:i?",
> >          .mhandler.cmd_new = qmp_marshal_input_memsave,
> > diff --git a/qmp.c b/qmp.c
> > index ed6c7ef..dd34be6 100644
> > --- a/qmp.c
> > +++ b/qmp.c
> > @@ -24,6 +24,7 @@
> >  #include "hw/qdev.h"
> >  #include "sysemu/blockdev.h"
> >  #include "qom/qom-qobject.h"
> > +#include "hw/boards.h"
> >  
> >  NameInfo *qmp_query_name(Error **errp)
> >  {
> > @@ -108,6 +109,15 @@ void qmp_cpu(int64_t index, Error **errp)
> >      /* Just do nothing */
> >  }
> >  
> > +void qmp_cpu_add(int64_t id, Error **errp)
> > +{
> > +    if (current_machine->hot_add_cpu) {
> > +        current_machine->hot_add_cpu(id, errp);
> > +    } else {
> > +        error_setg(errp, "Not supported");
> > +    }
> > +}
> > +
> >  #ifndef CONFIG_VNC
> >  /* If VNC support is enabled, the "true" query-vnc command is
> >     defined in the VNC subsystem */
> 
> 

Re: [Qemu-devel] [PATCH 21/21] QMP: add cpu-add command

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 940 bytes --]

On 04/23/2013 02:29 AM, Igor Mammedov wrote:
> Adds "cpu-add id=xxx" QMP command.
> 
> cpu-add's "id" argument is a CPU number in a range [0..max-cpus)
> 
> Example QMP command:
>  -> { "execute": "cpu-add", "arguments": { "id": 2 } }
>  <- { "return": {} }
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---

> +++ b/qapi-schema.json
> @@ -1387,6 +1387,17 @@
>  { 'command': 'cpu', 'data': {'index': 'int'} }
>  
>  ##
> +# @cpu-add
> +#
> +# Adds CPU with specified ID
> +#
> +# @id: ID of CPU to be created, valid values [0..max_cpus)
> +#
> +# Returns: Nothing on success

Missing a 'Since 1.5' designation.

> +##
> +{ 'command': 'cpu-add', 'data': {'id': 'int'} }
> +

As that's a trivial piece of documentation to add, you can use:

Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]