[Qemu-devel] [PATCH 00/16 v4] target-i386: CPU hot-add with cpu-add QMP command

[Qemu-devel] [PATCH 00/16 v4] target-i386: CPU hot-add with cpu-add QMP command

[Igor Mammedov]

Implements alternative way for hot-adding CPU using cpu-add QMP command,
wich could be useful until it would be possible to add CPUs via device_add.

All patches except the last are also applicable to device_add aprroach.

To hot-add CPU use following command from qmp-shell:
 cpu-add id=[0..max-cpus - 1)

git tree for testing: https://github.com/imammedo/qemu/tree/cpu_add.v4

based on qom-cpu tree

v4->v3:
  * 'id' in cpu-add command will be a thread number instead of APIC ID
  * split off resume_vcpu() into separate patch
  * move notifier from rtc code into pc.c

v2->v3:
  * use local error & propagate_error() instead of operating on
    passed in errp in several places
  * replace CPUClass.get_firmware_id() with CPUClass.get_arch_id()
  * leave IOAPIC creation to board and just set bus to icc-bus
  * include kvm-stub.o in cpu libary if no KVM is configured
  * create resume_vcpu() stub and include it in libqemustub,
    and use it directly instead of CPU method
  * acpi_piix4: s/cpu_add_notifier/cpu_added_notifier/

v1->v2:
  * generalize cpu sync to KVM, resume and hot-plug notification and
    invoke them form CPUClass, to make available to all targets.
  * introduce cpu_exists() and CPUClass.get_firmware_id() and use
    the last one in acpi_piix to make code target independent.
  * move IOAPIC to ICC bus, it was suggested and easy to convert.
  * leave kvmvapic as SysBusDevice, it doesn't affect hot-plug and
    created only once for all APIC instances. I haven't found yet
    good/clean enough way to convert it to ICCDevice. May be follow-up
    though.
  * split one big ICC patch into several, one per converted device
  * add cpu_hot_add hook to machine and implement it for target-i386,
    instead of adding stabs. Could be used by other targets to
    implement cpu-add.
  * pre-allocate links<CPU> for all possible CPUs and make them available
    at /machine/icc-bridge/cpu[0..N] QOM path, so users could find out
    possible/free CPU IDs to use in cpu-add command.

CC: pbonzini@redhat.com
CC: afaerber@suse.de
CC: ehabkost@redhat.com

Igor Mammedov (16):
  cpu: make kvm-stub.o a part of CPU library
  cpu: call cpu_synchronize_post_init() from CPUClass.realize() if
    hotplugged
  introduce resume_vcpu(), for single CPU
  cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is
    hot-plugged
  introduce CPU hot-plug notifier
  target-i386: pc: update rtc_cmos on CPU hot-plug
  cpu: introduce get_arch_id() method and override it for target-i386
  cpu: add helper cpu_exists(), to check if CPU with specified id exists
  acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest
  target-i386: introduce apic-id property
  introduce ICC bus/device/bridge
  target-i386: cpu: attach ICC bus to CPU on its creation
  target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE
  target-i386: move APIC to ICC bus
  target-i386: move IOAPIC to ICC bus
  add cpu-add qmp command and implement CPU hot-add for target-i386

 Makefile.target                    |  14 ++---
 cpus.c                             |  11 +++-
 default-configs/i386-softmmu.mak   |   1 +
 default-configs/x86_64-softmmu.mak |   1 +
 hw/acpi/piix4.c                    | 114 ++++++++++++++++++++++++++++++++++++-
 hw/cpu/Makefile.objs               |   1 +
 hw/cpu/icc_bus.c                   | 104 +++++++++++++++++++++++++++++++++
 hw/i386/kvm/apic.c                 |   2 +-
 hw/i386/kvm/ioapic.c               |   2 +-
 hw/i386/kvmvapic.c                 |   1 +
 hw/i386/pc.c                       |  91 +++++++++++++++++++++++++++--
 hw/i386/pc_piix.c                  |   7 +++
 hw/i386/pc_q35.c                   |   7 +++
 hw/intc/apic.c                     |   2 +-
 hw/intc/apic_common.c              |  17 ++++--
 hw/intc/ioapic_common.c            |  15 +++--
 hw/timer/mc146818rtc.c             |   7 +++
 hw/xen/xen_apic.c                  |   2 +-
 include/exec/memory.h              |  10 ++++
 include/hw/boards.h                |   3 +
 include/hw/i386/apic_internal.h    |   8 +--
 include/hw/i386/icc_bus.h          |  53 +++++++++++++++++
 include/hw/i386/ioapic_internal.h  |   6 +-
 include/hw/pci/msi.h               |   2 +
 include/hw/timer/mc146818rtc.h     |   1 +
 include/qom/cpu.h                  |  21 +++++++
 include/sysemu/kvm.h               |  22 +++----
 include/sysemu/sysemu.h            |   3 +
 kvm-all.c                          |   1 +
 kvm-stub.c                         |   3 +
 memory.c                           |  11 ++++
 qapi-schema.json                   |  11 ++++
 qmp-commands.hx                    |  23 ++++++++
 qmp.c                              |  10 ++++
 qom/cpu.c                          |  44 ++++++++++++++
 stubs/Makefile.objs                |   1 +
 stubs/cpus.c                       |   5 ++
 target-i386/cpu.c                  |  75 +++++++++++++++++++-----
 target-i386/cpu.h                  |   1 +
 vl.c                               |   7 ++-
 40 files changed, 656 insertions(+), 64 deletions(-)
 create mode 100644 hw/cpu/icc_bus.c
 create mode 100644 include/hw/i386/icc_bus.h
 create mode 100644 stubs/cpus.c

-- 
1.8.2

[Qemu-devel] [PATCH 01/16] cpu: make kvm-stub.o a part of CPU library

[Igor Mammedov]

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v2:
  remove unnecessary kvm-stub.o from "System emulator target" section
---
 Makefile.target      | 14 +++++++-------
 include/hw/pci/msi.h |  2 ++
 include/sysemu/kvm.h |  4 ++--
 kvm-stub.c           |  2 ++
 4 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/Makefile.target b/Makefile.target
index 2bd6d14..9bde8e5 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -64,6 +64,12 @@ all: $(PROGS) stap
 # Dummy command so that make thinks it has done something
 	@true
 
+CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
+CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
+CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
+CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
+CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
+
 #########################################################
 # cpu emulator library
 obj-y = exec.o translate-all.o cpu-exec.o
@@ -74,6 +80,7 @@ obj-y += fpu/softfloat.o
 obj-y += target-$(TARGET_BASE_ARCH)/
 obj-y += disas.o
 obj-$(CONFIG_GDBSTUB_XML) += gdbstub-xml.o
+obj-$(CONFIG_NO_KVM) += kvm-stub.o
 
 #########################################################
 # Linux user emulator target
@@ -102,18 +109,11 @@ endif #CONFIG_BSD_USER
 #########################################################
 # System emulator target
 ifdef CONFIG_SOFTMMU
-CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
-CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
-CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
-CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
-CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
-
 obj-y += arch_init.o cpus.o monitor.o gdbstub.o balloon.o ioport.o
 obj-y += qtest.o
 obj-y += hw/
 obj-$(CONFIG_FDT) += device_tree.o
 obj-$(CONFIG_KVM) += kvm-all.o
-obj-$(CONFIG_NO_KVM) += kvm-stub.o
 obj-y += memory.o savevm.o cputlb.o
 obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
 obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
diff --git a/include/hw/pci/msi.h b/include/hw/pci/msi.h
index 81a3848..d4d0450 100644
--- a/include/hw/pci/msi.h
+++ b/include/hw/pci/msi.h
@@ -21,6 +21,7 @@
 #ifndef QEMU_MSI_H
 #define QEMU_MSI_H
 
+#ifndef CONFIG_USER_ONLY
 #include "qemu-common.h"
 #include "hw/pci/pci.h"
 
@@ -47,4 +48,5 @@ static inline bool msi_present(const PCIDevice *dev)
     return dev->cap_present & QEMU_PCI_CAP_MSI;
 }
 
+#endif
 #endif /* QEMU_MSI_H */
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 495e6f8..93cef28 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
 #if !defined(CONFIG_USER_ONLY)
 void *kvm_vmalloc(ram_addr_t size);
 void *kvm_arch_vmalloc(ram_addr_t size);
-void kvm_setup_guest_memory(void *start, size_t size);
+#endif
 
+void kvm_setup_guest_memory(void *start, size_t size);
 void kvm_flush_coalesced_mmio_buffer(void);
-#endif
 
 int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
                           target_ulong len, int type);
diff --git a/kvm-stub.c b/kvm-stub.c
index 3e5c201..a131305 100644
--- a/kvm-stub.c
+++ b/kvm-stub.c
@@ -122,6 +122,7 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
+#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;
@@ -145,3 +146,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
 {
     return -ENOSYS;
 }
+#endif
-- 
1.8.2

[Qemu-devel] [PATCH 02/16] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

[Igor Mammedov]

... to synchronize CPU state to KVM

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v2:
  * linking kvm-stub.o to *-user target moved in separate patch
---
 include/sysemu/kvm.h | 18 ++++++++++--------
 kvm-all.c            |  1 +
 kvm-stub.c           |  1 +
 qom/cpu.c            |  4 ++++
 vl.c                 |  1 -
 5 files changed, 16 insertions(+), 9 deletions(-)

diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 93cef28..8fb2489 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -250,8 +250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension);
 uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
                                       uint32_t index, int reg);
 void kvm_cpu_synchronize_state(CPUArchState *env);
-void kvm_cpu_synchronize_post_reset(CPUState *cpu);
-void kvm_cpu_synchronize_post_init(CPUState *cpu);
 
 /* generic hooks - to be moved/refactored once there are more users */
 
@@ -262,6 +260,16 @@ static inline void cpu_synchronize_state(CPUArchState *env)
     }
 }
 
+#if !defined(CONFIG_USER_ONLY)
+int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
+                                       hwaddr *phys_addr);
+#endif
+
+#endif /* NEED_CPU_H */
+
+void kvm_cpu_synchronize_post_reset(CPUState *cpu);
+void kvm_cpu_synchronize_post_init(CPUState *cpu);
+
 static inline void cpu_synchronize_post_reset(CPUState *cpu)
 {
     if (kvm_enabled()) {
@@ -277,12 +285,6 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
 }
 
 
-#if !defined(CONFIG_USER_ONLY)
-int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
-                                       hwaddr *phys_addr);
-#endif
-
-#endif
 int kvm_set_ioeventfd_mmio(int fd, uint32_t adr, uint32_t val, bool assign,
                            uint32_t size);
 
diff --git a/kvm-all.c b/kvm-all.c
index fc4e17c..1d17128 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
     KVM_CAP_INFO(USER_MEMORY),
diff --git a/kvm-stub.c b/kvm-stub.c
index a131305..4282e09 100644
--- a/kvm-stub.c
+++ b/kvm-stub.c
@@ -23,6 +23,7 @@ bool kvm_async_interrupts_allowed;
 bool kvm_irqfds_allowed;
 bool kvm_msi_via_irqfd_allowed;
 bool kvm_gsi_routing_allowed;
+bool kvm_allowed;
 
 int kvm_init_vcpu(CPUState *cpu)
 {
diff --git a/qom/cpu.c b/qom/cpu.c
index e242dcb..0c76712 100644
--- a/qom/cpu.c
+++ b/qom/cpu.c
@@ -20,6 +20,7 @@
 
 #include "qom/cpu.h"
 #include "qemu-common.h"
+#include "sysemu/kvm.h"
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -57,6 +58,9 @@ static ObjectClass *cpu_common_class_by_name(const char *cpu_model)
 
 static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
+    if (dev->hotplugged) {
+        cpu_synchronize_post_init(CPU(dev));
+    }
 }
 
 static void cpu_class_init(ObjectClass *klass, void *data)
diff --git a/vl.c b/vl.c
index dd94549..bc9c016 100644
--- a/vl.c
+++ b/vl.c
@@ -267,7 +267,6 @@ static NotifierList machine_init_done_notifiers =
     NOTIFIER_LIST_INITIALIZER(machine_init_done_notifiers);
 
 static bool tcg_allowed = true;
-bool kvm_allowed;
 bool xen_allowed;
 uint32_t xen_domid;
 enum xen_mode xen_mode = XEN_EMULATE;
-- 
1.8.2

[Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

[Igor Mammedov]

... also add stub for it, to make possible to use it in qom/cpu.c,
which is shared with user emulators.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
Note:
  it will be re-used from qom/cpu.c by hotpluged CPU
---
 cpus.c              | 11 ++++++++---
 include/qom/cpu.h   |  9 +++++++++
 stubs/Makefile.objs |  1 +
 stubs/cpus.c        |  5 +++++
 4 files changed, 23 insertions(+), 3 deletions(-)
 create mode 100644 stubs/cpus.c

diff --git a/cpus.c b/cpus.c
index c15ff6c..c347db5 100644
--- a/cpus.c
+++ b/cpus.c
@@ -993,6 +993,13 @@ void pause_all_vcpus(void)
     }
 }
 
+void resume_vcpu(CPUState *cpu)
+{
+    cpu->stop = false;
+    cpu->stopped = false;
+    qemu_cpu_kick(cpu);
+}
+
 void resume_all_vcpus(void)
 {
     CPUArchState *penv = first_cpu;
@@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
     qemu_clock_enable(vm_clock, true);
     while (penv) {
         CPUState *pcpu = ENV_GET_CPU(penv);
-        pcpu->stop = false;
-        pcpu->stopped = false;
-        qemu_cpu_kick(pcpu);
+        resume_vcpu(pcpu);
         penv = penv->next_cpu;
     }
 }
diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index 3664a1b..6ce6f10 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
  */
 void cpu_reset_interrupt(CPUState *cpu, int mask);
 
+/**
+ * resume_vcpu:
+ * @cpu: The CPU to resume.
+ *
+ * Resumes CPU, i.e. puts CPU into runable state.
+ * KVM requires extra call cpu_synchronize_post_init() to make
+ * KVM's CPU instance runable.
+ */
+void resume_vcpu(CPUState *cpu);
 
 #endif
diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs
index 9c55b34..03dff20 100644
--- a/stubs/Makefile.objs
+++ b/stubs/Makefile.objs
@@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
 stub-obj-y += vm-stop.o
 stub-obj-y += vmstate.o
 stub-obj-$(CONFIG_WIN32) += fd-register.o
+stub-obj-y += cpus.o
diff --git a/stubs/cpus.c b/stubs/cpus.c
new file mode 100644
index 0000000..3d17451
--- /dev/null
+++ b/stubs/cpus.c
@@ -0,0 +1,5 @@
+#include "qom/cpu.h"
+
+void resume_vcpu(CPUState *cpu)
+{
+}
-- 
1.8.2

[Qemu-devel] [PATCH 04/16] cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is hot-plugged

[Igor Mammedov]

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v2:
 * remove unnecessary now sysemu/cpus.h header
 * move out introducing resume_vcpu() into a separate patch
---
 qom/cpu.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/qom/cpu.c b/qom/cpu.c
index 0c76712..40a4259 100644
--- a/qom/cpu.c
+++ b/qom/cpu.c
@@ -60,6 +60,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
 {
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
+        resume_vcpu(CPU(dev));
     }
 }
 
-- 
1.8.2

[Qemu-devel] [PATCH 05/16] introduce CPU hot-plug notifier

[Igor Mammedov]

hot-added CPU will be distributed to acpi_piix4 and rtc_cmos

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
---
v2:
  * move notifier to qom/cpu.c and call it from CPUClass.realize() on hotplug
  * remove get_firmware_id() since it belong to other patch
---
 include/sysemu/sysemu.h |  3 +++
 qom/cpu.c               | 12 ++++++++++++
 2 files changed, 15 insertions(+)

diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 6578782..a8c3de1 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const QDict *qdict);
 /* generic hotplug */
 void drive_hot_add(Monitor *mon, const QDict *qdict);
 
+/* CPU hotplug */
+void qemu_register_cpu_added_notifier(Notifier *notifier);
+
 /* pcie aer error injection */
 void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
 int do_pcie_aer_inject_error(Monitor *mon,
diff --git a/qom/cpu.c b/qom/cpu.c
index 40a4259..819986e 100644
--- a/qom/cpu.c
+++ b/qom/cpu.c
@@ -21,6 +21,17 @@
 #include "qom/cpu.h"
 #include "qemu-common.h"
 #include "sysemu/kvm.h"
+#include "qemu/notify.h"
+#include "sysemu/sysemu.h"
+
+/* CPU hot-plug notifiers */
+static NotifierList cpu_added_notifiers =
+    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
+
+void qemu_register_cpu_added_notifier(Notifier *notifier)
+{
+    notifier_list_add(&cpu_added_notifiers, notifier);
+}
 
 void cpu_reset_interrupt(CPUState *cpu, int mask)
 {
@@ -61,6 +72,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     if (dev->hotplugged) {
         cpu_synchronize_post_init(CPU(dev));
         resume_vcpu(CPU(dev));
+        notifier_list_notify(&cpu_added_notifiers, dev);
     }
 }
 
-- 
1.8.2

[Qemu-devel] [PATCH 06/16] target-i386: pc: update rtc_cmos on CPU hot-plug

[Igor Mammedov]

... so that on reboot BIOS could read current available CPU count

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 hw/i386/pc.c                   | 20 ++++++++++++++++++++
 hw/timer/mc146818rtc.c         |  7 +++++++
 include/hw/timer/mc146818rtc.h |  1 +
 3 files changed, 28 insertions(+)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 8d75b34..dc1a78b 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -337,6 +337,21 @@ static void pc_cmos_init_late(void *opaque)
     qemu_unregister_reset(pc_cmos_init_late, opaque);
 }
 
+typedef struct rtc_cpu_hotplug_arg {
+    ISADevice *rtc_state;
+    Notifier cpu_added_notifier;
+} rtc_cpu_hotplug_arg;
+
+static void rtc_notify_cpu_added(Notifier *notifier, void *data)
+{
+    rtc_cpu_hotplug_arg *arg = container_of(notifier, rtc_cpu_hotplug_arg,
+                                           cpu_added_notifier);
+    ISADevice *s = arg->rtc_state;
+
+    /* increment the number of CPUs */
+    rtc_set_memory(s, 0x5f, rtc_get_memory(s, 0x5f) + 1);
+}
+
 void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
                   const char *boot_device,
                   ISADevice *floppy, BusState *idebus0, BusState *idebus1,
@@ -345,6 +360,7 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
     int val, nb, i;
     FDriveType fd_type[2] = { FDRIVE_DRV_NONE, FDRIVE_DRV_NONE };
     static pc_cmos_init_late_arg arg;
+    static rtc_cpu_hotplug_arg cpu_hotplug_cb;
 
     /* various important CMOS locations needed by PC/Bochs bios */
 
@@ -383,6 +399,10 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
 
     /* set the number of CPU */
     rtc_set_memory(s, 0x5f, smp_cpus - 1);
+    /* init CPU hotplug notifier */
+    cpu_hotplug_cb.rtc_state = s;
+    cpu_hotplug_cb.cpu_added_notifier.notify = rtc_notify_cpu_added;
+    qemu_register_cpu_added_notifier(&cpu_hotplug_cb.cpu_added_notifier);
 
     /* set boot devices, and disable floppy signature check if requested */
     if (set_boot_dev(s, boot_device, fd_bootchk)) {
diff --git a/hw/timer/mc146818rtc.c b/hw/timer/mc146818rtc.c
index 69e6844..e639942 100644
--- a/hw/timer/mc146818rtc.c
+++ b/hw/timer/mc146818rtc.c
@@ -677,6 +677,13 @@ void rtc_set_memory(ISADevice *dev, int addr, int val)
         s->cmos_data[addr] = val;
 }
 
+int rtc_get_memory(ISADevice *dev, int addr)
+{
+    RTCState *s = DO_UPCAST(RTCState, dev, dev);
+    assert(addr >= 0 && addr <= 127);
+    return s->cmos_data[addr];
+}
+
 static void rtc_set_date_from_host(ISADevice *dev)
 {
     RTCState *s = DO_UPCAST(RTCState, dev, dev);
diff --git a/include/hw/timer/mc146818rtc.h b/include/hw/timer/mc146818rtc.h
index 854ea3f..09f37b7 100644
--- a/include/hw/timer/mc146818rtc.h
+++ b/include/hw/timer/mc146818rtc.h
@@ -6,6 +6,7 @@
 
 ISADevice *rtc_init(ISABus *bus, int base_year, qemu_irq intercept_irq);
 void rtc_set_memory(ISADevice *dev, int addr, int val);
+int rtc_get_memory(ISADevice *dev, int addr);
 void rtc_set_date(ISADevice *dev, const struct tm *tm);
 
 #endif /* !MC146818RTC_H */
-- 
1.8.2

[Qemu-devel] [PATCH 07/16] cpu: introduce get_arch_id() method and override it for target-i386

[Igor Mammedov]

get_arch_id() adds possibility for generic code to get guest visible
CPU ID without accessing CPUArchState. If target doesn't override it,
it will return cpu_index.

Override it on target-i386 to return APIC ID.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 * it will be used later by new cpu_exists() generic function and
   acpi_piix.
 * s/cpu_firmware_id/cpu_arch_id/
---
 include/qom/cpu.h |  2 ++
 qom/cpu.c         |  6 ++++++
 target-i386/cpu.c | 10 ++++++++++
 3 files changed, 18 insertions(+)

diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index 6ce6f10..428aaf0 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
  * instantiatable CPU type.
  * @reset: Callback to reset the #CPUState to its initial state.
  * @do_interrupt: Callback for interrupt handling.
+ * @get_arch_id: Callback for getting architecture depended CPU ID
  * @vmsd: State description for migration.
  *
  * Represents a CPU family or model.
@@ -58,6 +59,7 @@ typedef struct CPUClass {
 
     void (*reset)(CPUState *cpu);
     void (*do_interrupt)(CPUState *cpu);
+    int64_t (*get_arch_id)(CPUState *cpu);
 
     const struct VMStateDescription *vmsd;
 } CPUClass;
diff --git a/qom/cpu.c b/qom/cpu.c
index 819986e..a13ddda 100644
--- a/qom/cpu.c
+++ b/qom/cpu.c
@@ -76,6 +76,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     }
 }
 
+static int64_t cpu_common_get_arch_id(CPUState *cpu)
+{
+    return cpu->cpu_index;
+}
+
 static void cpu_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -83,6 +88,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
 
     k->class_by_name = cpu_common_class_by_name;
     k->reset = cpu_common_reset;
+    k->get_arch_id = cpu_common_get_arch_id;
     dc->realize = cpu_common_realizefn;
     dc->no_user = 1;
 }
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index e2302d8..a415fa3 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
     }
 }
 
+static int64_t x86_cpu_get_arch_id(CPUState *cpu)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+
+    return env->cpuid_apic_id;
+}
+
 static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 {
     X86CPUClass *xcc = X86_CPU_CLASS(oc);
@@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     cc->do_interrupt = x86_cpu_do_interrupt;
     cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
+
+    cc->get_arch_id = x86_cpu_get_arch_id;
 }
 
 static const TypeInfo x86_cpu_type_info = {
-- 
1.8.2

[Qemu-devel] [PATCH 08/16] cpu: add helper cpu_exists(), to check if CPU with specified id exists

[Igor Mammedov]

... it should be used only on slow path since it does recursive search
    on /machine QOM tree for objects of TYPE_CPU

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
v2:
  * s/get_firmware_id()/get_arch_id()/ rebase fixup
  * remove check for get_arch_id being NULL, since it's always defined
---
 include/qom/cpu.h | 10 ++++++++++
 qom/cpu.c         | 21 +++++++++++++++++++++
 2 files changed, 31 insertions(+)

diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index 428aaf0..33d5e91 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -225,6 +225,16 @@ void run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data);
  */
 CPUState *qemu_get_cpu(int index);
 
+/**
+ * cpu_exists:
+ * @id - guest exposed CPU ID to lookup
+ *
+ * Search for CPU with specified ID.
+ *
+ * Returns: true - CPU is found, false - CPU isn't found
+ */
+bool cpu_exists(int64_t id);
+
 #ifndef CONFIG_USER_ONLY
 
 typedef void (*CPUInterruptHandler)(CPUState *, int);
diff --git a/qom/cpu.c b/qom/cpu.c
index a13ddda..02fab9a 100644
--- a/qom/cpu.c
+++ b/qom/cpu.c
@@ -24,6 +24,27 @@
 #include "qemu/notify.h"
 #include "sysemu/sysemu.h"
 
+static int cpu_exist_cb(Object *obj, void *opaque)
+{
+    int64_t id = *(int64_t *)opaque;
+    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
+
+    if (cpu_obj) {
+        CPUState *cpu = CPU(cpu_obj);
+        CPUClass *klass = CPU_GET_CLASS(cpu);
+
+        if (klass->get_arch_id(cpu) == id) {
+            return 1;
+        }
+    }
+    return object_child_foreach(obj, cpu_exist_cb, opaque);
+}
+
+bool cpu_exists(int64_t id)
+{
+   return cpu_exist_cb(qdev_get_machine(), &id) ? true : false;
+}
+
 /* CPU hot-plug notifiers */
 static NotifierList cpu_added_notifiers =
     NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
-- 
1.8.2

[Qemu-devel] [PATCH 09/16] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Igor Mammedov]

* introduce processor status bitmask visible to guest at 0xaf00 addr,
  where ACPI asl code expects it
* set bit corresponding to APIC ID in processor status bitmask on
  receiving CPU hot-plug notification
* trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v3:
  * s/get_firmware_id()/get_arch_id()/ due rebase
  * s/cpu_add_notifier/cpu_added_notifier/
v2:
  * use CPUClass.get_firmware_id() to make code target independent
  * bump up vmstate_acpi version
---
 hw/acpi/piix4.c | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 113 insertions(+), 3 deletions(-)

diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
index 88386d7..eecdbb0 100644
--- a/hw/acpi/piix4.c
+++ b/hw/acpi/piix4.c
@@ -48,19 +48,28 @@
 #define PCI_EJ_BASE 0xae08
 #define PCI_RMV_BASE 0xae0c
 
+#define PROC_BASE 0xaf00
+#define PROC_LEN 32
+
 #define PIIX4_PCI_HOTPLUG_STATUS 2
+#define PIIX4_CPU_HOTPLUG_STATUS 4
 
 struct pci_status {
     uint32_t up; /* deprecated, maintained for migration compatibility */
     uint32_t down;
 };
 
+struct cpu_status {
+    uint8_t sts[PROC_LEN];
+};
+
 typedef struct PIIX4PMState {
     PCIDevice dev;
 
     MemoryRegion io;
     MemoryRegion io_gpe;
     MemoryRegion io_pci;
+    MemoryRegion io_cpu;
     ACPIREGS ar;
 
     APMState apm;
@@ -82,6 +91,9 @@ typedef struct PIIX4PMState {
     uint8_t disable_s3;
     uint8_t disable_s4;
     uint8_t s4_val;
+
+    struct cpu_status gpe_cpu;
+    Notifier cpu_added_notifier;
 } PIIX4PMState;
 
 static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
@@ -100,8 +112,8 @@ static void pm_update_sci(PIIX4PMState *s)
                    ACPI_BITMASK_POWER_BUTTON_ENABLE |
                    ACPI_BITMASK_GLOBAL_LOCK_ENABLE |
                    ACPI_BITMASK_TIMER_ENABLE)) != 0) ||
-        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0])
-          & PIIX4_PCI_HOTPLUG_STATUS) != 0);
+        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) &
+          (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0);
 
     qemu_set_irq(s->irq, sci_level);
     /* schedule a timer interruption if needed */
@@ -257,6 +269,17 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
     return ret;
 }
 
+#define VMSTATE_CPU_STATUS_ARRAY(_field, _state)                             \
+ {                                                                           \
+     .name       = (stringify(_field)),                                      \
+     .version_id = 0,                                                        \
+     .num        = PROC_LEN,                                                 \
+     .info       = &vmstate_info_uint8,                                      \
+     .size       = sizeof(uint8_t),                                          \
+     .flags      = VMS_ARRAY,                                                \
+     .offset     = vmstate_offset_array(_state, _field, uint8_t, PROC_LEN),  \
+ }
+
 /* qemu-kvm 1.2 uses version 3 but advertised as 2
  * To support incoming qemu-kvm 1.2 migration, change version_id
  * and minimum_version_id to 2 below (which breaks migration from
@@ -265,7 +288,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
  */
 static const VMStateDescription vmstate_acpi = {
     .name = "piix4_pm",
-    .version_id = 3,
+    .version_id = 4,
     .minimum_version_id = 3,
     .minimum_version_id_old = 1,
     .load_state_old = acpi_load_old,
@@ -281,6 +304,7 @@ static const VMStateDescription vmstate_acpi = {
         VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
         VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
                        struct pci_status),
+        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),
         VMSTATE_END_OF_LIST()
     }
 };
@@ -585,6 +609,85 @@ static const MemoryRegionOps piix4_pci_ops = {
     },
 };
 
+static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned width)
+{
+    PIIX4PMState *s = opaque;
+    struct cpu_status *cpus = &s->gpe_cpu;
+    uint64_t val = cpus->sts[addr];
+
+    return val;
+}
+
+static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
+                             unsigned int size)
+{
+    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
+}
+
+static const MemoryRegionOps cpu_hotplug_ops = {
+    .read = cpu_status_read,
+    .write = cpu_status_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+    .valid = {
+        .min_access_size = 1,
+        .max_access_size = 1,
+    },
+};
+
+typedef enum {
+    PLUG,
+    UNPLUG,
+} HotplugEventType;
+
+static void piix4_cpu_hotplug_req(PIIX4PMState *s, CPUState *cpu,
+                                  HotplugEventType action)
+{
+    struct cpu_status *g = &s->gpe_cpu;
+    ACPIGPE *gpe = &s->ar.gpe;
+    CPUClass *k = CPU_GET_CLASS(cpu);
+    int64_t cpu_id;
+
+    assert(s != NULL);
+
+    *gpe->sts = *gpe->sts | PIIX4_CPU_HOTPLUG_STATUS;
+    cpu_id = k->get_arch_id(CPU(cpu));
+    if (action == PLUG) {
+        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
+    } else {
+        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
+    }
+    pm_update_sci(s);
+}
+
+static void piix4_cpu_added_req(Notifier *n, void *opaque)
+{
+    PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
+
+    piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
+}
+
+static int piix4_init_cpu_status(Object *obj, void *opaque)
+{
+    struct cpu_status *g = (struct cpu_status *)opaque;
+    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
+
+    if (cpu_obj) {
+        struct Error *error = NULL;
+        CPUClass *k = CPU_GET_CLASS(cpu_obj);
+        int64_t id = k->get_arch_id(CPU(cpu_obj));
+
+        if (error) {
+            fprintf(stderr, "failed to initilize CPU status for ACPI: %s\n",
+                    error_get_pretty(error));
+            error_free(error);
+            abort();
+        }
+        g_assert((id / 8) < PROC_LEN);
+        g->sts[id / 8] |= (1 << (id % 8));
+    }
+    return object_child_foreach(obj, piix4_init_cpu_status, opaque);
+}
+
 static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
                                 PCIHotplugState state);
 
@@ -600,6 +703,13 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
     memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
                                 &s->io_pci);
     pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
+
+    piix4_init_cpu_status(qdev_get_machine(), &s->gpe_cpu);
+    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
+                          PROC_LEN);
+    memory_region_add_subregion(parent, PROC_BASE, &s->io_cpu);
+    s->cpu_added_notifier.notify = piix4_cpu_added_req;
+    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
 }
 
 static void enable_device(PIIX4PMState *s, int slot)
-- 
1.8.2

[Qemu-devel] [PATCH 10/16] target-i386: introduce apic-id property

[Igor Mammedov]

... and use it from board level to set APIC ID for CPUs
it creates.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
---
Note:
  * pc_new_cpu() function will be reused later in CPU hot-plug hook.

v3:
  * user error_propagate() in property setter
v2:
  * use generic cpu_exists() instead of custom one
  * make apic-id dynamic property, so it won't be possible to use it
    as global property, since it's instance specific
---
 hw/i386/pc.c      | 25 ++++++++++++++++++++++++-
 target-i386/cpu.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+), 1 deletion(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index dc1a78b..cb57878 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -889,9 +889,29 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
+static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+{
+    X86CPU *cpu;
+
+    cpu = cpu_x86_create(cpu_model, errp);
+    if (!cpu) {
+        return;
+    }
+
+    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
+    object_property_set_bool(OBJECT(cpu), true, "realized", errp);
+
+    if (error_is_set(errp)) {
+        if (cpu != NULL) {
+            object_unref(OBJECT(cpu));
+        }
+    }
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    Error *error = NULL;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -903,7 +923,10 @@ void pc_cpus_init(const char *cpu_model)
     }
 
     for (i = 0; i < smp_cpus; i++) {
-        if (!cpu_x86_init(cpu_model)) {
+        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        if (error) {
+            fprintf(stderr, "%s\n", error_get_pretty(error));
+            error_free(error);
             exit(1);
         }
     }
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index a415fa3..6d6c527 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -1271,6 +1271,45 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque,
     cpu->env.tsc_khz = value / 1000;
 }
 
+static void x86_cpuid_get_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    int64_t value = cpu->env.cpuid_apic_id;
+
+    visit_type_int(v, &value, name, errp);
+}
+
+static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
+                                  const char *name, Error **errp)
+{
+    X86CPU *cpu = X86_CPU(obj);
+    const int64_t min = 0;
+    const int64_t max = UINT32_MAX;
+    Error *error = NULL;
+    int64_t value;
+
+    visit_type_int(v, &value, name, &error);
+    if (error) {
+        error_propagate(errp, error);
+        return;
+    }
+    if (value < min || value > max) {
+        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
+                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
+                   object_get_typename(obj), name, value, min, max);
+        error_propagate(errp, error);
+        return;
+    }
+
+    if (cpu_exists(value)) {
+        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
+        error_propagate(errp, error);
+        return;
+    }
+    cpu->env.cpuid_apic_id = value;
+}
+
 static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, const char *name)
 {
     x86_def_t *def;
@@ -2259,6 +2298,9 @@ static void x86_cpu_initfn(Object *obj)
     object_property_add(obj, "tsc-frequency", "int",
                         x86_cpuid_get_tsc_freq,
                         x86_cpuid_set_tsc_freq, NULL, NULL, NULL);
+    object_property_add(obj, "apic-id", "int",
+                        x86_cpuid_get_apic_id,
+                        x86_cpuid_set_apic_id, NULL, NULL, NULL);
 
     env->cpuid_apic_id = x86_cpu_apic_id_from_index(cs->cpu_index);
 
-- 
1.8.2

[Qemu-devel] [PATCH 11/16] introduce ICC bus/device/bridge

[Igor Mammedov]

... to provide hotplug-able bus.

* icc-bridge will serve as a parent for icc-bus and provide
  mmio mapping services to child icc-devices.
* icc-device will replace SysBusDevice as a parent of APIC
  and IOAPIC devices.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v2:
  * Rebase on top the last HW reorganization series.
  * Move hw/icc_bus.c into hw/cpu/ and hw/icc_bus.h include/hw/i386/
---
 default-configs/i386-softmmu.mak   |  1 +
 default-configs/x86_64-softmmu.mak |  1 +
 hw/cpu/Makefile.objs               |  1 +
 hw/cpu/icc_bus.c                   | 88 ++++++++++++++++++++++++++++++++++++++
 hw/i386/pc_piix.c                  |  7 +++
 hw/i386/pc_q35.c                   |  7 +++
 include/hw/i386/icc_bus.h          | 53 +++++++++++++++++++++++
 7 files changed, 158 insertions(+)
 create mode 100644 hw/cpu/icc_bus.c
 create mode 100644 include/hw/i386/icc_bus.h

diff --git a/default-configs/i386-softmmu.mak b/default-configs/i386-softmmu.mak
index 4ac0694..0789303 100644
--- a/default-configs/i386-softmmu.mak
+++ b/default-configs/i386-softmmu.mak
@@ -45,3 +45,4 @@ CONFIG_LPC_ICH9=y
 CONFIG_Q35=y
 CONFIG_APIC=y
 CONFIG_IOAPIC=y
+CONFIG_ICC_BUS=y
diff --git a/default-configs/x86_64-softmmu.mak b/default-configs/x86_64-softmmu.mak
index bf4091c..8d03e9a 100644
--- a/default-configs/x86_64-softmmu.mak
+++ b/default-configs/x86_64-softmmu.mak
@@ -45,3 +45,4 @@ CONFIG_LPC_ICH9=y
 CONFIG_Q35=y
 CONFIG_APIC=y
 CONFIG_IOAPIC=y
+CONFIG_ICC_BUS=y
diff --git a/hw/cpu/Makefile.objs b/hw/cpu/Makefile.objs
index a49ca04..4461ece 100644
--- a/hw/cpu/Makefile.objs
+++ b/hw/cpu/Makefile.objs
@@ -1,4 +1,5 @@
 obj-$(CONFIG_ARM11MPCORE) += arm11mpcore.o
 obj-$(CONFIG_ARM9MPCORE) += a9mpcore.o
 obj-$(CONFIG_ARM15MPCORE) += a15mpcore.o
+obj-$(CONFIG_ICC_BUS) += icc_bus.o
 
diff --git a/hw/cpu/icc_bus.c b/hw/cpu/icc_bus.c
new file mode 100644
index 0000000..00b9be3
--- /dev/null
+++ b/hw/cpu/icc_bus.c
@@ -0,0 +1,88 @@
+/* icc_bus.c
+ * emulate x86 ICC(INTERRUPT CONTROLLER COMMUNICATIONS) bus
+ *
+ * Copyright (c) 2013 Red Hat, Inc
+ *
+ * Authors:
+ *     Igor Mammedov <imammedo@redhat.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>
+ */
+#include "hw/i386/icc_bus.h"
+#include "hw/sysbus.h"
+
+static void icc_bus_initfn(Object *obj)
+{
+    BusState *b = BUS(obj);
+    b->allow_hotplug = true;
+}
+
+static const TypeInfo icc_bus_info = {
+    .name = TYPE_ICC_BUS,
+    .parent = TYPE_BUS,
+    .instance_size = sizeof(ICCBus),
+    .instance_init = icc_bus_initfn,
+};
+
+static int icc_device_init(DeviceState *dev)
+{
+    ICCDevice *id = ICC_DEVICE(dev);
+    ICCDeviceClass *k = ICC_DEVICE_GET_CLASS(id);
+
+    return k->init(id);
+}
+
+static void icc_device_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *k = DEVICE_CLASS(klass);
+
+    k->init = icc_device_init;
+    k->bus_type = TYPE_ICC_BUS;
+}
+
+static const TypeInfo icc_device_info = {
+    .name = TYPE_ICC_DEVICE,
+    .parent = TYPE_DEVICE,
+    .abstract = true,
+    .instance_size = sizeof(ICCDevice),
+    .class_size = sizeof(ICCDeviceClass),
+    .class_init = icc_device_class_init,
+};
+
+typedef struct ICCBridgeState {
+    SysBusDevice busdev;
+} ICCBridgeState;
+#define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
+
+
+static void icc_bridge_initfn(Object *obj)
+{
+    qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus");
+}
+
+static const TypeInfo icc_bridge_info = {
+    .name  = "icc-bridge",
+    .parent = TYPE_SYS_BUS_DEVICE,
+    .instance_init  = icc_bridge_initfn,
+    .instance_size  = sizeof(ICCBridgeState),
+};
+
+static void icc_bus_register_types(void)
+{
+    type_register_static(&icc_bus_info);
+    type_register_static(&icc_device_info);
+    type_register_static(&icc_bridge_info);
+}
+
+type_init(icc_bus_register_types)
diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index cff8013..d8c3e1a 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -37,6 +37,7 @@
 #include "hw/kvm/clock.h"
 #include "sysemu/sysemu.h"
 #include "hw/sysbus.h"
+#include "hw/i386/icc_bus.h"
 #include "sysemu/arch_init.h"
 #include "sysemu/blockdev.h"
 #include "hw/i2c/smbus.h"
@@ -84,8 +85,13 @@ static void pc_init1(MemoryRegion *system_memory,
     MemoryRegion *ram_memory;
     MemoryRegion *pci_memory;
     MemoryRegion *rom_memory;
+    DeviceState *icc_bridge;
     void *fw_cfg = NULL;
 
+    icc_bridge = qdev_create(NULL, TYPE_ICC_BRIDGE);
+    object_property_add_child(qdev_get_machine(), "icc-bridge",
+                              OBJECT(icc_bridge), NULL);
+
     pc_cpus_init(cpu_model);
     pc_acpi_init("acpi-dsdt.aml");
 
@@ -160,6 +166,7 @@ static void pc_init1(MemoryRegion *system_memory,
     if (pci_enabled) {
         ioapic_init_gsi(gsi_state, "i440fx");
     }
+    qdev_init_nofail(icc_bridge);
 
     pc_register_ferr_irq(gsi[13]);
 
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 6ac1a89..9055145 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -41,6 +41,7 @@
 #include "hw/ide/pci.h"
 #include "hw/ide/ahci.h"
 #include "hw/usb.h"
+#include "hw/i386/icc_bus.h"
 
 /* ICH9 AHCI has 6 ports */
 #define MAX_SATA_PORTS     6
@@ -85,6 +86,11 @@ static void pc_q35_init(QEMUMachineInitArgs *args)
     ICH9LPCState *ich9_lpc;
     PCIDevice *ahci;
     qemu_irq *cmos_s3;
+    DeviceState *icc_bridge;
+
+    icc_bridge = qdev_create(NULL, TYPE_ICC_BRIDGE);
+    object_property_add_child(qdev_get_machine(), "icc-bridge",
+                              OBJECT(icc_bridge), NULL);
 
     pc_cpus_init(cpu_model);
     pc_acpi_init("q35-acpi-dsdt.aml");
@@ -168,6 +174,7 @@ static void pc_q35_init(QEMUMachineInitArgs *args)
     if (pci_enabled) {
         ioapic_init_gsi(gsi_state, NULL);
     }
+    qdev_init_nofail(icc_bridge);
 
     pc_register_ferr_irq(gsi[13]);
 
diff --git a/include/hw/i386/icc_bus.h b/include/hw/i386/icc_bus.h
new file mode 100644
index 0000000..aa30e0c
--- /dev/null
+++ b/include/hw/i386/icc_bus.h
@@ -0,0 +1,53 @@
+/* icc_bus.h
+ * emulate x86 ICC(INTERRUPT CONTROLLER COMMUNICATIONS) bus
+ *
+ * Copyright (c) 2013 Red Hat, Inc
+ *
+ * Authors:
+ *     Igor Mammedov <imammedo@redhat.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>
+ */
+#ifndef ICC_BUS_H
+#define ICC_BUS_H
+
+#include "hw/qdev-core.h"
+
+#define TYPE_ICC_BUS "icc-bus"
+
+#ifndef CONFIG_USER_ONLY
+typedef struct ICCBus {
+    BusState qbus;
+} ICCBus;
+#define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
+
+typedef struct ICCDevice {
+    DeviceState qdev;
+} ICCDevice;
+
+typedef struct ICCDeviceClass {
+    DeviceClass parent_class;
+    int (*init)(ICCDevice *dev);
+} ICCDeviceClass;
+#define TYPE_ICC_DEVICE "icc-device"
+#define ICC_DEVICE(obj) OBJECT_CHECK(ICCDevice, (obj), TYPE_ICC_DEVICE)
+#define ICC_DEVICE_CLASS(klass) \
+     OBJECT_CLASS_CHECK(ICCDeviceClass, (klass), TYPE_ICC_DEVICE)
+#define ICC_DEVICE_GET_CLASS(obj) \
+     OBJECT_GET_CLASS(ICCDeviceClass, (obj), TYPE_ICC_DEVICE)
+
+#define TYPE_ICC_BRIDGE "icc-bridge"
+
+#endif /* CONFIG_USER_ONLY */
+#endif
-- 
1.8.2

[Qemu-devel] [PATCH 12/16] target-i386: cpu: attach ICC bus to CPU on its creation

[Igor Mammedov]

... during startup,  so it would be possible to unplug it later
and set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add
attach hotplugged CPU to ICC bus.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 target-i386/cpu.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 6d6c527..3b5f90b 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -41,6 +41,7 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/i386/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
 #include "hw/sysbus.h"
@@ -1609,6 +1610,7 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
     gchar **model_pieces;
     char *name, *features;
     Error *error = NULL;
+    Object *icc_bus = object_resolve_path_type("icc-bus", TYPE_ICC_BUS, NULL);
 
     model_pieces = g_strsplit(cpu_model, ",", 2);
     if (!model_pieces[0]) {
@@ -1619,6 +1621,10 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
     features = model_pieces[1];
 
     cpu = X86_CPU(object_new(TYPE_X86_CPU));
+    if (icc_bus) {
+        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
+        object_unref(OBJECT(cpu));
+    }
     env = &cpu->env;
     env->cpu_model_str = cpu_model;
 
@@ -2330,6 +2336,7 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
 
     xcc->parent_realize = dc->realize;
     dc->realize = x86_cpu_realizefn;
+    dc->bus_type = TYPE_ICC_BUS;
 
     xcc->parent_reset = cc->reset;
     cc->reset = x86_cpu_reset;
-- 
1.8.2

[Qemu-devel] [PATCH 13/16] target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE

[Igor Mammedov]

... and put APIC_SPACE_SIZE in public header so that it could be
reused later elsewhere.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 hw/i386/kvm/apic.c              | 2 +-
 hw/intc/apic.c                  | 2 +-
 hw/xen/xen_apic.c               | 2 +-
 include/hw/i386/apic_internal.h | 2 --
 target-i386/cpu.h               | 1 +
 5 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/hw/i386/kvm/apic.c b/hw/i386/kvm/apic.c
index c6ff982..8f80425 100644
--- a/hw/i386/kvm/apic.c
+++ b/hw/i386/kvm/apic.c
@@ -174,7 +174,7 @@ static const MemoryRegionOps kvm_apic_io_ops = {
 static void kvm_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &kvm_apic_io_ops, s, "kvm-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     if (kvm_has_gsi_routing()) {
         msi_supported = true;
diff --git a/hw/intc/apic.c b/hw/intc/apic.c
index 2d79a9e..756dff0 100644
--- a/hw/intc/apic.c
+++ b/hw/intc/apic.c
@@ -874,7 +874,7 @@ static const MemoryRegionOps apic_io_ops = {
 static void apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &apic_io_ops, s, "apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
     s->timer = qemu_new_timer_ns(vm_clock, apic_timer, s);
     local_apics[s->idx] = s;
diff --git a/hw/xen/xen_apic.c b/hw/xen/xen_apic.c
index a2eb8a1..a958782 100644
--- a/hw/xen/xen_apic.c
+++ b/hw/xen/xen_apic.c
@@ -39,7 +39,7 @@ static const MemoryRegionOps xen_apic_io_ops = {
 static void xen_apic_init(APICCommonState *s)
 {
     memory_region_init_io(&s->io_memory, &xen_apic_io_ops, s, "xen-apic-msi",
-                          MSI_SPACE_SIZE);
+                          APIC_SPACE_SIZE);
 
 #if defined(CONFIG_XEN_CTRL_INTERFACE_VERSION) \
     && CONFIG_XEN_CTRL_INTERFACE_VERSION >= 420
diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index 578241f..aac6290 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -66,8 +66,6 @@
 
 #define MAX_APICS 255
 
-#define MSI_SPACE_SIZE                  0x100000
-
 typedef struct APICCommonState APICCommonState;
 
 #define TYPE_APIC_COMMON "apic-common"
diff --git a/target-i386/cpu.h b/target-i386/cpu.h
index cf1b05c..65cf97d 100644
--- a/target-i386/cpu.h
+++ b/target-i386/cpu.h
@@ -1269,5 +1269,6 @@ uint32_t x86_cpu_apic_id_from_index(unsigned int cpu_index);
 void enable_compat_apic_id_mode(void);
 
 #define APIC_DEFAULT_ADDRESS 0xfee00000
+#define APIC_SPACE_SIZE      0x100000
 
 #endif /* CPU_I386_H */
-- 
1.8.2

[Qemu-devel] [PATCH 14/16] target-i386: move APIC to ICC bus

[Igor Mammedov]

... to allow it to be hotplugged

 * map APIC's mmio at board level if it is present
 * do not register mmio region for each APIC, since
   only one is used/mapped

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 hw/cpu/icc_bus.c                | 15 ++++++++++++++-
 hw/i386/kvmvapic.c              |  1 +
 hw/i386/pc.c                    | 20 +++++++++++++++++---
 hw/intc/apic_common.c           | 17 ++++++++++++-----
 include/hw/i386/apic_internal.h |  6 +++---
 include/hw/i386/icc_bus.h       |  2 ++
 target-i386/cpu.c               | 16 +++-------------
 7 files changed, 52 insertions(+), 25 deletions(-)

diff --git a/hw/cpu/icc_bus.c b/hw/cpu/icc_bus.c
index 00b9be3..5078c38 100644
--- a/hw/cpu/icc_bus.c
+++ b/hw/cpu/icc_bus.c
@@ -62,13 +62,26 @@ static const TypeInfo icc_device_info = {
 
 typedef struct ICCBridgeState {
     SysBusDevice busdev;
+    MemoryRegion apic_container;
 } ICCBridgeState;
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
 
 
 static void icc_bridge_initfn(Object *obj)
 {
-    qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus");
+    ICCBridgeState *s = ICC_BRIGDE(obj);
+    SysBusDevice *sb = SYS_BUS_DEVICE(obj);
+    ICCBus *ibus;
+
+    ibus = ICC_BUS(qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus"));
+
+    /* Do not change order of registering regions,
+     * APIC must be first registered region, board maps it by 0 index
+     */
+    memory_region_init(&s->apic_container, "icc-apic-container",
+                       APIC_SPACE_SIZE);
+    sysbus_init_mmio(sb, &s->apic_container);
+    ibus->apic_address_space = &s->apic_container;
 }
 
 static const TypeInfo icc_bridge_info = {
diff --git a/hw/i386/kvmvapic.c b/hw/i386/kvmvapic.c
index 3a10c07..5b558aa 100644
--- a/hw/i386/kvmvapic.c
+++ b/hw/i386/kvmvapic.c
@@ -12,6 +12,7 @@
 #include "sysemu/cpus.h"
 #include "sysemu/kvm.h"
 #include "hw/i386/apic_internal.h"
+#include "hw/sysbus.h"
 
 #define VAPIC_IO_PORT           0x7e
 
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index cb57878..31b5294 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -52,6 +52,7 @@
 #include "sysemu/arch_init.h"
 #include "qemu/bitmap.h"
 #include "qemu/config-file.h"
+#include "hw/i386/icc_bus.h"
 
 /* debug PC/ISA interrupts */
 //#define DEBUG_IRQ
@@ -889,13 +890,13 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
     }
 }
 
-static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
+static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
 {
     X86CPU *cpu;
 
     cpu = cpu_x86_create(cpu_model, errp);
     if (!cpu) {
-        return;
+        return cpu;
     }
 
     object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
@@ -904,14 +905,18 @@ static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
     if (error_is_set(errp)) {
         if (cpu != NULL) {
             object_unref(OBJECT(cpu));
+            cpu = NULL;
         }
     }
+    return cpu;
 }
 
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
+    X86CPU *cpu = NULL;
     Error *error = NULL;
+    SysBusDevice *ib;
 
     /* init CPUs */
     if (cpu_model == NULL) {
@@ -922,14 +927,23 @@ void pc_cpus_init(const char *cpu_model)
 #endif
     }
 
+    ib = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge",
+                                                 TYPE_ICC_BRIDGE, NULL));
+
     for (i = 0; i < smp_cpus; i++) {
-        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
+        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
         if (error) {
             fprintf(stderr, "%s\n", error_get_pretty(error));
             error_free(error);
             exit(1);
         }
     }
+
+    /* map APIC MMIO area if CPU has APIC */
+    if (cpu && cpu->env.apic_state) {
+        /* XXX: what if the base changes? */
+        sysbus_mmio_map_overlap(ib, 0, APIC_DEFAULT_ADDRESS, 0x1000);
+    }
 }
 
 void pc_acpi_init(const char *default_dsdt)
diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
index e0ae07a..39396f1 100644
--- a/hw/intc/apic_common.c
+++ b/hw/intc/apic_common.c
@@ -21,6 +21,8 @@
 #include "hw/i386/apic_internal.h"
 #include "trace.h"
 #include "sysemu/kvm.h"
+#include "hw/qdev.h"
+#include "hw/sysbus.h"
 
 static int apic_irq_delivered;
 bool apic_report_tpr_access;
@@ -282,12 +284,14 @@ static int apic_load_old(QEMUFile *f, void *opaque, int version_id)
     return 0;
 }
 
-static int apic_init_common(SysBusDevice *dev)
+static int apic_init_common(ICCDevice *dev)
 {
     APICCommonState *s = APIC_COMMON(dev);
+    DeviceState *d = DEVICE(dev);
     APICCommonClass *info;
     static DeviceState *vapic;
     static int apic_no;
+    static bool mmio_registered;
 
     if (apic_no >= MAX_APICS) {
         return -1;
@@ -296,8 +300,11 @@ static int apic_init_common(SysBusDevice *dev)
 
     info = APIC_COMMON_GET_CLASS(s);
     info->init(s);
-
-    sysbus_init_mmio(dev, &s->io_memory);
+    if (!mmio_registered) {
+        MemoryRegion *as = ICC_BUS(d->parent_bus)->apic_address_space;
+        memory_region_add_subregion(as, 0, &s->io_memory);
+        mmio_registered = true;
+    }
 
     /* Note: We need at least 1M to map the VAPIC option ROM */
     if (!vapic && s->vapic_control & VAPIC_ENABLE_MASK &&
@@ -375,7 +382,7 @@ static Property apic_properties_common[] = {
 
 static void apic_common_class_init(ObjectClass *klass, void *data)
 {
-    SysBusDeviceClass *sc = SYS_BUS_DEVICE_CLASS(klass);
+    ICCDeviceClass *sc = ICC_DEVICE_CLASS(klass);
     DeviceClass *dc = DEVICE_CLASS(klass);
 
     dc->vmsd = &vmstate_apic_common;
@@ -387,7 +394,7 @@ static void apic_common_class_init(ObjectClass *klass, void *data)
 
 static const TypeInfo apic_common_type = {
     .name = TYPE_APIC_COMMON,
-    .parent = TYPE_SYS_BUS_DEVICE,
+    .parent = TYPE_ICC_DEVICE,
     .instance_size = sizeof(APICCommonState),
     .class_size = sizeof(APICCommonClass),
     .class_init = apic_common_class_init,
diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index aac6290..05acf4b 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -21,7 +21,7 @@
 #define QEMU_APIC_INTERNAL_H
 
 #include "exec/memory.h"
-#include "hw/sysbus.h"
+#include "hw/i386/icc_bus.h"
 #include "qemu/timer.h"
 
 /* APIC Local Vector Table */
@@ -78,7 +78,7 @@ typedef struct APICCommonState APICCommonState;
 
 typedef struct APICCommonClass
 {
-    SysBusDeviceClass parent_class;
+    ICCDeviceClass parent_class;
 
     void (*init)(APICCommonState *s);
     void (*set_base)(APICCommonState *s, uint64_t val);
@@ -92,7 +92,7 @@ typedef struct APICCommonClass
 } APICCommonClass;
 
 struct APICCommonState {
-    SysBusDevice busdev;
+    ICCDevice busdev;
 
     MemoryRegion io_memory;
     X86CPU *cpu;
diff --git a/include/hw/i386/icc_bus.h b/include/hw/i386/icc_bus.h
index aa30e0c..1b07619 100644
--- a/include/hw/i386/icc_bus.h
+++ b/include/hw/i386/icc_bus.h
@@ -22,6 +22,7 @@
 #ifndef ICC_BUS_H
 #define ICC_BUS_H
 
+#include "exec/memory.h"
 #include "hw/qdev-core.h"
 
 #define TYPE_ICC_BUS "icc-bus"
@@ -29,6 +30,7 @@
 #ifndef CONFIG_USER_ONLY
 typedef struct ICCBus {
     BusState qbus;
+    MemoryRegion *apic_address_space;
 } ICCBus;
 #define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
 
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 3b5f90b..bdac6ea 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -41,10 +41,10 @@
 #endif
 
 #include "sysemu/sysemu.h"
+#include "hw/qdev-properties.h"
 #include "hw/i386/icc_bus.h"
 #ifndef CONFIG_USER_ONLY
 #include "hw/xen/xen.h"
-#include "hw/sysbus.h"
 #include "hw/i386/apic_internal.h"
 #endif
 
@@ -2111,6 +2111,7 @@ static void mce_init(X86CPU *cpu)
 static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
 {
     CPUX86State *env = &cpu->env;
+    DeviceState *dev = DEVICE(cpu);
     APICCommonState *apic;
     const char *apic_type = "apic";
 
@@ -2120,7 +2121,7 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
         apic_type = "xen-apic";
     }
 
-    env->apic_state = qdev_try_create(NULL, apic_type);
+    env->apic_state = qdev_try_create(dev->parent_bus, apic_type);
     if (env->apic_state == NULL) {
         error_setg(errp, "APIC device '%s' could not be created", apic_type);
         return;
@@ -2137,7 +2138,6 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
 static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
 {
     CPUX86State *env = &cpu->env;
-    static int apic_mapped;
 
     if (env->apic_state == NULL) {
         return;
@@ -2148,16 +2148,6 @@ static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
                    object_get_typename(OBJECT(env->apic_state)));
         return;
     }
-
-    /* XXX: mapping more APICs at the same memory location */
-    if (apic_mapped == 0) {
-        /* NOTE: the APIC is directly connected to the CPU - it is not
-           on the global memory bus. */
-        /* XXX: what if the base changes? */
-        sysbus_mmio_map_overlap(SYS_BUS_DEVICE(env->apic_state), 0,
-                                APIC_DEFAULT_ADDRESS, 0x1000);
-        apic_mapped = 1;
-    }
 }
 #else
 static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
-- 
1.8.2

[Qemu-devel] [PATCH 15/16] target-i386: move IOAPIC to ICC bus

[Igor Mammedov]

 * inherit IOAPICs from ICCDevice and attach them to ICC bus
 * map IOAPIC's mmio at board level via indirect icc-bridge
   mmio region that provides address space to IOAPIC via
   icc-bus

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 * do not create IOAPIC in icc-bridge and do not make it QOM child
   of icc-bridge, just attach it to icc-bus.
---
 hw/cpu/icc_bus.c                  |  6 ++++++
 hw/i386/kvm/ioapic.c              |  2 +-
 hw/i386/pc.c                      | 10 ++++++----
 hw/intc/ioapic_common.c           | 15 +++++++++++----
 include/exec/memory.h             | 10 ++++++++++
 include/hw/i386/icc_bus.h         |  1 +
 include/hw/i386/ioapic_internal.h |  6 +++---
 memory.c                          | 11 +++++++++++
 8 files changed, 49 insertions(+), 12 deletions(-)

diff --git a/hw/cpu/icc_bus.c b/hw/cpu/icc_bus.c
index 5078c38..d241438 100644
--- a/hw/cpu/icc_bus.c
+++ b/hw/cpu/icc_bus.c
@@ -63,6 +63,7 @@ static const TypeInfo icc_device_info = {
 typedef struct ICCBridgeState {
     SysBusDevice busdev;
     MemoryRegion apic_container;
+    MemoryRegion ioapic_container;
 } ICCBridgeState;
 #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
 
@@ -82,6 +83,11 @@ static void icc_bridge_initfn(Object *obj)
                        APIC_SPACE_SIZE);
     sysbus_init_mmio(sb, &s->apic_container);
     ibus->apic_address_space = &s->apic_container;
+
+    /* must be second registered region, board maps it by 1 index */
+    memory_region_init(&s->ioapic_container, "icc-ioapic-container", 0x1000);
+    sysbus_init_mmio(sb, &s->ioapic_container);
+    ibus->ioapic_address_space = &s->ioapic_container;
 }
 
 static const TypeInfo icc_bridge_info = {
diff --git a/hw/i386/kvm/ioapic.c b/hw/i386/kvm/ioapic.c
index a3bd519..b80d41a 100644
--- a/hw/i386/kvm/ioapic.c
+++ b/hw/i386/kvm/ioapic.c
@@ -96,7 +96,7 @@ static void kvm_ioapic_put(IOAPICCommonState *s)
 
     kioapic->id = s->id;
     kioapic->ioregsel = s->ioregsel;
-    kioapic->base_address = s->busdev.mmio[0].addr;
+    kioapic->base_address = memory_region_get_address(&s->io_memory);
     kioapic->irr = s->irr;
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         kioapic->redirtbl[i].bits = s->ioredtbl[i];
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 31b5294..a5c19a2 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1217,20 +1217,22 @@ void ioapic_init_gsi(GSIState *gsi_state, const char *parent_name)
     DeviceState *dev;
     SysBusDevice *d;
     unsigned int i;
+    BusState *b = BUS(object_resolve_path_type("icc-bus", TYPE_ICC_BUS, NULL));
 
     if (kvm_irqchip_in_kernel()) {
-        dev = qdev_create(NULL, "kvm-ioapic");
+        dev = qdev_create(b, "kvm-ioapic");
     } else {
-        dev = qdev_create(NULL, "ioapic");
+        dev = qdev_create(b, "ioapic");
     }
     if (parent_name) {
         object_property_add_child(object_resolve_path(parent_name, NULL),
                                   "ioapic", OBJECT(dev), NULL);
     }
     qdev_init_nofail(dev);
-    d = SYS_BUS_DEVICE(dev);
-    sysbus_mmio_map(d, 0, IO_APIC_DEFAULT_ADDRESS);
 
+    d = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge", TYPE_ICC_BRIDGE,
+                                                NULL));
+    sysbus_mmio_map(d, 1, IO_APIC_DEFAULT_ADDRESS);
     for (i = 0; i < IOAPIC_NUM_PINS; i++) {
         gsi_state->ioapic_irq[i] = qdev_get_gpio_in(dev, i);
     }
diff --git a/hw/intc/ioapic_common.c b/hw/intc/ioapic_common.c
index 5c5bb3c..72c48a8 100644
--- a/hw/intc/ioapic_common.c
+++ b/hw/intc/ioapic_common.c
@@ -57,11 +57,13 @@ static int ioapic_dispatch_post_load(void *opaque, int version_id)
     return 0;
 }
 
-static int ioapic_init_common(SysBusDevice *dev)
+static int ioapic_init_common(ICCDevice *dev)
 {
     IOAPICCommonState *s = IOAPIC_COMMON(dev);
+    DeviceState *d = DEVICE(dev);
     IOAPICCommonClass *info;
     static int ioapic_no;
+    static bool mmio_registered;
 
     if (ioapic_no >= MAX_IOAPICS) {
         return -1;
@@ -70,7 +72,12 @@ static int ioapic_init_common(SysBusDevice *dev)
     info = IOAPIC_COMMON_GET_CLASS(s);
     info->init(s, ioapic_no);
 
-    sysbus_init_mmio(&s->busdev, &s->io_memory);
+    if (!mmio_registered) {
+        MemoryRegion *as = ICC_BUS(d->parent_bus)->ioapic_address_space;
+        memory_region_add_subregion(as, 0, &s->io_memory);
+        mmio_registered = true;
+    }
+
     ioapic_no++;
 
     return 0;
@@ -95,7 +102,7 @@ static const VMStateDescription vmstate_ioapic_common = {
 
 static void ioapic_common_class_init(ObjectClass *klass, void *data)
 {
-    SysBusDeviceClass *sc = SYS_BUS_DEVICE_CLASS(klass);
+    ICCDeviceClass *sc = ICC_DEVICE_CLASS(klass);
     DeviceClass *dc = DEVICE_CLASS(klass);
 
     sc->init = ioapic_init_common;
@@ -105,7 +112,7 @@ static void ioapic_common_class_init(ObjectClass *klass, void *data)
 
 static const TypeInfo ioapic_common_type = {
     .name = TYPE_IOAPIC_COMMON,
-    .parent = TYPE_SYS_BUS_DEVICE,
+    .parent = TYPE_ICC_DEVICE,
     .instance_size = sizeof(IOAPICCommonState),
     .class_size = sizeof(IOAPICCommonClass),
     .class_init = ioapic_common_class_init,
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 2322732..826e401 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -711,6 +711,16 @@ void memory_region_set_enabled(MemoryRegion *mr, bool enabled);
 void memory_region_set_address(MemoryRegion *mr, hwaddr addr);
 
 /*
+ * memory_region_get_address: get current the address of a region
+ *
+ * Returns the absolute address of a region.
+ * May be used on regions that are currently part of a memory hierarchy.
+ *
+ * @mr: the region being queried
+ */
+hwaddr memory_region_get_address(MemoryRegion *mr);
+
+/*
  * memory_region_set_alias_offset: dynamically update a memory alias's offset
  *
  * Dynamically updates the offset into the target region that an alias points
diff --git a/include/hw/i386/icc_bus.h b/include/hw/i386/icc_bus.h
index 1b07619..d92c6ab 100644
--- a/include/hw/i386/icc_bus.h
+++ b/include/hw/i386/icc_bus.h
@@ -31,6 +31,7 @@
 typedef struct ICCBus {
     BusState qbus;
     MemoryRegion *apic_address_space;
+    MemoryRegion *ioapic_address_space;
 } ICCBus;
 #define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
 
diff --git a/include/hw/i386/ioapic_internal.h b/include/hw/i386/ioapic_internal.h
index 25576c8..4768c5b 100644
--- a/include/hw/i386/ioapic_internal.h
+++ b/include/hw/i386/ioapic_internal.h
@@ -24,7 +24,7 @@
 
 #include "hw/hw.h"
 #include "exec/memory.h"
-#include "hw/sysbus.h"
+#include "hw/i386/icc_bus.h"
 
 #define MAX_IOAPICS                     1
 
@@ -82,14 +82,14 @@ typedef struct IOAPICCommonState IOAPICCommonState;
      OBJECT_GET_CLASS(IOAPICCommonClass, (obj), TYPE_IOAPIC_COMMON)
 
 typedef struct IOAPICCommonClass {
-    SysBusDeviceClass parent_class;
+    ICCDeviceClass parent_class;
     void (*init)(IOAPICCommonState *s, int instance_no);
     void (*pre_save)(IOAPICCommonState *s);
     void (*post_load)(IOAPICCommonState *s);
 } IOAPICCommonClass;
 
 struct IOAPICCommonState {
-    SysBusDevice busdev;
+    ICCDevice busdev;
     MemoryRegion io_memory;
     uint8_t id;
     uint8_t ioregsel;
diff --git a/memory.c b/memory.c
index 75ca281..0651050 100644
--- a/memory.c
+++ b/memory.c
@@ -1413,6 +1413,17 @@ void memory_region_set_address(MemoryRegion *mr, hwaddr addr)
     memory_region_transaction_commit();
 }
 
+hwaddr memory_region_get_address(MemoryRegion *mr)
+{
+    hwaddr addr = mr->addr;
+
+    while (mr->parent) {
+        mr = mr->parent;
+        addr += mr->addr;
+    }
+    return addr;
+}
+
 void memory_region_set_alias_offset(MemoryRegion *mr, hwaddr offset)
 {
     assert(mr->alias);
-- 
1.8.2

[Qemu-devel] [PATCH 16/16] add cpu-add qmp command and implement CPU hot-add for target-i386

[Igor Mammedov]

... via current_machine->cpu_hot_add() hook called by cpu-set QMP command,
for x86 target.

cpu-add's "id" argument is a CPU thread number in a range [0..max-cpus - 1)

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v5:
  * accept id=[0..max_cpus) range in cpu-add command
v4:
  * merge "qmp: add cpu-add qmp command" & "target-i386: implement CPU hot-add" patches
  * move notifier call to CPUCLass.realize()
  * add hook cpu_hot_add to QEMUMachine
  * make QEMUMachineInitArgs global and keep default cpu_model there

v3:
  * it appears that 'online/offline' in cpu-set are confusing people
    with what command actually does and users might have to distinguish
    if 'offline' is not implemented by parsing error message. To simplify
    things replace cpu-set with cpu-add command to show more clear what
    command does and just add cpu-del when CPU remove is implemented.

v2:
  * s/cpu_set/cpu-set/
  * qmp doc style fix
  * use bool type instead of opencodding online/offline string
     suggested-by: Eric Blake <eblake@redhat.com>
---
 hw/i386/pc.c        | 22 ++++++++++++++++++++++
 include/hw/boards.h |  3 +++
 qapi-schema.json    | 11 +++++++++++
 qmp-commands.hx     | 23 +++++++++++++++++++++++
 qmp.c               | 10 ++++++++++
 vl.c                |  6 +++++-
 6 files changed, 74 insertions(+), 1 deletion(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index a5c19a2..f09e44b 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -53,6 +53,7 @@
 #include "qemu/bitmap.h"
 #include "qemu/config-file.h"
 #include "hw/i386/icc_bus.h"
+#include "hw/boards.h"
 
 /* debug PC/ISA interrupts */
 //#define DEBUG_IRQ
@@ -911,6 +912,25 @@ static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
     return cpu;
 }
 
+static void do_cpu_hot_add(const int64_t id, Error **errp)
+{
+    int64_t apic_id = x86_cpu_apic_id_from_index(id);
+
+    if (cpu_exists(apic_id)) {
+        error_setg(errp, "Unable to add CPU: %" PRIi64
+                   ", it already exists", id);
+        return;
+    }
+
+    if (id >= max_cpus) {
+        error_setg(errp, "Unable to add CPU: %" PRIi64
+                   ", max allowed: %d", id, max_cpus - 1);
+        return;
+    }
+
+    pc_new_cpu(machine_args->cpu_model, apic_id, errp);
+}
+
 void pc_cpus_init(const char *cpu_model)
 {
     int i;
@@ -925,7 +945,9 @@ void pc_cpus_init(const char *cpu_model)
 #else
         cpu_model = "qemu32";
 #endif
+        machine_args->cpu_model = cpu_model;
     }
+    current_machine->hot_add_cpu = do_cpu_hot_add;
 
     ib = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge",
                                                  TYPE_ICC_BRIDGE, NULL));
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 425bdc7..de8f92a 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -18,6 +18,8 @@ typedef struct QEMUMachineInitArgs {
     const char *cpu_model;
 } QEMUMachineInitArgs;
 
+extern QEMUMachineInitArgs *machine_args;
+
 typedef void QEMUMachineInitFunc(QEMUMachineInitArgs *args);
 
 typedef void QEMUMachineResetFunc(void);
@@ -43,6 +45,7 @@ typedef struct QEMUMachine {
     GlobalProperty *compat_props;
     struct QEMUMachine *next;
     const char *hw_version;
+    void (*hot_add_cpu)(const int64_t id, Error **errp);
 } QEMUMachine;
 
 int qemu_register_machine(QEMUMachine *m);
diff --git a/qapi-schema.json b/qapi-schema.json
index 751d3c2..34d3e84 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1387,6 +1387,17 @@
 { 'command': 'cpu', 'data': {'index': 'int'} }
 
 ##
+# @cpu-add
+#
+# Adds CPU with specified id
+#
+# @id: cpu id of CPU to be created
+#
+# Returns: Nothing on success
+##
+{ 'command': 'cpu-add', 'data': {'id': 'int'} }
+
+##
 # @memsave:
 #
 # Save a portion of guest memory to a file.
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 4d65422..1e5d299 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -385,6 +385,29 @@ Note: CPUs' indexes are obtained with the 'query-cpus' command.
 EQMP
 
     {
+        .name       = "cpu-add",
+        .args_type  = "id:i",
+        .mhandler.cmd_new = qmp_marshal_input_cpu_add,
+    },
+
+SQMP
+cpu-add
+-------
+
+Adds virtual cpu
+
+Arguments:
+
+- "id": cpu id (json-int)
+
+Example:
+
+-> { "execute": "cpu-add", "arguments": { "id": 2 } }
+<- { "return": {} }
+
+EQMP
+
+    {
         .name       = "memsave",
         .args_type  = "val:l,size:i,filename:s,cpu:i?",
         .mhandler.cmd_new = qmp_marshal_input_memsave,
diff --git a/qmp.c b/qmp.c
index 55b056b..8a9fd9e 100644
--- a/qmp.c
+++ b/qmp.c
@@ -24,6 +24,7 @@
 #include "hw/qdev.h"
 #include "sysemu/blockdev.h"
 #include "qom/qom-qobject.h"
+#include "hw/boards.h"
 
 NameInfo *qmp_query_name(Error **errp)
 {
@@ -108,6 +109,15 @@ void qmp_cpu(int64_t index, Error **errp)
     /* Just do nothing */
 }
 
+void qmp_cpu_add(int64_t id, Error **errp)
+{
+    if (current_machine->hot_add_cpu) {
+        current_machine->hot_add_cpu(id, errp);
+    } else {
+        error_setg(errp, "Not supported");
+    }
+}
+
 #ifndef CONFIG_VNC
 /* If VNC support is enabled, the "true" query-vnc command is
    defined in the VNC subsystem */
diff --git a/vl.c b/vl.c
index bc9c016..8d87d97 100644
--- a/vl.c
+++ b/vl.c
@@ -179,6 +179,8 @@ int main(int argc, char **argv)
 #define MAX_VIRTIO_CONSOLES 1
 #define MAX_SCLP_CONSOLES 1
 
+QEMUMachineInitArgs *machine_args;
+
 static const char *data_dir[16];
 static int data_dir_idx;
 const char *bios_name = NULL;
@@ -4306,13 +4308,15 @@ int main(int argc, char **argv, char **envp)
                                  .kernel_cmdline = kernel_cmdline,
                                  .initrd_filename = initrd_filename,
                                  .cpu_model = cpu_model };
+    machine_args = &args;
+    current_machine = machine;
+
     machine->init(&args);
 
     cpu_synchronize_all_post_init();
 
     set_numa_modes();
 
-    current_machine = machine;
 
     /* init USB devices */
     if (usb_enabled(false)) {
-- 
1.8.2

Re: [Qemu-devel] [PATCH 16/16] add cpu-add qmp command and implement CPU hot-add for target-i386

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1461 bytes --]

On 04/15/2013 04:12 PM, Igor Mammedov wrote:
> ... via current_machine->cpu_hot_add() hook called by cpu-set QMP command,
> for x86 target.
> 
> cpu-add's "id" argument is a CPU thread number in a range [0..max-cpus - 1)

Off by one.  It's either [0..max-cpus) or [0..max-cpus - 1] (they mean
the same thing).

It might be worth including a sample QMP command in the commit message.

> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
> v5:
>   * accept id=[0..max_cpus) range in cpu-add command

This notation is right, unlike the commit message.

Reviewing just the QMP portion:

> +++ b/qapi-schema.json
> @@ -1387,6 +1387,17 @@
>  { 'command': 'cpu', 'data': {'index': 'int'} }
>  
>  ##
> +# @cpu-add
> +#
> +# Adds CPU with specified id
> +#
> +# @id: cpu id of CPU to be created

Here it would be helpful to mention what forms a valid id (your
[0..max-cpus) notation from the commit message, for example).

> +#
> +# Returns: Nothing on success
> +##
> +{ 'command': 'cpu-add', 'data': {'id': 'int'} }
> +

Should be usable from libvirt's perspective, even if hot-plugging more
than one cpu requires more than one QMP call.  Do we have a counterpart
QMP call to easily determine which cpu ids can still be hotplugged?  If
so, should we mention that in the documentation of this command?

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]

Re: [Qemu-devel] [PATCH 00/16 v4] target-i386: CPU hot-add with cpu-add QMP command

[Jan Kiszka]

On 2013-04-16 00:12, Igor Mammedov wrote:
> Implements alternative way for hot-adding CPU using cpu-add QMP command,
> wich could be useful until it would be possible to add CPUs via device_add.

Didn't track the full story: What prevents currently a device_add
approach? And that so effectively that we have to create a
to-be-deprecated-again QMP API first?

Thanks,
Jan

> 
> All patches except the last are also applicable to device_add aprroach.
> 
> To hot-add CPU use following command from qmp-shell:
>  cpu-add id=[0..max-cpus - 1)
> 
> git tree for testing: https://github.com/imammedo/qemu/tree/cpu_add.v4
> 
> based on qom-cpu tree
> 
> v4->v3:
>   * 'id' in cpu-add command will be a thread number instead of APIC ID
>   * split off resume_vcpu() into separate patch
>   * move notifier from rtc code into pc.c
> 
> v2->v3:
>   * use local error & propagate_error() instead of operating on
>     passed in errp in several places
>   * replace CPUClass.get_firmware_id() with CPUClass.get_arch_id()
>   * leave IOAPIC creation to board and just set bus to icc-bus
>   * include kvm-stub.o in cpu libary if no KVM is configured
>   * create resume_vcpu() stub and include it in libqemustub,
>     and use it directly instead of CPU method
>   * acpi_piix4: s/cpu_add_notifier/cpu_added_notifier/
> 
> v1->v2:
>   * generalize cpu sync to KVM, resume and hot-plug notification and
>     invoke them form CPUClass, to make available to all targets.
>   * introduce cpu_exists() and CPUClass.get_firmware_id() and use
>     the last one in acpi_piix to make code target independent.
>   * move IOAPIC to ICC bus, it was suggested and easy to convert.
>   * leave kvmvapic as SysBusDevice, it doesn't affect hot-plug and
>     created only once for all APIC instances. I haven't found yet
>     good/clean enough way to convert it to ICCDevice. May be follow-up
>     though.
>   * split one big ICC patch into several, one per converted device
>   * add cpu_hot_add hook to machine and implement it for target-i386,
>     instead of adding stabs. Could be used by other targets to
>     implement cpu-add.
>   * pre-allocate links<CPU> for all possible CPUs and make them available
>     at /machine/icc-bridge/cpu[0..N] QOM path, so users could find out
>     possible/free CPU IDs to use in cpu-add command.
> 
> CC: pbonzini@redhat.com
> CC: afaerber@suse.de
> CC: ehabkost@redhat.com
> 
> Igor Mammedov (16):
>   cpu: make kvm-stub.o a part of CPU library
>   cpu: call cpu_synchronize_post_init() from CPUClass.realize() if
>     hotplugged
>   introduce resume_vcpu(), for single CPU
>   cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is
>     hot-plugged
>   introduce CPU hot-plug notifier
>   target-i386: pc: update rtc_cmos on CPU hot-plug
>   cpu: introduce get_arch_id() method and override it for target-i386
>   cpu: add helper cpu_exists(), to check if CPU with specified id exists
>   acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest
>   target-i386: introduce apic-id property
>   introduce ICC bus/device/bridge
>   target-i386: cpu: attach ICC bus to CPU on its creation
>   target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE
>   target-i386: move APIC to ICC bus
>   target-i386: move IOAPIC to ICC bus
>   add cpu-add qmp command and implement CPU hot-add for target-i386
> 
>  Makefile.target                    |  14 ++---
>  cpus.c                             |  11 +++-
>  default-configs/i386-softmmu.mak   |   1 +
>  default-configs/x86_64-softmmu.mak |   1 +
>  hw/acpi/piix4.c                    | 114 ++++++++++++++++++++++++++++++++++++-
>  hw/cpu/Makefile.objs               |   1 +
>  hw/cpu/icc_bus.c                   | 104 +++++++++++++++++++++++++++++++++
>  hw/i386/kvm/apic.c                 |   2 +-
>  hw/i386/kvm/ioapic.c               |   2 +-
>  hw/i386/kvmvapic.c                 |   1 +
>  hw/i386/pc.c                       |  91 +++++++++++++++++++++++++++--
>  hw/i386/pc_piix.c                  |   7 +++
>  hw/i386/pc_q35.c                   |   7 +++
>  hw/intc/apic.c                     |   2 +-
>  hw/intc/apic_common.c              |  17 ++++--
>  hw/intc/ioapic_common.c            |  15 +++--
>  hw/timer/mc146818rtc.c             |   7 +++
>  hw/xen/xen_apic.c                  |   2 +-
>  include/exec/memory.h              |  10 ++++
>  include/hw/boards.h                |   3 +
>  include/hw/i386/apic_internal.h    |   8 +--
>  include/hw/i386/icc_bus.h          |  53 +++++++++++++++++
>  include/hw/i386/ioapic_internal.h  |   6 +-
>  include/hw/pci/msi.h               |   2 +
>  include/hw/timer/mc146818rtc.h     |   1 +
>  include/qom/cpu.h                  |  21 +++++++
>  include/sysemu/kvm.h               |  22 +++----
>  include/sysemu/sysemu.h            |   3 +
>  kvm-all.c                          |   1 +
>  kvm-stub.c                         |   3 +
>  memory.c                           |  11 ++++
>  qapi-schema.json                   |  11 ++++
>  qmp-commands.hx                    |  23 ++++++++
>  qmp.c                              |  10 ++++
>  qom/cpu.c                          |  44 ++++++++++++++
>  stubs/Makefile.objs                |   1 +
>  stubs/cpus.c                       |   5 ++
>  target-i386/cpu.c                  |  75 +++++++++++++++++++-----
>  target-i386/cpu.h                  |   1 +
>  vl.c                               |   7 ++-
>  40 files changed, 656 insertions(+), 64 deletions(-)
>  create mode 100644 hw/cpu/icc_bus.c
>  create mode 100644 include/hw/i386/icc_bus.h
>  create mode 100644 stubs/cpus.c
> 

-- 
Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
Corporate Competence Center Embedded Linux

Re: [Qemu-devel] [PATCH 00/16 v4] target-i386: CPU hot-add with cpu-add QMP command

[Igor Mammedov]

On Tue, 16 Apr 2013 10:30:55 +0200
Jan Kiszka <jan.kiszka@siemens.com> wrote:

> On 2013-04-16 00:12, Igor Mammedov wrote:
> > Implements alternative way for hot-adding CPU using cpu-add QMP command,
> > wich could be useful until it would be possible to add CPUs via device_add.
> 
> Didn't track the full story: What prevents currently a device_add
> approach? And that so effectively that we have to create a
> to-be-deprecated-again QMP API first?
I guess mostly it's review bandwidth and time concerns, it moves but quite slow
and won't be able to make into 1.5 and possibly miss 1.6 with current speed.

In this series all patches except the last one applicable to device_add as
well so any feedback is appreciated.

To make device_add usable for CPU, I have on my TODO list following
series/topics:

1. convert cpuid features to static properties.
http://lists.gnu.org/archive/html/qemu-devel/2013-02/msg04426.html

2. x86 CPU subclasses
http://lists.gnu.org/archive/html/qemu-devel/2013-02/msg00673.html

Summarized issues/ways to go with CPU subclasses could be found here:
http://wiki.qemu.org/Features/CPUHotplug section "CPU models as CPU subclasses"

[optional]
3. nice to have, unify cpu_model handling and convert it to utilizing global
properties. With it hotplug could look as simple as:
 device_add apic_id=xxx
without specifying all flags that were on -cpu command line.


BTW:
 You were advocating using APIC ID on device_add to identify CPU but Eduardo
 would like avoid its usage on external interfaces.
 It would be nice to have your opinion on subject.
  relevant discussion threads are here:
    1. http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg02205.html
    2. http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg02274.html

> Thanks,
> Jan
> 
> > 
> > All patches except the last are also applicable to device_add aprroach.
> > 
> > To hot-add CPU use following command from qmp-shell:
> >  cpu-add id=[0..max-cpus - 1)
> > 
> > git tree for testing: https://github.com/imammedo/qemu/tree/cpu_add.v4
> > 
> > based on qom-cpu tree
> > 
> > v4->v3:
> >   * 'id' in cpu-add command will be a thread number instead of APIC ID
> >   * split off resume_vcpu() into separate patch
> >   * move notifier from rtc code into pc.c
> > 
> > v2->v3:
> >   * use local error & propagate_error() instead of operating on
> >     passed in errp in several places
> >   * replace CPUClass.get_firmware_id() with CPUClass.get_arch_id()
> >   * leave IOAPIC creation to board and just set bus to icc-bus
> >   * include kvm-stub.o in cpu libary if no KVM is configured
> >   * create resume_vcpu() stub and include it in libqemustub,
> >     and use it directly instead of CPU method
> >   * acpi_piix4: s/cpu_add_notifier/cpu_added_notifier/
> > 
> > v1->v2:
> >   * generalize cpu sync to KVM, resume and hot-plug notification and
> >     invoke them form CPUClass, to make available to all targets.
> >   * introduce cpu_exists() and CPUClass.get_firmware_id() and use
> >     the last one in acpi_piix to make code target independent.
> >   * move IOAPIC to ICC bus, it was suggested and easy to convert.
> >   * leave kvmvapic as SysBusDevice, it doesn't affect hot-plug and
> >     created only once for all APIC instances. I haven't found yet
> >     good/clean enough way to convert it to ICCDevice. May be follow-up
> >     though.
> >   * split one big ICC patch into several, one per converted device
> >   * add cpu_hot_add hook to machine and implement it for target-i386,
> >     instead of adding stabs. Could be used by other targets to
> >     implement cpu-add.
> >   * pre-allocate links<CPU> for all possible CPUs and make them available
> >     at /machine/icc-bridge/cpu[0..N] QOM path, so users could find out
> >     possible/free CPU IDs to use in cpu-add command.
> > 
> > CC: pbonzini@redhat.com
> > CC: afaerber@suse.de
> > CC: ehabkost@redhat.com
> > 
> > Igor Mammedov (16):
> >   cpu: make kvm-stub.o a part of CPU library
> >   cpu: call cpu_synchronize_post_init() from CPUClass.realize() if
> >     hotplugged
> >   introduce resume_vcpu(), for single CPU
> >   cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is
> >     hot-plugged
> >   introduce CPU hot-plug notifier
> >   target-i386: pc: update rtc_cmos on CPU hot-plug
> >   cpu: introduce get_arch_id() method and override it for target-i386
> >   cpu: add helper cpu_exists(), to check if CPU with specified id exists
> >   acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest
> >   target-i386: introduce apic-id property
> >   introduce ICC bus/device/bridge
> >   target-i386: cpu: attach ICC bus to CPU on its creation
> >   target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE
> >   target-i386: move APIC to ICC bus
> >   target-i386: move IOAPIC to ICC bus
> >   add cpu-add qmp command and implement CPU hot-add for target-i386
> > 
> >  Makefile.target                    |  14 ++---
> >  cpus.c                             |  11 +++-
> >  default-configs/i386-softmmu.mak   |   1 +
> >  default-configs/x86_64-softmmu.mak |   1 +
> >  hw/acpi/piix4.c                    | 114 ++++++++++++++++++++++++++++++++++++-
> >  hw/cpu/Makefile.objs               |   1 +
> >  hw/cpu/icc_bus.c                   | 104 +++++++++++++++++++++++++++++++++
> >  hw/i386/kvm/apic.c                 |   2 +-
> >  hw/i386/kvm/ioapic.c               |   2 +-
> >  hw/i386/kvmvapic.c                 |   1 +
> >  hw/i386/pc.c                       |  91 +++++++++++++++++++++++++++--
> >  hw/i386/pc_piix.c                  |   7 +++
> >  hw/i386/pc_q35.c                   |   7 +++
> >  hw/intc/apic.c                     |   2 +-
> >  hw/intc/apic_common.c              |  17 ++++--
> >  hw/intc/ioapic_common.c            |  15 +++--
> >  hw/timer/mc146818rtc.c             |   7 +++
> >  hw/xen/xen_apic.c                  |   2 +-
> >  include/exec/memory.h              |  10 ++++
> >  include/hw/boards.h                |   3 +
> >  include/hw/i386/apic_internal.h    |   8 +--
> >  include/hw/i386/icc_bus.h          |  53 +++++++++++++++++
> >  include/hw/i386/ioapic_internal.h  |   6 +-
> >  include/hw/pci/msi.h               |   2 +
> >  include/hw/timer/mc146818rtc.h     |   1 +
> >  include/qom/cpu.h                  |  21 +++++++
> >  include/sysemu/kvm.h               |  22 +++----
> >  include/sysemu/sysemu.h            |   3 +
> >  kvm-all.c                          |   1 +
> >  kvm-stub.c                         |   3 +
> >  memory.c                           |  11 ++++
> >  qapi-schema.json                   |  11 ++++
> >  qmp-commands.hx                    |  23 ++++++++
> >  qmp.c                              |  10 ++++
> >  qom/cpu.c                          |  44 ++++++++++++++
> >  stubs/Makefile.objs                |   1 +
> >  stubs/cpus.c                       |   5 ++
> >  target-i386/cpu.c                  |  75 +++++++++++++++++++-----
> >  target-i386/cpu.h                  |   1 +
> >  vl.c                               |   7 ++-
> >  40 files changed, 656 insertions(+), 64 deletions(-)
> >  create mode 100644 hw/cpu/icc_bus.c
> >  create mode 100644 include/hw/i386/icc_bus.h
> >  create mode 100644 stubs/cpus.c
> > 
> 
> -- 
> Siemens AG, Corporate Technology, CT RTC ITP SDP-DE
> Corporate Competence Center Embedded Linux
> 


-- 
Regards,
  Igor

Re: [Qemu-devel] [PATCH 00/16 v4] target-i386: CPU hot-add with cpu-add QMP command

[Eduardo Habkost]

On Tue, Apr 16, 2013 at 11:43:42AM +0200, Igor Mammedov wrote:
[...]
> BTW:
>  You were advocating using APIC ID on device_add to identify CPU but Eduardo
>  would like avoid its usage on external interfaces.
>  It would be nice to have your opinion on subject.
>   relevant discussion threads are here:
>     1. http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg02205.html
>     2. http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg02274.html

Trying to summarize my concerns quickly:

 0) APIC IDs must be calculated by QEMU based on CPU topology and can't
    be chosen arbitrarily.
 1) Requiring APIC ID as the CPU ID requires a target-independent
    probing interface to find out which IDs are valid, and how they
    correspond to locations in the CPU topology. We don't have such
    interface yet.
    * This makes it impossible to use APIC IDs to identify CPUs in the
      command-line (before QMP is available), so we still need another
      kind of identifier (topology-based, probably).
 2) apic-id is not compatible with the existing IDs used for the "-numa"
    command-line option.

So, using APIC ID is possible, but I believe it introduces too much
unnecessary complexity in the external interface. The day we introduce a
full socket/core/thread-based device_add interface, setting APIC ID
directly externally won't be necessary anymore (QEMU could calculate it
based on the object topology), but while we don't have the full-featured
core/socket/thread-based interface we can simply use IDs in the range
[0..(max_cpus-1)], that are already used by the "-numa" command-line
option.

-- 
Eduardo

Re: [Qemu-devel] [PATCH 16/16] add cpu-add qmp command and implement CPU hot-add for target-i386

[Igor Mammedov]

On Mon, 15 Apr 2013 16:20:15 -0600
Eric Blake <eblake@redhat.com> wrote:

> On 04/15/2013 04:12 PM, Igor Mammedov wrote:
> > ... via current_machine->cpu_hot_add() hook called by cpu-set QMP command,
> > for x86 target.
> > 
> > cpu-add's "id" argument is a CPU thread number in a range [0..max-cpus - 1)
> 
> Off by one.  It's either [0..max-cpus) or [0..max-cpus - 1] (they mean
> the same thing).
> 
> It might be worth including a sample QMP command in the commit message.
Ok, will ammend it on next respin.

> 
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > ---
> > v5:
> >   * accept id=[0..max_cpus) range in cpu-add command
> 
> This notation is right, unlike the commit message.
> 
> Reviewing just the QMP portion:
> 
> > +++ b/qapi-schema.json
> > @@ -1387,6 +1387,17 @@
> >  { 'command': 'cpu', 'data': {'index': 'int'} }
> >  
> >  ##
> > +# @cpu-add
> > +#
> > +# Adds CPU with specified id
> > +#
> > +# @id: cpu id of CPU to be created
> 
> Here it would be helpful to mention what forms a valid id (your
> [0..max-cpus) notation from the commit message, for example).
ditto

> 
> > +#
> > +# Returns: Nothing on success
> > +##
> > +{ 'command': 'cpu-add', 'data': {'id': 'int'} }
> > +
> 
> Should be usable from libvirt's perspective, even if hot-plugging more
> than one cpu requires more than one QMP call.  Do we have a counterpart
> QMP call to easily determine which cpu ids can still be hotplugged?  If
> so, should we mention that in the documentation of this command?
We do not have it yet. Despite interface allowing to plug arbitrary CPU,
libvirt shouldn't do it in order not to break migration support. Since
migration target should be started with all CPU from source (including
hot-plugged ones). And current command line doesn't have means for this.

I'd propose do implement in libvirt something like:

hotplug_id = current_cpu_count
{ "execute": "cpu-add", "arguments": { "id": hotplug_id } }
current_cpu_count += 1

until arbitrary CPU hotplug and interface for enumerating possible CPUs
settle down.

There was a proposal for enumerating possible CPUs in previous version,
but it was target specific, and I was convinced to drop it for 1.5 and
aim for generic way to expose this information later.
http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg02286.html
Opinion from libvirt POV would be nice to have though.

> 
> -- 
> Eric Blake   eblake redhat com    +1-919-301-3266
> Libvirt virtualization library http://libvirt.org
> 


-- 
Regards,
  Igor

Re: [Qemu-devel] [PATCH 01/16] cpu: make kvm-stub.o a part of CPU library

[Paolo Bonzini]

> > +#ifndef CONFIG_USER_ONLY
> >  #include "qemu-common.h"
> >  #include "hw/pci/pci.h"
> >  
> > @@ -47,4 +48,5 @@ static inline bool msi_present(const PCIDevice *dev)
> >      return dev->cap_present & QEMU_PCI_CAP_MSI;
> >  }
> >  
> > +#endif

Please put the #ifndef in the .c file instead.

Paolo

> >  #endif /* QEMU_MSI_H */
> > diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
> > index 495e6f8..93cef28 100644
> > --- a/include/sysemu/kvm.h
> > +++ b/include/sysemu/kvm.h
> > @@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
> >  #if !defined(CONFIG_USER_ONLY)
> >  void *kvm_vmalloc(ram_addr_t size);
> >  void *kvm_arch_vmalloc(ram_addr_t size);
> > -void kvm_setup_guest_memory(void *start, size_t size);
> > +#endif
> >  
> > +void kvm_setup_guest_memory(void *start, size_t size);
> >  void kvm_flush_coalesced_mmio_buffer(void);
> > -#endif
> >  
> >  int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
> >                            target_ulong len, int type);
> > diff --git a/kvm-stub.c b/kvm-stub.c
> > index 3e5c201..a131305 100644
> > --- a/kvm-stub.c
> > +++ b/kvm-stub.c
> > @@ -122,6 +122,7 @@ int kvm_on_sigbus(int code, void *addr)
> >      return 1;
> >  }
> >  
> > +#ifndef CONFIG_USER_ONLY
> >  int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
> >  {
> >      return -ENOSYS;
> > @@ -145,3 +146,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s,
> > EventNotifier *n, int virq)
> >  {
> >      return -ENOSYS;
> >  }
> > +#endif
> > --
> > 1.8.2
> > 
> > 
> 
> --
> Eduardo
> 

[Qemu-devel] [PATCH 01/16 v2] cpu: make kvm-stub.o a part of CPU library

[Igor Mammedov]

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v3:
  put ifndef around msi.h in kvm-stub.c instead of in msi.h

v2:
  remove unnecessary kvm-stub.o from "System emulator target" section
---
 Makefile.target      | 14 +++++++-------
 include/sysemu/kvm.h |  4 ++--
 kvm-stub.c           |  7 ++++++-
 3 files changed, 15 insertions(+), 10 deletions(-)

diff --git a/Makefile.target b/Makefile.target
index 2bd6d14..9bde8e5 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -64,6 +64,12 @@ all: $(PROGS) stap
 # Dummy command so that make thinks it has done something
 	@true
 
+CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
+CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
+CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
+CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
+CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
+
 #########################################################
 # cpu emulator library
 obj-y = exec.o translate-all.o cpu-exec.o
@@ -74,6 +80,7 @@ obj-y += fpu/softfloat.o
 obj-y += target-$(TARGET_BASE_ARCH)/
 obj-y += disas.o
 obj-$(CONFIG_GDBSTUB_XML) += gdbstub-xml.o
+obj-$(CONFIG_NO_KVM) += kvm-stub.o
 
 #########################################################
 # Linux user emulator target
@@ -102,18 +109,11 @@ endif #CONFIG_BSD_USER
 #########################################################
 # System emulator target
 ifdef CONFIG_SOFTMMU
-CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
-CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
-CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
-CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
-CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
-
 obj-y += arch_init.o cpus.o monitor.o gdbstub.o balloon.o ioport.o
 obj-y += qtest.o
 obj-y += hw/
 obj-$(CONFIG_FDT) += device_tree.o
 obj-$(CONFIG_KVM) += kvm-all.o
-obj-$(CONFIG_NO_KVM) += kvm-stub.o
 obj-y += memory.o savevm.o cputlb.o
 obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
 obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 495e6f8..93cef28 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
 #if !defined(CONFIG_USER_ONLY)
 void *kvm_vmalloc(ram_addr_t size);
 void *kvm_arch_vmalloc(ram_addr_t size);
-void kvm_setup_guest_memory(void *start, size_t size);
+#endif
 
+void kvm_setup_guest_memory(void *start, size_t size);
 void kvm_flush_coalesced_mmio_buffer(void);
-#endif
 
 int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
                           target_ulong len, int type);
diff --git a/kvm-stub.c b/kvm-stub.c
index 3e5c201..e5d2906 100644
--- a/kvm-stub.c
+++ b/kvm-stub.c
@@ -12,11 +12,14 @@
 
 #include "qemu-common.h"
 #include "hw/hw.h"
-#include "hw/pci/msi.h"
 #include "cpu.h"
 #include "exec/gdbstub.h"
 #include "sysemu/kvm.h"
 
+#ifndef CONFIG_USER_ONLY
+#include "hw/pci/msi.h"
+#endif
+
 KVMState *kvm_state;
 bool kvm_kernel_irqchip;
 bool kvm_async_interrupts_allowed;
@@ -122,6 +125,7 @@ int kvm_on_sigbus(int code, void *addr)
     return 1;
 }
 
+#ifndef CONFIG_USER_ONLY
 int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
 {
     return -ENOSYS;
@@ -145,3 +149,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
 {
     return -ENOSYS;
 }
+#endif
-- 
1.8.2

Re: [Qemu-devel] [PATCH 01/16 v2] cpu: make kvm-stub.o a part of CPU library

[Eduardo Habkost]

On Thu, Apr 18, 2013 at 06:23:50AM +0200, Igor Mammedov wrote:
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>

Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>


> ---
> v3:
>   put ifndef around msi.h in kvm-stub.c instead of in msi.h
> 
> v2:
>   remove unnecessary kvm-stub.o from "System emulator target" section
> ---
>  Makefile.target      | 14 +++++++-------
>  include/sysemu/kvm.h |  4 ++--
>  kvm-stub.c           |  7 ++++++-
>  3 files changed, 15 insertions(+), 10 deletions(-)
> 
> diff --git a/Makefile.target b/Makefile.target
> index 2bd6d14..9bde8e5 100644
> --- a/Makefile.target
> +++ b/Makefile.target
> @@ -64,6 +64,12 @@ all: $(PROGS) stap
>  # Dummy command so that make thinks it has done something
>  	@true
>  
> +CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
> +CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
> +CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
> +CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
> +CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
> +
>  #########################################################
>  # cpu emulator library
>  obj-y = exec.o translate-all.o cpu-exec.o
> @@ -74,6 +80,7 @@ obj-y += fpu/softfloat.o
>  obj-y += target-$(TARGET_BASE_ARCH)/
>  obj-y += disas.o
>  obj-$(CONFIG_GDBSTUB_XML) += gdbstub-xml.o
> +obj-$(CONFIG_NO_KVM) += kvm-stub.o
>  
>  #########################################################
>  # Linux user emulator target
> @@ -102,18 +109,11 @@ endif #CONFIG_BSD_USER
>  #########################################################
>  # System emulator target
>  ifdef CONFIG_SOFTMMU
> -CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
> -CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
> -CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
> -CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
> -CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
> -
>  obj-y += arch_init.o cpus.o monitor.o gdbstub.o balloon.o ioport.o
>  obj-y += qtest.o
>  obj-y += hw/
>  obj-$(CONFIG_FDT) += device_tree.o
>  obj-$(CONFIG_KVM) += kvm-all.o
> -obj-$(CONFIG_NO_KVM) += kvm-stub.o
>  obj-y += memory.o savevm.o cputlb.o
>  obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
>  obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
> diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
> index 495e6f8..93cef28 100644
> --- a/include/sysemu/kvm.h
> +++ b/include/sysemu/kvm.h
> @@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
>  #if !defined(CONFIG_USER_ONLY)
>  void *kvm_vmalloc(ram_addr_t size);
>  void *kvm_arch_vmalloc(ram_addr_t size);
> -void kvm_setup_guest_memory(void *start, size_t size);
> +#endif
>  
> +void kvm_setup_guest_memory(void *start, size_t size);
>  void kvm_flush_coalesced_mmio_buffer(void);
> -#endif
>  
>  int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
>                            target_ulong len, int type);
> diff --git a/kvm-stub.c b/kvm-stub.c
> index 3e5c201..e5d2906 100644
> --- a/kvm-stub.c
> +++ b/kvm-stub.c
> @@ -12,11 +12,14 @@
>  
>  #include "qemu-common.h"
>  #include "hw/hw.h"
> -#include "hw/pci/msi.h"
>  #include "cpu.h"
>  #include "exec/gdbstub.h"
>  #include "sysemu/kvm.h"
>  
> +#ifndef CONFIG_USER_ONLY
> +#include "hw/pci/msi.h"
> +#endif
> +
>  KVMState *kvm_state;
>  bool kvm_kernel_irqchip;
>  bool kvm_async_interrupts_allowed;
> @@ -122,6 +125,7 @@ int kvm_on_sigbus(int code, void *addr)
>      return 1;
>  }
>  
> +#ifndef CONFIG_USER_ONLY
>  int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
>  {
>      return -ENOSYS;
> @@ -145,3 +149,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
>  {
>      return -ENOSYS;
>  }
> +#endif
> -- 
> 1.8.2
> 

-- 
Eduardo

Re: [Qemu-devel] [PATCH 02/16] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

[Eduardo Habkost]

On Tue, Apr 16, 2013 at 12:12:42AM +0200, Igor Mammedov wrote:
> ... to synchronize CPU state to KVM
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>

Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>

> ---
> v2:
>   * linking kvm-stub.o to *-user target moved in separate patch
> ---
>  include/sysemu/kvm.h | 18 ++++++++++--------
>  kvm-all.c            |  1 +
>  kvm-stub.c           |  1 +
>  qom/cpu.c            |  4 ++++
>  vl.c                 |  1 -
>  5 files changed, 16 insertions(+), 9 deletions(-)
> 
> diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
> index 93cef28..8fb2489 100644
> --- a/include/sysemu/kvm.h
> +++ b/include/sysemu/kvm.h
> @@ -250,8 +250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension);
>  uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
>                                        uint32_t index, int reg);
>  void kvm_cpu_synchronize_state(CPUArchState *env);
> -void kvm_cpu_synchronize_post_reset(CPUState *cpu);
> -void kvm_cpu_synchronize_post_init(CPUState *cpu);
>  
>  /* generic hooks - to be moved/refactored once there are more users */
>  
> @@ -262,6 +260,16 @@ static inline void cpu_synchronize_state(CPUArchState *env)
>      }
>  }
>  
> +#if !defined(CONFIG_USER_ONLY)
> +int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
> +                                       hwaddr *phys_addr);
> +#endif
> +
> +#endif /* NEED_CPU_H */
> +
> +void kvm_cpu_synchronize_post_reset(CPUState *cpu);
> +void kvm_cpu_synchronize_post_init(CPUState *cpu);
> +
>  static inline void cpu_synchronize_post_reset(CPUState *cpu)
>  {
>      if (kvm_enabled()) {
> @@ -277,12 +285,6 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
>  }
>  
>  
> -#if !defined(CONFIG_USER_ONLY)
> -int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
> -                                       hwaddr *phys_addr);
> -#endif
> -
> -#endif
>  int kvm_set_ioeventfd_mmio(int fd, uint32_t adr, uint32_t val, bool assign,
>                             uint32_t size);
>  
> diff --git a/kvm-all.c b/kvm-all.c
> index fc4e17c..1d17128 100644
> --- a/kvm-all.c
> +++ b/kvm-all.c
> @@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
>  bool kvm_irqfds_allowed;
>  bool kvm_msi_via_irqfd_allowed;
>  bool kvm_gsi_routing_allowed;
> +bool kvm_allowed;
>  
>  static const KVMCapabilityInfo kvm_required_capabilites[] = {
>      KVM_CAP_INFO(USER_MEMORY),
> diff --git a/kvm-stub.c b/kvm-stub.c
> index a131305..4282e09 100644
> --- a/kvm-stub.c
> +++ b/kvm-stub.c
> @@ -23,6 +23,7 @@ bool kvm_async_interrupts_allowed;
>  bool kvm_irqfds_allowed;
>  bool kvm_msi_via_irqfd_allowed;
>  bool kvm_gsi_routing_allowed;
> +bool kvm_allowed;
>  
>  int kvm_init_vcpu(CPUState *cpu)
>  {
> diff --git a/qom/cpu.c b/qom/cpu.c
> index e242dcb..0c76712 100644
> --- a/qom/cpu.c
> +++ b/qom/cpu.c
> @@ -20,6 +20,7 @@
>  
>  #include "qom/cpu.h"
>  #include "qemu-common.h"
> +#include "sysemu/kvm.h"
>  
>  void cpu_reset_interrupt(CPUState *cpu, int mask)
>  {
> @@ -57,6 +58,9 @@ static ObjectClass *cpu_common_class_by_name(const char *cpu_model)
>  
>  static void cpu_common_realizefn(DeviceState *dev, Error **errp)
>  {
> +    if (dev->hotplugged) {
> +        cpu_synchronize_post_init(CPU(dev));
> +    }
>  }
>  
>  static void cpu_class_init(ObjectClass *klass, void *data)
> diff --git a/vl.c b/vl.c
> index dd94549..bc9c016 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -267,7 +267,6 @@ static NotifierList machine_init_done_notifiers =
>      NOTIFIER_LIST_INITIALIZER(machine_init_done_notifiers);
>  
>  static bool tcg_allowed = true;
> -bool kvm_allowed;
>  bool xen_allowed;
>  uint32_t xen_domid;
>  enum xen_mode xen_mode = XEN_EMULATE;
> -- 
> 1.8.2
> 
> 

-- 
Eduardo

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

[Eduardo Habkost]

On Tue, Apr 16, 2013 at 12:12:43AM +0200, Igor Mammedov wrote:
> ... also add stub for it, to make possible to use it in qom/cpu.c,
> which is shared with user emulators.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>

Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>


> ---
> Note:
>   it will be re-used from qom/cpu.c by hotpluged CPU
> ---
>  cpus.c              | 11 ++++++++---
>  include/qom/cpu.h   |  9 +++++++++
>  stubs/Makefile.objs |  1 +
>  stubs/cpus.c        |  5 +++++
>  4 files changed, 23 insertions(+), 3 deletions(-)
>  create mode 100644 stubs/cpus.c
> 
> diff --git a/cpus.c b/cpus.c
> index c15ff6c..c347db5 100644
> --- a/cpus.c
> +++ b/cpus.c
> @@ -993,6 +993,13 @@ void pause_all_vcpus(void)
>      }
>  }
>  
> +void resume_vcpu(CPUState *cpu)
> +{
> +    cpu->stop = false;
> +    cpu->stopped = false;
> +    qemu_cpu_kick(cpu);
> +}
> +
>  void resume_all_vcpus(void)
>  {
>      CPUArchState *penv = first_cpu;
> @@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
>      qemu_clock_enable(vm_clock, true);
>      while (penv) {
>          CPUState *pcpu = ENV_GET_CPU(penv);
> -        pcpu->stop = false;
> -        pcpu->stopped = false;
> -        qemu_cpu_kick(pcpu);
> +        resume_vcpu(pcpu);
>          penv = penv->next_cpu;
>      }
>  }
> diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> index 3664a1b..6ce6f10 100644
> --- a/include/qom/cpu.h
> +++ b/include/qom/cpu.h
> @@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
>   */
>  void cpu_reset_interrupt(CPUState *cpu, int mask);
>  
> +/**
> + * resume_vcpu:
> + * @cpu: The CPU to resume.
> + *
> + * Resumes CPU, i.e. puts CPU into runable state.
> + * KVM requires extra call cpu_synchronize_post_init() to make
> + * KVM's CPU instance runable.
> + */
> +void resume_vcpu(CPUState *cpu);
>  
>  #endif
> diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs
> index 9c55b34..03dff20 100644
> --- a/stubs/Makefile.objs
> +++ b/stubs/Makefile.objs
> @@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
>  stub-obj-y += vm-stop.o
>  stub-obj-y += vmstate.o
>  stub-obj-$(CONFIG_WIN32) += fd-register.o
> +stub-obj-y += cpus.o
> diff --git a/stubs/cpus.c b/stubs/cpus.c
> new file mode 100644
> index 0000000..3d17451
> --- /dev/null
> +++ b/stubs/cpus.c
> @@ -0,0 +1,5 @@
> +#include "qom/cpu.h"
> +
> +void resume_vcpu(CPUState *cpu)
> +{
> +}
> -- 
> 1.8.2
> 
> 

-- 
Eduardo

Re: [Qemu-devel] [PATCH 04/16] cpu: resume CPU from CPUClass.cpu_common_realizefn() when it is hot-plugged

[Eduardo Habkost]

On Tue, Apr 16, 2013 at 12:12:44AM +0200, Igor Mammedov wrote:
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>

Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>


> ---
> v2:
>  * remove unnecessary now sysemu/cpus.h header
>  * move out introducing resume_vcpu() into a separate patch
> ---
>  qom/cpu.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/qom/cpu.c b/qom/cpu.c
> index 0c76712..40a4259 100644
> --- a/qom/cpu.c
> +++ b/qom/cpu.c
> @@ -60,6 +60,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
>  {
>      if (dev->hotplugged) {
>          cpu_synchronize_post_init(CPU(dev));
> +        resume_vcpu(CPU(dev));
>      }
>  }
>  
> -- 
> 1.8.2
> 
> 

-- 
Eduardo

Re: [Qemu-devel] [PATCH 06/16] target-i386: pc: update rtc_cmos on CPU hot-plug

[Eduardo Habkost]

On Tue, Apr 16, 2013 at 12:12:46AM +0200, Igor Mammedov wrote:
> ... so that on reboot BIOS could read current available CPU count
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>

Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>

> ---
>  hw/i386/pc.c                   | 20 ++++++++++++++++++++
>  hw/timer/mc146818rtc.c         |  7 +++++++
>  include/hw/timer/mc146818rtc.h |  1 +
>  3 files changed, 28 insertions(+)
> 
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 8d75b34..dc1a78b 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -337,6 +337,21 @@ static void pc_cmos_init_late(void *opaque)
>      qemu_unregister_reset(pc_cmos_init_late, opaque);
>  }
>  
> +typedef struct rtc_cpu_hotplug_arg {
> +    ISADevice *rtc_state;
> +    Notifier cpu_added_notifier;
> +} rtc_cpu_hotplug_arg;
> +
> +static void rtc_notify_cpu_added(Notifier *notifier, void *data)
> +{
> +    rtc_cpu_hotplug_arg *arg = container_of(notifier, rtc_cpu_hotplug_arg,
> +                                           cpu_added_notifier);
> +    ISADevice *s = arg->rtc_state;
> +
> +    /* increment the number of CPUs */
> +    rtc_set_memory(s, 0x5f, rtc_get_memory(s, 0x5f) + 1);
> +}
> +
>  void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
>                    const char *boot_device,
>                    ISADevice *floppy, BusState *idebus0, BusState *idebus1,
> @@ -345,6 +360,7 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
>      int val, nb, i;
>      FDriveType fd_type[2] = { FDRIVE_DRV_NONE, FDRIVE_DRV_NONE };
>      static pc_cmos_init_late_arg arg;
> +    static rtc_cpu_hotplug_arg cpu_hotplug_cb;
>  
>      /* various important CMOS locations needed by PC/Bochs bios */
>  
> @@ -383,6 +399,10 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
>  
>      /* set the number of CPU */
>      rtc_set_memory(s, 0x5f, smp_cpus - 1);
> +    /* init CPU hotplug notifier */
> +    cpu_hotplug_cb.rtc_state = s;
> +    cpu_hotplug_cb.cpu_added_notifier.notify = rtc_notify_cpu_added;
> +    qemu_register_cpu_added_notifier(&cpu_hotplug_cb.cpu_added_notifier);
>  
>      /* set boot devices, and disable floppy signature check if requested */
>      if (set_boot_dev(s, boot_device, fd_bootchk)) {
> diff --git a/hw/timer/mc146818rtc.c b/hw/timer/mc146818rtc.c
> index 69e6844..e639942 100644
> --- a/hw/timer/mc146818rtc.c
> +++ b/hw/timer/mc146818rtc.c
> @@ -677,6 +677,13 @@ void rtc_set_memory(ISADevice *dev, int addr, int val)
>          s->cmos_data[addr] = val;
>  }
>  
> +int rtc_get_memory(ISADevice *dev, int addr)
> +{
> +    RTCState *s = DO_UPCAST(RTCState, dev, dev);
> +    assert(addr >= 0 && addr <= 127);
> +    return s->cmos_data[addr];
> +}
> +
>  static void rtc_set_date_from_host(ISADevice *dev)
>  {
>      RTCState *s = DO_UPCAST(RTCState, dev, dev);
> diff --git a/include/hw/timer/mc146818rtc.h b/include/hw/timer/mc146818rtc.h
> index 854ea3f..09f37b7 100644
> --- a/include/hw/timer/mc146818rtc.h
> +++ b/include/hw/timer/mc146818rtc.h
> @@ -6,6 +6,7 @@
>  
>  ISADevice *rtc_init(ISABus *bus, int base_year, qemu_irq intercept_irq);
>  void rtc_set_memory(ISADevice *dev, int addr, int val);
> +int rtc_get_memory(ISADevice *dev, int addr);
>  void rtc_set_date(ISADevice *dev, const struct tm *tm);
>  
>  #endif /* !MC146818RTC_H */
> -- 
> 1.8.2
> 
> 

-- 
Eduardo

Re: [Qemu-devel] [PATCH 07/16] cpu: introduce get_arch_id() method and override it for target-i386

[Eduardo Habkost]

On Tue, Apr 16, 2013 at 12:12:47AM +0200, Igor Mammedov wrote:
> get_arch_id() adds possibility for generic code to get guest visible
> CPU ID without accessing CPUArchState. If target doesn't override it,
> it will return cpu_index.
> 
> Override it on target-i386 to return APIC ID.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>

Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>


> ---
>  * it will be used later by new cpu_exists() generic function and
>    acpi_piix.
>  * s/cpu_firmware_id/cpu_arch_id/
> ---
>  include/qom/cpu.h |  2 ++
>  qom/cpu.c         |  6 ++++++
>  target-i386/cpu.c | 10 ++++++++++
>  3 files changed, 18 insertions(+)
> 
> diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> index 6ce6f10..428aaf0 100644
> --- a/include/qom/cpu.h
> +++ b/include/qom/cpu.h
> @@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
>   * instantiatable CPU type.
>   * @reset: Callback to reset the #CPUState to its initial state.
>   * @do_interrupt: Callback for interrupt handling.
> + * @get_arch_id: Callback for getting architecture depended CPU ID
>   * @vmsd: State description for migration.
>   *
>   * Represents a CPU family or model.
> @@ -58,6 +59,7 @@ typedef struct CPUClass {
>  
>      void (*reset)(CPUState *cpu);
>      void (*do_interrupt)(CPUState *cpu);
> +    int64_t (*get_arch_id)(CPUState *cpu);
>  
>      const struct VMStateDescription *vmsd;
>  } CPUClass;
> diff --git a/qom/cpu.c b/qom/cpu.c
> index 819986e..a13ddda 100644
> --- a/qom/cpu.c
> +++ b/qom/cpu.c
> @@ -76,6 +76,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
>      }
>  }
>  
> +static int64_t cpu_common_get_arch_id(CPUState *cpu)
> +{
> +    return cpu->cpu_index;
> +}
> +
>  static void cpu_class_init(ObjectClass *klass, void *data)
>  {
>      DeviceClass *dc = DEVICE_CLASS(klass);
> @@ -83,6 +88,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
>  
>      k->class_by_name = cpu_common_class_by_name;
>      k->reset = cpu_common_reset;
> +    k->get_arch_id = cpu_common_get_arch_id;
>      dc->realize = cpu_common_realizefn;
>      dc->no_user = 1;
>  }
> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> index e2302d8..a415fa3 100644
> --- a/target-i386/cpu.c
> +++ b/target-i386/cpu.c
> @@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
>      }
>  }
>  
> +static int64_t x86_cpu_get_arch_id(CPUState *cpu)
> +{
> +    X86CPU *x86cpu = X86_CPU(cpu);
> +    CPUX86State *env = &x86cpu->env;
> +
> +    return env->cpuid_apic_id;
> +}
> +
>  static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
>  {
>      X86CPUClass *xcc = X86_CPU_CLASS(oc);
> @@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
>  
>      cc->do_interrupt = x86_cpu_do_interrupt;
>      cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
> +
> +    cc->get_arch_id = x86_cpu_get_arch_id;
>  }
>  
>  static const TypeInfo x86_cpu_type_info = {
> -- 
> 1.8.2
> 
> 

-- 
Eduardo

Re: [Qemu-devel] [PATCH 07/16] cpu: introduce get_arch_id() method and override it for target-i386

[li guang]

在 2013-04-16二的 00:12 +0200，Igor Mammedov写道：
> get_arch_id() adds possibility for generic code to get guest visible
> CPU ID without accessing CPUArchState. If target doesn't override it,
> it will return cpu_index.
> 
> Override it on target-i386 to return APIC ID.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>

Reviewed-by: liguang <lig.fnst@cn.fujitsu.com>

> ---
>  * it will be used later by new cpu_exists() generic function and
>    acpi_piix.
>  * s/cpu_firmware_id/cpu_arch_id/
> ---
>  include/qom/cpu.h |  2 ++
>  qom/cpu.c         |  6 ++++++
>  target-i386/cpu.c | 10 ++++++++++
>  3 files changed, 18 insertions(+)
> 
> diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> index 6ce6f10..428aaf0 100644
> --- a/include/qom/cpu.h
> +++ b/include/qom/cpu.h
> @@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
>   * instantiatable CPU type.
>   * @reset: Callback to reset the #CPUState to its initial state.
>   * @do_interrupt: Callback for interrupt handling.
> + * @get_arch_id: Callback for getting architecture depended CPU ID
>   * @vmsd: State description for migration.
>   *
>   * Represents a CPU family or model.
> @@ -58,6 +59,7 @@ typedef struct CPUClass {
>  
>      void (*reset)(CPUState *cpu);
>      void (*do_interrupt)(CPUState *cpu);
> +    int64_t (*get_arch_id)(CPUState *cpu);
>  
>      const struct VMStateDescription *vmsd;
>  } CPUClass;
> diff --git a/qom/cpu.c b/qom/cpu.c
> index 819986e..a13ddda 100644
> --- a/qom/cpu.c
> +++ b/qom/cpu.c
> @@ -76,6 +76,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
>      }
>  }
>  
> +static int64_t cpu_common_get_arch_id(CPUState *cpu)
> +{
> +    return cpu->cpu_index;
> +}
> +
>  static void cpu_class_init(ObjectClass *klass, void *data)
>  {
>      DeviceClass *dc = DEVICE_CLASS(klass);
> @@ -83,6 +88,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
>  
>      k->class_by_name = cpu_common_class_by_name;
>      k->reset = cpu_common_reset;
> +    k->get_arch_id = cpu_common_get_arch_id;
>      dc->realize = cpu_common_realizefn;
>      dc->no_user = 1;
>  }
> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> index e2302d8..a415fa3 100644
> --- a/target-i386/cpu.c
> +++ b/target-i386/cpu.c
> @@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
>      }
>  }
>  
> +static int64_t x86_cpu_get_arch_id(CPUState *cpu)
> +{
> +    X86CPU *x86cpu = X86_CPU(cpu);
> +    CPUX86State *env = &x86cpu->env;
> +
> +    return env->cpuid_apic_id;
> +}
> +
>  static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
>  {
>      X86CPUClass *xcc = X86_CPU_CLASS(oc);
> @@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
>  
>      cc->do_interrupt = x86_cpu_do_interrupt;
>      cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
> +
> +    cc->get_arch_id = x86_cpu_get_arch_id;
>  }
>  
>  static const TypeInfo x86_cpu_type_info = {

Re: [Qemu-devel] [PATCH 07/16] cpu: introduce get_arch_id() method and override it for target-i386

[Michael S. Tsirkin]

On Tue, Apr 16, 2013 at 12:12:47AM +0200, Igor Mammedov wrote:
> get_arch_id() adds possibility for generic code to get guest visible
> CPU ID without accessing CPUArchState. If target doesn't override it,
> it will return cpu_index.
> 
> Override it on target-i386 to return APIC ID.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>

Will also be helpful for dynamic ACPI Laszlo and me are
working on.

Acked-by: Michael S. Tsirkin <mst@redhat.com>

> ---
>  * it will be used later by new cpu_exists() generic function and
>    acpi_piix.
>  * s/cpu_firmware_id/cpu_arch_id/
> ---
>  include/qom/cpu.h |  2 ++
>  qom/cpu.c         |  6 ++++++
>  target-i386/cpu.c | 10 ++++++++++
>  3 files changed, 18 insertions(+)
> 
> diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> index 6ce6f10..428aaf0 100644
> --- a/include/qom/cpu.h
> +++ b/include/qom/cpu.h
> @@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
>   * instantiatable CPU type.
>   * @reset: Callback to reset the #CPUState to its initial state.
>   * @do_interrupt: Callback for interrupt handling.
> + * @get_arch_id: Callback for getting architecture depended CPU ID
>   * @vmsd: State description for migration.
>   *
>   * Represents a CPU family or model.
> @@ -58,6 +59,7 @@ typedef struct CPUClass {
>  
>      void (*reset)(CPUState *cpu);
>      void (*do_interrupt)(CPUState *cpu);
> +    int64_t (*get_arch_id)(CPUState *cpu);
>  
>      const struct VMStateDescription *vmsd;
>  } CPUClass;
> diff --git a/qom/cpu.c b/qom/cpu.c
> index 819986e..a13ddda 100644
> --- a/qom/cpu.c
> +++ b/qom/cpu.c
> @@ -76,6 +76,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
>      }
>  }
>  
> +static int64_t cpu_common_get_arch_id(CPUState *cpu)
> +{
> +    return cpu->cpu_index;
> +}
> +
>  static void cpu_class_init(ObjectClass *klass, void *data)
>  {
>      DeviceClass *dc = DEVICE_CLASS(klass);
> @@ -83,6 +88,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
>  
>      k->class_by_name = cpu_common_class_by_name;
>      k->reset = cpu_common_reset;
> +    k->get_arch_id = cpu_common_get_arch_id;
>      dc->realize = cpu_common_realizefn;
>      dc->no_user = 1;
>  }
> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> index e2302d8..a415fa3 100644
> --- a/target-i386/cpu.c
> +++ b/target-i386/cpu.c
> @@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
>      }
>  }
>  
> +static int64_t x86_cpu_get_arch_id(CPUState *cpu)
> +{
> +    X86CPU *x86cpu = X86_CPU(cpu);
> +    CPUX86State *env = &x86cpu->env;
> +
> +    return env->cpuid_apic_id;
> +}
> +
>  static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
>  {
>      X86CPUClass *xcc = X86_CPU_CLASS(oc);
> @@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
>  
>      cc->do_interrupt = x86_cpu_do_interrupt;
>      cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
> +
> +    cc->get_arch_id = x86_cpu_get_arch_id;
>  }
>  
>  static const TypeInfo x86_cpu_type_info = {
> -- 
> 1.8.2
> 

Re: [Qemu-devel] [PATCH 10/16] target-i386: introduce apic-id property

[Michael S. Tsirkin]

On Tue, Apr 16, 2013 at 12:12:50AM +0200, Igor Mammedov wrote:
> ... and use it from board level to set APIC ID for CPUs
> it creates.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

Reviewed-by: Michael S. Tsirkin <mst@redhat.com>

> ---
> Note:
>   * pc_new_cpu() function will be reused later in CPU hot-plug hook.
> 
> v3:
>   * user error_propagate() in property setter
> v2:
>   * use generic cpu_exists() instead of custom one
>   * make apic-id dynamic property, so it won't be possible to use it
>     as global property, since it's instance specific
> ---
>  hw/i386/pc.c      | 25 ++++++++++++++++++++++++-
>  target-i386/cpu.c | 42 ++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 66 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index dc1a78b..cb57878 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -889,9 +889,29 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
>      }
>  }
>  
> +static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
> +{
> +    X86CPU *cpu;
> +
> +    cpu = cpu_x86_create(cpu_model, errp);
> +    if (!cpu) {
> +        return;
> +    }
> +
> +    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
> +    object_property_set_bool(OBJECT(cpu), true, "realized", errp);
> +
> +    if (error_is_set(errp)) {
> +        if (cpu != NULL) {
> +            object_unref(OBJECT(cpu));
> +        }
> +    }
> +}
> +
>  void pc_cpus_init(const char *cpu_model)
>  {
>      int i;
> +    Error *error = NULL;
>  
>      /* init CPUs */
>      if (cpu_model == NULL) {
> @@ -903,7 +923,10 @@ void pc_cpus_init(const char *cpu_model)
>      }
>  
>      for (i = 0; i < smp_cpus; i++) {
> -        if (!cpu_x86_init(cpu_model)) {
> +        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
> +        if (error) {
> +            fprintf(stderr, "%s\n", error_get_pretty(error));
> +            error_free(error);
>              exit(1);
>          }
>      }
> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> index a415fa3..6d6c527 100644
> --- a/target-i386/cpu.c
> +++ b/target-i386/cpu.c
> @@ -1271,6 +1271,45 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque,
>      cpu->env.tsc_khz = value / 1000;
>  }
>  
> +static void x86_cpuid_get_apic_id(Object *obj, Visitor *v, void *opaque,
> +                                  const char *name, Error **errp)
> +{
> +    X86CPU *cpu = X86_CPU(obj);
> +    int64_t value = cpu->env.cpuid_apic_id;
> +
> +    visit_type_int(v, &value, name, errp);
> +}
> +
> +static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
> +                                  const char *name, Error **errp)
> +{
> +    X86CPU *cpu = X86_CPU(obj);
> +    const int64_t min = 0;
> +    const int64_t max = UINT32_MAX;
> +    Error *error = NULL;
> +    int64_t value;
> +
> +    visit_type_int(v, &value, name, &error);
> +    if (error) {
> +        error_propagate(errp, error);
> +        return;
> +    }
> +    if (value < min || value > max) {
> +        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
> +                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
> +                   object_get_typename(obj), name, value, min, max);
> +        error_propagate(errp, error);
> +        return;
> +    }
> +
> +    if (cpu_exists(value)) {
> +        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
> +        error_propagate(errp, error);
> +        return;
> +    }
> +    cpu->env.cpuid_apic_id = value;
> +}
> +
>  static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, const char *name)
>  {
>      x86_def_t *def;
> @@ -2259,6 +2298,9 @@ static void x86_cpu_initfn(Object *obj)
>      object_property_add(obj, "tsc-frequency", "int",
>                          x86_cpuid_get_tsc_freq,
>                          x86_cpuid_set_tsc_freq, NULL, NULL, NULL);
> +    object_property_add(obj, "apic-id", "int",
> +                        x86_cpuid_get_apic_id,
> +                        x86_cpuid_set_apic_id, NULL, NULL, NULL);
>  
>      env->cpuid_apic_id = x86_cpu_apic_id_from_index(cs->cpu_index);
>  
> -- 
> 1.8.2
> 

Re: [Qemu-devel] [PATCH 09/16] acpi_piix4: add infrastructure to send CPU hot-plug GPE to guest

[Michael S. Tsirkin]

On Tue, Apr 16, 2013 at 12:12:49AM +0200, Igor Mammedov wrote:
> * introduce processor status bitmask visible to guest at 0xaf00 addr,
>   where ACPI asl code expects it
> * set bit corresponding to APIC ID in processor status bitmask on
>   receiving CPU hot-plug notification
> * trigger CPU hot-plug SCI, to notify guest about CPU hot-plug event
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
> v3:
>   * s/get_firmware_id()/get_arch_id()/ due rebase
>   * s/cpu_add_notifier/cpu_added_notifier/
> v2:
>   * use CPUClass.get_firmware_id() to make code target independent
>   * bump up vmstate_acpi version
> ---
>  hw/acpi/piix4.c | 116 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 113 insertions(+), 3 deletions(-)
> 
> diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
> index 88386d7..eecdbb0 100644
> --- a/hw/acpi/piix4.c
> +++ b/hw/acpi/piix4.c
> @@ -48,19 +48,28 @@
>  #define PCI_EJ_BASE 0xae08
>  #define PCI_RMV_BASE 0xae0c
>  
> +#define PROC_BASE 0xaf00
> +#define PROC_LEN 32

Better prefix macros with PIIX4 or something.

> +
>  #define PIIX4_PCI_HOTPLUG_STATUS 2
> +#define PIIX4_CPU_HOTPLUG_STATUS 4
>  
>  struct pci_status {
>      uint32_t up; /* deprecated, maintained for migration compatibility */
>      uint32_t down;
>  };
>  
> +struct cpu_status {
> +    uint8_t sts[PROC_LEN];
> +};
> +
>  typedef struct PIIX4PMState {
>      PCIDevice dev;
>  
>      MemoryRegion io;
>      MemoryRegion io_gpe;
>      MemoryRegion io_pci;
> +    MemoryRegion io_cpu;
>      ACPIREGS ar;
>  
>      APMState apm;
> @@ -82,6 +91,9 @@ typedef struct PIIX4PMState {
>      uint8_t disable_s3;
>      uint8_t disable_s4;
>      uint8_t s4_val;
> +
> +    struct cpu_status gpe_cpu;
> +    Notifier cpu_added_notifier;
>  } PIIX4PMState;
>  
>  static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
> @@ -100,8 +112,8 @@ static void pm_update_sci(PIIX4PMState *s)
>                     ACPI_BITMASK_POWER_BUTTON_ENABLE |
>                     ACPI_BITMASK_GLOBAL_LOCK_ENABLE |
>                     ACPI_BITMASK_TIMER_ENABLE)) != 0) ||
> -        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0])
> -          & PIIX4_PCI_HOTPLUG_STATUS) != 0);
> +        (((s->ar.gpe.sts[0] & s->ar.gpe.en[0]) &
> +          (PIIX4_PCI_HOTPLUG_STATUS | PIIX4_CPU_HOTPLUG_STATUS)) != 0);
>  
>      qemu_set_irq(s->irq, sci_level);
>      /* schedule a timer interruption if needed */
> @@ -257,6 +269,17 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
>      return ret;
>  }
>  
> +#define VMSTATE_CPU_STATUS_ARRAY(_field, _state)                             \
> + {                                                                           \
> +     .name       = (stringify(_field)),                                      \
> +     .version_id = 0,                                                        \
> +     .num        = PROC_LEN,                                                 \
> +     .info       = &vmstate_info_uint8,                                      \
> +     .size       = sizeof(uint8_t),                                          \
> +     .flags      = VMS_ARRAY,                                                \
> +     .offset     = vmstate_offset_array(_state, _field, uint8_t, PROC_LEN),  \
> + }
> +
>  /* qemu-kvm 1.2 uses version 3 but advertised as 2
>   * To support incoming qemu-kvm 1.2 migration, change version_id
>   * and minimum_version_id to 2 below (which breaks migration from
> @@ -265,7 +288,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
>   */
>  static const VMStateDescription vmstate_acpi = {
>      .name = "piix4_pm",
> -    .version_id = 3,
> +    .version_id = 4,
>      .minimum_version_id = 3,
>      .minimum_version_id_old = 1,
>      .load_state_old = acpi_load_old,
> @@ -281,6 +304,7 @@ static const VMStateDescription vmstate_acpi = {
>          VMSTATE_STRUCT(ar.gpe, PIIX4PMState, 2, vmstate_gpe, ACPIGPE),
>          VMSTATE_STRUCT(pci0_status, PIIX4PMState, 2, vmstate_pci_status,
>                         struct pci_status),
> +        VMSTATE_CPU_STATUS_ARRAY(gpe_cpu.sts, PIIX4PMState),
>          VMSTATE_END_OF_LIST()
>      }
>  };
> @@ -585,6 +609,85 @@ static const MemoryRegionOps piix4_pci_ops = {
>      },
>  };
>  
> +static uint64_t cpu_status_read(void *opaque, hwaddr addr, unsigned width)
> +{
> +    PIIX4PMState *s = opaque;
> +    struct cpu_status *cpus = &s->gpe_cpu;
> +    uint64_t val = cpus->sts[addr];
> +
> +    return val;
> +}
> +
> +static void cpu_status_write(void *opaque, hwaddr addr, uint64_t data,
> +                             unsigned int size)
> +{
> +    /* TODO: implement VCPU removal on guest signal that CPU can be removed */
> +}
> +
> +static const MemoryRegionOps cpu_hotplug_ops = {
> +    .read = cpu_status_read,
> +    .write = cpu_status_write,
> +    .endianness = DEVICE_LITTLE_ENDIAN,
> +    .valid = {
> +        .min_access_size = 1,
> +        .max_access_size = 1,
> +    },
> +};
> +
> +typedef enum {
> +    PLUG,
> +    UNPLUG,
> +} HotplugEventType;
> +
> +static void piix4_cpu_hotplug_req(PIIX4PMState *s, CPUState *cpu,
> +                                  HotplugEventType action)
> +{
> +    struct cpu_status *g = &s->gpe_cpu;
> +    ACPIGPE *gpe = &s->ar.gpe;
> +    CPUClass *k = CPU_GET_CLASS(cpu);
> +    int64_t cpu_id;
> +
> +    assert(s != NULL);
> +
> +    *gpe->sts = *gpe->sts | PIIX4_CPU_HOTPLUG_STATUS;
> +    cpu_id = k->get_arch_id(CPU(cpu));
> +    if (action == PLUG) {
> +        g->sts[cpu_id / 8] |= (1 << (cpu_id % 8));
> +    } else {
> +        g->sts[cpu_id / 8] &= ~(1 << (cpu_id % 8));
> +    }
> +    pm_update_sci(s);
> +}
> +
> +static void piix4_cpu_added_req(Notifier *n, void *opaque)
> +{
> +    PIIX4PMState *s = container_of(n, PIIX4PMState, cpu_added_notifier);
> +
> +    piix4_cpu_hotplug_req(s, CPU(opaque), PLUG);
> +}
> +
> +static int piix4_init_cpu_status(Object *obj, void *opaque)
> +{
> +    struct cpu_status *g = (struct cpu_status *)opaque;
> +    Object *cpu_obj = object_dynamic_cast(obj, TYPE_CPU);
> +
> +    if (cpu_obj) {
> +        struct Error *error = NULL;
> +        CPUClass *k = CPU_GET_CLASS(cpu_obj);
> +        int64_t id = k->get_arch_id(CPU(cpu_obj));
> +
> +        if (error) {
> +            fprintf(stderr, "failed to initilize CPU status for ACPI: %s\n",
> +                    error_get_pretty(error));
> +            error_free(error);
> +            abort();
> +        }
> +        g_assert((id / 8) < PROC_LEN);
> +        g->sts[id / 8] |= (1 << (id % 8));
> +    }
> +    return object_child_foreach(obj, piix4_init_cpu_status, opaque);
> +}
> +
>  static int piix4_device_hotplug(DeviceState *qdev, PCIDevice *dev,
>                                  PCIHotplugState state);
>  
> @@ -600,6 +703,13 @@ static void piix4_acpi_system_hot_add_init(MemoryRegion *parent,
>      memory_region_add_subregion(parent, PCI_HOTPLUG_ADDR,
>                                  &s->io_pci);
>      pci_bus_hotplug(bus, piix4_device_hotplug, &s->dev.qdev);
> +
> +    piix4_init_cpu_status(qdev_get_machine(), &s->gpe_cpu);
> +    memory_region_init_io(&s->io_cpu, &cpu_hotplug_ops, s, "apci-cpu-hotplug",
> +                          PROC_LEN);
> +    memory_region_add_subregion(parent, PROC_BASE, &s->io_cpu);

Is there documentation for this new interface?

> +    s->cpu_added_notifier.notify = piix4_cpu_added_req;
> +    qemu_register_cpu_added_notifier(&s->cpu_added_notifier);
>  }
>  
>  static void enable_device(PIIX4PMState *s, int slot)
> -- 
> 1.8.2
> 

Re: [Qemu-devel] [PATCH 08/16] cpu: add helper cpu_exists(), to check if CPU with specified id exists

[Michael S. Tsirkin]

On Tue, Apr 16, 2013 at 12:12:48AM +0200, Igor Mammedov wrote:
> ... it should be used only on slow path since it does recursive search
>     on /machine QOM tree for objects of TYPE_CPU
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>

I would prefer qemu_for_each_cpu, that is also useful for ACPI.  And do
we need to scan all QOM?  This will take a while to repeat if there are
many devices.
How about the below? You can add a wrapper that sets
a bit if CPU exists.


commit 4ff1332ff56069574f450bb44819156bd91bd105
Author: Michael S. Tsirkin <mst@redhat.com>
Date:   Sun Apr 21 15:44:47 2013 +0300

    exec: add qemu_for_each_cpu
    
    Will be used by ACPI table generation.
    
    Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

diff --git a/exec.c b/exec.c
index fa1e0c3..2b44cbe 100644
--- a/exec.c
+++ b/exec.c
@@ -265,6 +265,19 @@ CPUState *qemu_get_cpu(int index)
     return env ? cpu : NULL;
 }
 
+void qemu_for_each_cpu( void (*func)( CPUState *c, void *arg), void *arg)
+{
+    CPUArchState *env = first_cpu;
+    CPUState *cpu;
+
+    while (env) {
+        cpu = ENV_GET_CPU(env);
+        func(cpu, arg);
+        env = env->next_cpu;
+    }
+}
+
+
 void cpu_exec_init(CPUArchState *env)
 {
     CPUState *cpu = ENV_GET_CPU(env);
diff --git a/include/qom/cpu.h b/include/qom/cpu.h
index 3664a1b..db857e3 100644
--- a/include/qom/cpu.h
+++ b/include/qom/cpu.h
@@ -223,6 +223,8 @@ void run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data);
  */
 CPUState *qemu_get_cpu(int index);
 
+void qemu_for_each_cpu( void (*func)( CPUState *c, void *arg), void *arg);
+
 #ifndef CONFIG_USER_ONLY
 
 typedef void (*CPUInterruptHandler)(CPUState *, int);

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

[Gleb Natapov]

On Tue, Apr 16, 2013 at 12:12:43AM +0200, Igor Mammedov wrote:
> ... also add stub for it, to make possible to use it in qom/cpu.c,
> which is shared with user emulators.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
> Note:
>   it will be re-used from qom/cpu.c by hotpluged CPU
> ---
>  cpus.c              | 11 ++++++++---
>  include/qom/cpu.h   |  9 +++++++++
>  stubs/Makefile.objs |  1 +
>  stubs/cpus.c        |  5 +++++
>  4 files changed, 23 insertions(+), 3 deletions(-)
>  create mode 100644 stubs/cpus.c
> 
> diff --git a/cpus.c b/cpus.c
> index c15ff6c..c347db5 100644
> --- a/cpus.c
> +++ b/cpus.c
> @@ -993,6 +993,13 @@ void pause_all_vcpus(void)
>      }
>  }
>  
> +void resume_vcpu(CPUState *cpu)
> +{
> +    cpu->stop = false;
> +    cpu->stopped = false;
> +    qemu_cpu_kick(cpu);
> +}
> +
>  void resume_all_vcpus(void)
>  {
>      CPUArchState *penv = first_cpu;
> @@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
>      qemu_clock_enable(vm_clock, true);
>      while (penv) {
>          CPUState *pcpu = ENV_GET_CPU(penv);
> -        pcpu->stop = false;
> -        pcpu->stopped = false;
> -        qemu_cpu_kick(pcpu);
> +        resume_vcpu(pcpu);
>          penv = penv->next_cpu;
>      }
>  }
> diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> index 3664a1b..6ce6f10 100644
> --- a/include/qom/cpu.h
> +++ b/include/qom/cpu.h
> @@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
>   */
>  void cpu_reset_interrupt(CPUState *cpu, int mask);
>  
> +/**
> + * resume_vcpu:
> + * @cpu: The CPU to resume.
> + *
> + * Resumes CPU, i.e. puts CPU into runable state.
> + * KVM requires extra call cpu_synchronize_post_init() to make
> + * KVM's CPU instance runable.
> + */
Can you elaborate on this a bit. Why is it needed and why is it relevant
for this function?

> +void resume_vcpu(CPUState *cpu);
>  
>  #endif
> diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs
> index 9c55b34..03dff20 100644
> --- a/stubs/Makefile.objs
> +++ b/stubs/Makefile.objs
> @@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
>  stub-obj-y += vm-stop.o
>  stub-obj-y += vmstate.o
>  stub-obj-$(CONFIG_WIN32) += fd-register.o
> +stub-obj-y += cpus.o
> diff --git a/stubs/cpus.c b/stubs/cpus.c
> new file mode 100644
> index 0000000..3d17451
> --- /dev/null
> +++ b/stubs/cpus.c
> @@ -0,0 +1,5 @@
> +#include "qom/cpu.h"
> +
> +void resume_vcpu(CPUState *cpu)
> +{
> +}
> -- 
> 1.8.2
> 
> 

--
			Gleb.

Re: [Qemu-devel] [PATCH 08/16] cpu: add helper cpu_exists(), to check if CPU with specified id exists

[Igor Mammedov]

On Mon, 22 Apr 2013 13:28:12 +0300
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> On Tue, Apr 16, 2013 at 12:12:48AM +0200, Igor Mammedov wrote:
> > ... it should be used only on slow path since it does recursive search
> >     on /machine QOM tree for objects of TYPE_CPU
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
> 
> I would prefer qemu_for_each_cpu, that is also useful for ACPI.  And do
> we need to scan all QOM?  This will take a while to repeat if there are
> many devices.
> How about the below? You can add a wrapper that sets
> a bit if CPU exists.
qemu_for_each_cpu looks usefull, it could replace a lot of place that do
open-coded exactly this operation and hide open-codded access to next_cpu
pointer. 

I can easily change to using it, and take it this patch into this series.

> 
> 
> commit 4ff1332ff56069574f450bb44819156bd91bd105
> Author: Michael S. Tsirkin <mst@redhat.com>
> Date:   Sun Apr 21 15:44:47 2013 +0300
> 
>     exec: add qemu_for_each_cpu
>     
>     Will be used by ACPI table generation.
>     
>     Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> 
> diff --git a/exec.c b/exec.c
> index fa1e0c3..2b44cbe 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -265,6 +265,19 @@ CPUState *qemu_get_cpu(int index)
>      return env ? cpu : NULL;
>  }
>  
> +void qemu_for_each_cpu( void (*func)( CPUState *c, void *arg), void *arg)
> +{
> +    CPUArchState *env = first_cpu;
> +    CPUState *cpu;
> +
> +    while (env) {
> +        cpu = ENV_GET_CPU(env);
> +        func(cpu, arg);
> +        env = env->next_cpu;
> +    }
> +}
> +
> +
>  void cpu_exec_init(CPUArchState *env)
>  {
>      CPUState *cpu = ENV_GET_CPU(env);
> diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> index 3664a1b..db857e3 100644
> --- a/include/qom/cpu.h
> +++ b/include/qom/cpu.h
> @@ -223,6 +223,8 @@ void run_on_cpu(CPUState *cpu, void (*func)(void
> *data), void *data); */
>  CPUState *qemu_get_cpu(int index);
>  
> +void qemu_for_each_cpu( void (*func)( CPUState *c, void *arg), void *arg);
> +
>  #ifndef CONFIG_USER_ONLY
>  
>  typedef void (*CPUInterruptHandler)(CPUState *, int);
> 

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

[Igor Mammedov]

On Mon, 22 Apr 2013 13:40:02 +0300
Gleb Natapov <gleb@redhat.com> wrote:

> On Tue, Apr 16, 2013 at 12:12:43AM +0200, Igor Mammedov wrote:
> > ... also add stub for it, to make possible to use it in qom/cpu.c,
> > which is shared with user emulators.
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > ---
> > Note:
> >   it will be re-used from qom/cpu.c by hotpluged CPU
> > ---
> >  cpus.c              | 11 ++++++++---
> >  include/qom/cpu.h   |  9 +++++++++
> >  stubs/Makefile.objs |  1 +
> >  stubs/cpus.c        |  5 +++++
> >  4 files changed, 23 insertions(+), 3 deletions(-)
> >  create mode 100644 stubs/cpus.c
> > 
> > diff --git a/cpus.c b/cpus.c
> > index c15ff6c..c347db5 100644
> > --- a/cpus.c
> > +++ b/cpus.c
> > @@ -993,6 +993,13 @@ void pause_all_vcpus(void)
> >      }
> >  }
> >  
> > +void resume_vcpu(CPUState *cpu)
> > +{
> > +    cpu->stop = false;
> > +    cpu->stopped = false;
> > +    qemu_cpu_kick(cpu);
> > +}
> > +
> >  void resume_all_vcpus(void)
> >  {
> >      CPUArchState *penv = first_cpu;
> > @@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
> >      qemu_clock_enable(vm_clock, true);
> >      while (penv) {
> >          CPUState *pcpu = ENV_GET_CPU(penv);
> > -        pcpu->stop = false;
> > -        pcpu->stopped = false;
> > -        qemu_cpu_kick(pcpu);
> > +        resume_vcpu(pcpu);
> >          penv = penv->next_cpu;
> >      }
> >  }
> > diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> > index 3664a1b..6ce6f10 100644
> > --- a/include/qom/cpu.h
> > +++ b/include/qom/cpu.h
> > @@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
> >   */
> >  void cpu_reset_interrupt(CPUState *cpu, int mask);
> >  
> > +/**
> > + * resume_vcpu:
> > + * @cpu: The CPU to resume.
> > + *
> > + * Resumes CPU, i.e. puts CPU into runable state.
> > + * KVM requires extra call cpu_synchronize_post_init() to make
> > + * KVM's CPU instance runable.
> > + */
> Can you elaborate on this a bit. Why is it needed and why is it relevant
> for this function?
If running in KVM mode and resume_vcpu() is called without
cpu_synchronize_post_init(), then CPU instance in KVM will be
left in UNCONFIGURED state. I've added comment here to document 
that resume_vcpu() isn't enough to make CPU run-able in case of KVM.

> 
> > +void resume_vcpu(CPUState *cpu);
> >  
> >  #endif
> > diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs
> > index 9c55b34..03dff20 100644
> > --- a/stubs/Makefile.objs
> > +++ b/stubs/Makefile.objs
> > @@ -23,3 +23,4 @@ stub-obj-y += sysbus.o
> >  stub-obj-y += vm-stop.o
> >  stub-obj-y += vmstate.o
> >  stub-obj-$(CONFIG_WIN32) += fd-register.o
> > +stub-obj-y += cpus.o
> > diff --git a/stubs/cpus.c b/stubs/cpus.c
> > new file mode 100644
> > index 0000000..3d17451
> > --- /dev/null
> > +++ b/stubs/cpus.c
> > @@ -0,0 +1,5 @@
> > +#include "qom/cpu.h"
> > +
> > +void resume_vcpu(CPUState *cpu)
> > +{
> > +}
> > -- 
> > 1.8.2
> > 
> > 
> 
> --
> 			Gleb.

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

[Gleb Natapov]

On Mon, Apr 22, 2013 at 12:54:04PM +0200, Igor Mammedov wrote:
> On Mon, 22 Apr 2013 13:40:02 +0300
> Gleb Natapov <gleb@redhat.com> wrote:
> 
> > On Tue, Apr 16, 2013 at 12:12:43AM +0200, Igor Mammedov wrote:
> > > ... also add stub for it, to make possible to use it in qom/cpu.c,
> > > which is shared with user emulators.
> > > 
> > > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > > ---
> > > Note:
> > >   it will be re-used from qom/cpu.c by hotpluged CPU
> > > ---
> > >  cpus.c              | 11 ++++++++---
> > >  include/qom/cpu.h   |  9 +++++++++
> > >  stubs/Makefile.objs |  1 +
> > >  stubs/cpus.c        |  5 +++++
> > >  4 files changed, 23 insertions(+), 3 deletions(-)
> > >  create mode 100644 stubs/cpus.c
> > > 
> > > diff --git a/cpus.c b/cpus.c
> > > index c15ff6c..c347db5 100644
> > > --- a/cpus.c
> > > +++ b/cpus.c
> > > @@ -993,6 +993,13 @@ void pause_all_vcpus(void)
> > >      }
> > >  }
> > >  
> > > +void resume_vcpu(CPUState *cpu)
> > > +{
> > > +    cpu->stop = false;
> > > +    cpu->stopped = false;
> > > +    qemu_cpu_kick(cpu);
> > > +}
> > > +
> > >  void resume_all_vcpus(void)
> > >  {
> > >      CPUArchState *penv = first_cpu;
> > > @@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
> > >      qemu_clock_enable(vm_clock, true);
> > >      while (penv) {
> > >          CPUState *pcpu = ENV_GET_CPU(penv);
> > > -        pcpu->stop = false;
> > > -        pcpu->stopped = false;
> > > -        qemu_cpu_kick(pcpu);
> > > +        resume_vcpu(pcpu);
> > >          penv = penv->next_cpu;
> > >      }
> > >  }
> > > diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> > > index 3664a1b..6ce6f10 100644
> > > --- a/include/qom/cpu.h
> > > +++ b/include/qom/cpu.h
> > > @@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
> > >   */
> > >  void cpu_reset_interrupt(CPUState *cpu, int mask);
> > >  
> > > +/**
> > > + * resume_vcpu:
> > > + * @cpu: The CPU to resume.
> > > + *
> > > + * Resumes CPU, i.e. puts CPU into runable state.
> > > + * KVM requires extra call cpu_synchronize_post_init() to make
> > > + * KVM's CPU instance runable.
> > > + */
> > Can you elaborate on this a bit. Why is it needed and why is it relevant
> > for this function?
> If running in KVM mode and resume_vcpu() is called without
> cpu_synchronize_post_init(), then CPU instance in KVM will be
> left in UNCONFIGURED state. I've added comment here to document 
> that resume_vcpu() isn't enough to make CPU run-able in case of KVM.
> 
But this really relevant only during vcpu creation after that
resume_vcpu() should just work. My be have launch_vcpu() function that
does the right thing and add assert to resume_vcpu() if it is called on
vcpu that is not launched yet. In other words made the dependency clear
in the code, not comment.

--
			Gleb.

Re: [Qemu-devel] [PATCH 05/16] introduce CPU hot-plug notifier

[Gleb Natapov]

On Tue, Apr 16, 2013 at 12:12:45AM +0200, Igor Mammedov wrote:
> hot-added CPU will be distributed to acpi_piix4 and rtc_cmos
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
> ---
> v2:
>   * move notifier to qom/cpu.c and call it from CPUClass.realize() on hotplug
>   * remove get_firmware_id() since it belong to other patch
> ---
>  include/sysemu/sysemu.h |  3 +++
>  qom/cpu.c               | 12 ++++++++++++
>  2 files changed, 15 insertions(+)
> 
> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> index 6578782..a8c3de1 100644
> --- a/include/sysemu/sysemu.h
> +++ b/include/sysemu/sysemu.h
> @@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const QDict *qdict);
>  /* generic hotplug */
>  void drive_hot_add(Monitor *mon, const QDict *qdict);
>  
> +/* CPU hotplug */
> +void qemu_register_cpu_added_notifier(Notifier *notifier);
> +
>  /* pcie aer error injection */
>  void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
>  int do_pcie_aer_inject_error(Monitor *mon,
> diff --git a/qom/cpu.c b/qom/cpu.c
> index 40a4259..819986e 100644
> --- a/qom/cpu.c
> +++ b/qom/cpu.c
> @@ -21,6 +21,17 @@
>  #include "qom/cpu.h"
>  #include "qemu-common.h"
>  #include "sysemu/kvm.h"
> +#include "qemu/notify.h"
> +#include "sysemu/sysemu.h"
> +
> +/* CPU hot-plug notifiers */
> +static NotifierList cpu_added_notifiers =
> +    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
> +
> +void qemu_register_cpu_added_notifier(Notifier *notifier)
> +{
> +    notifier_list_add(&cpu_added_notifiers, notifier);
> +}
>  
>  void cpu_reset_interrupt(CPUState *cpu, int mask)
>  {
> @@ -61,6 +72,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
>      if (dev->hotplugged) {
>          cpu_synchronize_post_init(CPU(dev));
>          resume_vcpu(CPU(dev));
> +        notifier_list_notify(&cpu_added_notifiers, dev);
Why do it after vcpu is running? May be notifier want to do something
that should be done before vcpu is running.

>      }
>  }
>  
> -- 
> 1.8.2
> 
> 

--
			Gleb.

Re: [Qemu-devel] [PATCH 05/16] introduce CPU hot-plug notifier

[Igor Mammedov]

On Mon, 22 Apr 2013 14:00:01 +0300
Gleb Natapov <gleb@redhat.com> wrote:

> On Tue, Apr 16, 2013 at 12:12:45AM +0200, Igor Mammedov wrote:
> > hot-added CPU will be distributed to acpi_piix4 and rtc_cmos
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
> > Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
> > ---
> > v2:
> >   * move notifier to qom/cpu.c and call it from CPUClass.realize() on
> > hotplug
> >   * remove get_firmware_id() since it belong to other patch
> > ---
> >  include/sysemu/sysemu.h |  3 +++
> >  qom/cpu.c               | 12 ++++++++++++
> >  2 files changed, 15 insertions(+)
> > 
> > diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> > index 6578782..a8c3de1 100644
> > --- a/include/sysemu/sysemu.h
> > +++ b/include/sysemu/sysemu.h
> > @@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const
> > QDict *qdict); /* generic hotplug */
> >  void drive_hot_add(Monitor *mon, const QDict *qdict);
> >  
> > +/* CPU hotplug */
> > +void qemu_register_cpu_added_notifier(Notifier *notifier);
> > +
> >  /* pcie aer error injection */
> >  void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
> >  int do_pcie_aer_inject_error(Monitor *mon,
> > diff --git a/qom/cpu.c b/qom/cpu.c
> > index 40a4259..819986e 100644
> > --- a/qom/cpu.c
> > +++ b/qom/cpu.c
> > @@ -21,6 +21,17 @@
> >  #include "qom/cpu.h"
> >  #include "qemu-common.h"
> >  #include "sysemu/kvm.h"
> > +#include "qemu/notify.h"
> > +#include "sysemu/sysemu.h"
> > +
> > +/* CPU hot-plug notifiers */
> > +static NotifierList cpu_added_notifiers =
> > +    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
> > +
> > +void qemu_register_cpu_added_notifier(Notifier *notifier)
> > +{
> > +    notifier_list_add(&cpu_added_notifiers, notifier);
> > +}
> >  
> >  void cpu_reset_interrupt(CPUState *cpu, int mask)
> >  {
> > @@ -61,6 +72,7 @@ static void cpu_common_realizefn(DeviceState *dev,
> > Error **errp) if (dev->hotplugged) {
> >          cpu_synchronize_post_init(CPU(dev));
> >          resume_vcpu(CPU(dev));
> > +        notifier_list_notify(&cpu_added_notifiers, dev);
> Why do it after vcpu is running? May be notifier want to do something
> that should be done before vcpu is running.
currently there is not need for adding_cpu notifier.
This one signals that there is a new fully functional CPU available.

> >      }
> >  }
> >  
> > -- 
> > 1.8.2
> > 
> > 
> 
> --
> 			Gleb.

Re: [Qemu-devel] [PATCH 08/16] cpu: add helper cpu_exists(), to check if CPU with specified id exists

[Andreas Färber]

Am 22.04.2013 12:28, schrieb Michael S. Tsirkin:
> On Tue, Apr 16, 2013 at 12:12:48AM +0200, Igor Mammedov wrote:
>> ... it should be used only on slow path since it does recursive search
>>     on /machine QOM tree for objects of TYPE_CPU
>>
>> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
>> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
> 
> I would prefer qemu_for_each_cpu, that is also useful for ACPI.  And do
> we need to scan all QOM?  This will take a while to repeat if there are
> many devices.
> How about the below? You can add a wrapper that sets
> a bit if CPU exists.

I really like it! Didn't get around to writing such a helper myself yet
- this will allow us to drop quite some CPUArchState usages in the tree.
If you can submit this as a proper patch (git-send-email, with a gtk-doc
comment, is the spacing intentional?) along with a trivial loop ->
helper conversion as proof of concept I'll queue it immediately.

Toda,
Andreas

> commit 4ff1332ff56069574f450bb44819156bd91bd105
> Author: Michael S. Tsirkin <mst@redhat.com>
> Date:   Sun Apr 21 15:44:47 2013 +0300
> 
>     exec: add qemu_for_each_cpu
>     
>     Will be used by ACPI table generation.
>     
>     Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> 
> diff --git a/exec.c b/exec.c
> index fa1e0c3..2b44cbe 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -265,6 +265,19 @@ CPUState *qemu_get_cpu(int index)
>      return env ? cpu : NULL;
>  }
>  
> +void qemu_for_each_cpu( void (*func)( CPUState *c, void *arg), void *arg)
> +{
> +    CPUArchState *env = first_cpu;
> +    CPUState *cpu;
> +
> +    while (env) {
> +        cpu = ENV_GET_CPU(env);
> +        func(cpu, arg);
> +        env = env->next_cpu;
> +    }
> +}
> +
> +
>  void cpu_exec_init(CPUArchState *env)
>  {
>      CPUState *cpu = ENV_GET_CPU(env);
> diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> index 3664a1b..db857e3 100644
> --- a/include/qom/cpu.h
> +++ b/include/qom/cpu.h
> @@ -223,6 +223,8 @@ void run_on_cpu(CPUState *cpu, void (*func)(void *data), void *data);
>   */
>  CPUState *qemu_get_cpu(int index);
>  
> +void qemu_for_each_cpu( void (*func)( CPUState *c, void *arg), void *arg);
> +
>  #ifndef CONFIG_USER_ONLY
>  
>  typedef void (*CPUInterruptHandler)(CPUState *, int);
> 


-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 05/16] introduce CPU hot-plug notifier

[Gleb Natapov]

On Mon, Apr 22, 2013 at 01:09:19PM +0200, Igor Mammedov wrote:
> On Mon, 22 Apr 2013 14:00:01 +0300
> Gleb Natapov <gleb@redhat.com> wrote:
> 
> > On Tue, Apr 16, 2013 at 12:12:45AM +0200, Igor Mammedov wrote:
> > > hot-added CPU will be distributed to acpi_piix4 and rtc_cmos
> > > 
> > > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > > Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
> > > Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
> > > ---
> > > v2:
> > >   * move notifier to qom/cpu.c and call it from CPUClass.realize() on
> > > hotplug
> > >   * remove get_firmware_id() since it belong to other patch
> > > ---
> > >  include/sysemu/sysemu.h |  3 +++
> > >  qom/cpu.c               | 12 ++++++++++++
> > >  2 files changed, 15 insertions(+)
> > > 
> > > diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> > > index 6578782..a8c3de1 100644
> > > --- a/include/sysemu/sysemu.h
> > > +++ b/include/sysemu/sysemu.h
> > > @@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const
> > > QDict *qdict); /* generic hotplug */
> > >  void drive_hot_add(Monitor *mon, const QDict *qdict);
> > >  
> > > +/* CPU hotplug */
> > > +void qemu_register_cpu_added_notifier(Notifier *notifier);
> > > +
> > >  /* pcie aer error injection */
> > >  void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
> > >  int do_pcie_aer_inject_error(Monitor *mon,
> > > diff --git a/qom/cpu.c b/qom/cpu.c
> > > index 40a4259..819986e 100644
> > > --- a/qom/cpu.c
> > > +++ b/qom/cpu.c
> > > @@ -21,6 +21,17 @@
> > >  #include "qom/cpu.h"
> > >  #include "qemu-common.h"
> > >  #include "sysemu/kvm.h"
> > > +#include "qemu/notify.h"
> > > +#include "sysemu/sysemu.h"
> > > +
> > > +/* CPU hot-plug notifiers */
> > > +static NotifierList cpu_added_notifiers =
> > > +    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
> > > +
> > > +void qemu_register_cpu_added_notifier(Notifier *notifier)
> > > +{
> > > +    notifier_list_add(&cpu_added_notifiers, notifier);
> > > +}
> > >  
> > >  void cpu_reset_interrupt(CPUState *cpu, int mask)
> > >  {
> > > @@ -61,6 +72,7 @@ static void cpu_common_realizefn(DeviceState *dev,
> > > Error **errp) if (dev->hotplugged) {
> > >          cpu_synchronize_post_init(CPU(dev));
> > >          resume_vcpu(CPU(dev));
> > > +        notifier_list_notify(&cpu_added_notifiers, dev);
> > Why do it after vcpu is running? May be notifier want to do something
> > that should be done before vcpu is running.
> currently there is not need for adding_cpu notifier.
> This one signals that there is a new fully functional CPU available.
> 
You are using it to update cmos information for BIOS. It would be strange
for newly added vcpu to see incorrect number of cpus in cmos if, for
some reason, it will check it. The "new cpu is added and ready to tun,
do your last things before it is launched" callback seams more reasonable
then "new cpu is running wild" one.

--
			Gleb.

Re: [Qemu-devel] [PATCH 11/16] introduce ICC bus/device/bridge

[Peter Maydell]

On 15 April 2013 23:12, Igor Mammedov <imammedo@redhat.com> wrote:
> ... to provide hotplug-able bus.
>
> * icc-bridge will serve as a parent for icc-bus and provide
>   mmio mapping services to child icc-devices.
> * icc-device will replace SysBusDevice as a parent of APIC
>   and IOAPIC devices.
>
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
> v2:
>   * Rebase on top the last HW reorganization series.
>   * Move hw/icc_bus.c into hw/cpu/ and hw/icc_bus.h include/hw/i386/

This seems inconsistent. I think the header should be in
include/hw/cpu/ to match the location of the .c file.

thanks
-- PMM

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

[Igor Mammedov]

On Mon, 22 Apr 2013 13:59:17 +0300
Gleb Natapov <gleb@redhat.com> wrote:

> On Mon, Apr 22, 2013 at 12:54:04PM +0200, Igor Mammedov wrote:
> > On Mon, 22 Apr 2013 13:40:02 +0300
> > Gleb Natapov <gleb@redhat.com> wrote:
> > 
> > > On Tue, Apr 16, 2013 at 12:12:43AM +0200, Igor Mammedov wrote:
> > > > ... also add stub for it, to make possible to use it in qom/cpu.c,
> > > > which is shared with user emulators.
> > > > 
> > > > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > > > ---
> > > > Note:
> > > >   it will be re-used from qom/cpu.c by hotpluged CPU
> > > > ---
> > > >  cpus.c              | 11 ++++++++---
> > > >  include/qom/cpu.h   |  9 +++++++++
> > > >  stubs/Makefile.objs |  1 +
> > > >  stubs/cpus.c        |  5 +++++
> > > >  4 files changed, 23 insertions(+), 3 deletions(-)
> > > >  create mode 100644 stubs/cpus.c
> > > > 
> > > > diff --git a/cpus.c b/cpus.c
> > > > index c15ff6c..c347db5 100644
> > > > --- a/cpus.c
> > > > +++ b/cpus.c
> > > > @@ -993,6 +993,13 @@ void pause_all_vcpus(void)
> > > >      }
> > > >  }
> > > >  
> > > > +void resume_vcpu(CPUState *cpu)
> > > > +{
> > > > +    cpu->stop = false;
> > > > +    cpu->stopped = false;
> > > > +    qemu_cpu_kick(cpu);
> > > > +}
> > > > +
> > > >  void resume_all_vcpus(void)
> > > >  {
> > > >      CPUArchState *penv = first_cpu;
> > > > @@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
> > > >      qemu_clock_enable(vm_clock, true);
> > > >      while (penv) {
> > > >          CPUState *pcpu = ENV_GET_CPU(penv);
> > > > -        pcpu->stop = false;
> > > > -        pcpu->stopped = false;
> > > > -        qemu_cpu_kick(pcpu);
> > > > +        resume_vcpu(pcpu);
> > > >          penv = penv->next_cpu;
> > > >      }
> > > >  }
> > > > diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> > > > index 3664a1b..6ce6f10 100644
> > > > --- a/include/qom/cpu.h
> > > > +++ b/include/qom/cpu.h
> > > > @@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
> > > >   */
> > > >  void cpu_reset_interrupt(CPUState *cpu, int mask);
> > > >  
> > > > +/**
> > > > + * resume_vcpu:
> > > > + * @cpu: The CPU to resume.
> > > > + *
> > > > + * Resumes CPU, i.e. puts CPU into runable state.
> > > > + * KVM requires extra call cpu_synchronize_post_init() to make
> > > > + * KVM's CPU instance runable.
> > > > + */
> > > Can you elaborate on this a bit. Why is it needed and why is it relevant
> > > for this function?
> > If running in KVM mode and resume_vcpu() is called without
> > cpu_synchronize_post_init(), then CPU instance in KVM will be
> > left in UNCONFIGURED state. I've added comment here to document 
> > that resume_vcpu() isn't enough to make CPU run-able in case of KVM.
> > 
> But this really relevant only during vcpu creation after that
> resume_vcpu() should just work. My be have launch_vcpu() function that
Agreed, it's applicable only during CPU creation.

> does the right thing and add assert to resume_vcpu() if it is called on
> vcpu that is not launched yet. In other words made the dependency clear
> in the code, not comment.
Would dropping this comment, suffice
and re-factoring to use launch_vcpu() done later?

> --
> 			Gleb.

Re: [Qemu-devel] [PATCH 11/16] introduce ICC bus/device/bridge

[Paolo Bonzini]

Il 22/04/2013 13:39, Peter Maydell ha scritto:
>> > ... to provide hotplug-able bus.
>> >
>> > * icc-bridge will serve as a parent for icc-bus and provide
>> >   mmio mapping services to child icc-devices.
>> > * icc-device will replace SysBusDevice as a parent of APIC
>> >   and IOAPIC devices.
>> >
>> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
>> > ---
>> > v2:
>> >   * Rebase on top the last HW reorganization series.
>> >   * Move hw/icc_bus.c into hw/cpu/ and hw/icc_bus.h include/hw/i386/
> This seems inconsistent. I think the header should be in
> include/hw/cpu/ to match the location of the .c file.

Agreed.

Paolo

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

[Gleb Natapov]

On Mon, Apr 22, 2013 at 01:45:25PM +0200, Igor Mammedov wrote:
> On Mon, 22 Apr 2013 13:59:17 +0300
> Gleb Natapov <gleb@redhat.com> wrote:
> 
> > On Mon, Apr 22, 2013 at 12:54:04PM +0200, Igor Mammedov wrote:
> > > On Mon, 22 Apr 2013 13:40:02 +0300
> > > Gleb Natapov <gleb@redhat.com> wrote:
> > > 
> > > > On Tue, Apr 16, 2013 at 12:12:43AM +0200, Igor Mammedov wrote:
> > > > > ... also add stub for it, to make possible to use it in qom/cpu.c,
> > > > > which is shared with user emulators.
> > > > > 
> > > > > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > > > > ---
> > > > > Note:
> > > > >   it will be re-used from qom/cpu.c by hotpluged CPU
> > > > > ---
> > > > >  cpus.c              | 11 ++++++++---
> > > > >  include/qom/cpu.h   |  9 +++++++++
> > > > >  stubs/Makefile.objs |  1 +
> > > > >  stubs/cpus.c        |  5 +++++
> > > > >  4 files changed, 23 insertions(+), 3 deletions(-)
> > > > >  create mode 100644 stubs/cpus.c
> > > > > 
> > > > > diff --git a/cpus.c b/cpus.c
> > > > > index c15ff6c..c347db5 100644
> > > > > --- a/cpus.c
> > > > > +++ b/cpus.c
> > > > > @@ -993,6 +993,13 @@ void pause_all_vcpus(void)
> > > > >      }
> > > > >  }
> > > > >  
> > > > > +void resume_vcpu(CPUState *cpu)
> > > > > +{
> > > > > +    cpu->stop = false;
> > > > > +    cpu->stopped = false;
> > > > > +    qemu_cpu_kick(cpu);
> > > > > +}
> > > > > +
> > > > >  void resume_all_vcpus(void)
> > > > >  {
> > > > >      CPUArchState *penv = first_cpu;
> > > > > @@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
> > > > >      qemu_clock_enable(vm_clock, true);
> > > > >      while (penv) {
> > > > >          CPUState *pcpu = ENV_GET_CPU(penv);
> > > > > -        pcpu->stop = false;
> > > > > -        pcpu->stopped = false;
> > > > > -        qemu_cpu_kick(pcpu);
> > > > > +        resume_vcpu(pcpu);
> > > > >          penv = penv->next_cpu;
> > > > >      }
> > > > >  }
> > > > > diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> > > > > index 3664a1b..6ce6f10 100644
> > > > > --- a/include/qom/cpu.h
> > > > > +++ b/include/qom/cpu.h
> > > > > @@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
> > > > >   */
> > > > >  void cpu_reset_interrupt(CPUState *cpu, int mask);
> > > > >  
> > > > > +/**
> > > > > + * resume_vcpu:
> > > > > + * @cpu: The CPU to resume.
> > > > > + *
> > > > > + * Resumes CPU, i.e. puts CPU into runable state.
> > > > > + * KVM requires extra call cpu_synchronize_post_init() to make
> > > > > + * KVM's CPU instance runable.
> > > > > + */
> > > > Can you elaborate on this a bit. Why is it needed and why is it relevant
> > > > for this function?
> > > If running in KVM mode and resume_vcpu() is called without
> > > cpu_synchronize_post_init(), then CPU instance in KVM will be
> > > left in UNCONFIGURED state. I've added comment here to document 
> > > that resume_vcpu() isn't enough to make CPU run-able in case of KVM.
> > > 
> > But this really relevant only during vcpu creation after that
> > resume_vcpu() should just work. My be have launch_vcpu() function that
> Agreed, it's applicable only during CPU creation.
> 
> > does the right thing and add assert to resume_vcpu() if it is called on
> > vcpu that is not launched yet. In other words made the dependency clear
> > in the code, not comment.
> Would dropping this comment, suffice
> and re-factoring to use launch_vcpu() done later?
> 
Yes.

--
			Gleb.

Re: [Qemu-devel] [PATCH 11/16] introduce ICC bus/device/bridge

[Andreas Färber]

Am 16.04.2013 00:12, schrieb Igor Mammedov:
> ... to provide hotplug-able bus.
> 
> * icc-bridge will serve as a parent for icc-bus and provide
>   mmio mapping services to child icc-devices.
> * icc-device will replace SysBusDevice as a parent of APIC
>   and IOAPIC devices.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
> v2:
>   * Rebase on top the last HW reorganization series.
>   * Move hw/icc_bus.c into hw/cpu/ and hw/icc_bus.h include/hw/i386/

http://wiki.qemu.org/QOMConventions

Comments below...

> ---
>  default-configs/i386-softmmu.mak   |  1 +
>  default-configs/x86_64-softmmu.mak |  1 +
>  hw/cpu/Makefile.objs               |  1 +
>  hw/cpu/icc_bus.c                   | 88 ++++++++++++++++++++++++++++++++++++++
>  hw/i386/pc_piix.c                  |  7 +++
>  hw/i386/pc_q35.c                   |  7 +++
>  include/hw/i386/icc_bus.h          | 53 +++++++++++++++++++++++
>  7 files changed, 158 insertions(+)
>  create mode 100644 hw/cpu/icc_bus.c
>  create mode 100644 include/hw/i386/icc_bus.h
> 
> diff --git a/default-configs/i386-softmmu.mak b/default-configs/i386-softmmu.mak
> index 4ac0694..0789303 100644
> --- a/default-configs/i386-softmmu.mak
> +++ b/default-configs/i386-softmmu.mak
> @@ -45,3 +45,4 @@ CONFIG_LPC_ICH9=y
>  CONFIG_Q35=y
>  CONFIG_APIC=y
>  CONFIG_IOAPIC=y
> +CONFIG_ICC_BUS=y
> diff --git a/default-configs/x86_64-softmmu.mak b/default-configs/x86_64-softmmu.mak
> index bf4091c..8d03e9a 100644
> --- a/default-configs/x86_64-softmmu.mak
> +++ b/default-configs/x86_64-softmmu.mak
> @@ -45,3 +45,4 @@ CONFIG_LPC_ICH9=y
>  CONFIG_Q35=y
>  CONFIG_APIC=y
>  CONFIG_IOAPIC=y
> +CONFIG_ICC_BUS=y
> diff --git a/hw/cpu/Makefile.objs b/hw/cpu/Makefile.objs
> index a49ca04..4461ece 100644
> --- a/hw/cpu/Makefile.objs
> +++ b/hw/cpu/Makefile.objs
> @@ -1,4 +1,5 @@
>  obj-$(CONFIG_ARM11MPCORE) += arm11mpcore.o
>  obj-$(CONFIG_ARM9MPCORE) += a9mpcore.o
>  obj-$(CONFIG_ARM15MPCORE) += a15mpcore.o
> +obj-$(CONFIG_ICC_BUS) += icc_bus.o
>  
> diff --git a/hw/cpu/icc_bus.c b/hw/cpu/icc_bus.c
> new file mode 100644
> index 0000000..00b9be3
> --- /dev/null
> +++ b/hw/cpu/icc_bus.c
> @@ -0,0 +1,88 @@
> +/* icc_bus.c
> + * emulate x86 ICC(INTERRUPT CONTROLLER COMMUNICATIONS) bus

Does this need to be all-capitals? :)

> + *
> + * Copyright (c) 2013 Red Hat, Inc
> + *
> + * Authors:
> + *     Igor Mammedov <imammedo@redhat.com>
> + *
> + * This library is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2 of the License, or (at your option) any later version.
> + *
> + * This library is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with this library; if not, see <http://www.gnu.org/licenses/>
> + */
> +#include "hw/i386/icc_bus.h"
> +#include "hw/sysbus.h"
> +
> +static void icc_bus_initfn(Object *obj)
> +{
> +    BusState *b = BUS(obj);

Whiteline as below?

> +    b->allow_hotplug = true;
> +}
> +
> +static const TypeInfo icc_bus_info = {
> +    .name = TYPE_ICC_BUS,
> +    .parent = TYPE_BUS,
> +    .instance_size = sizeof(ICCBus),
> +    .instance_init = icc_bus_initfn,
> +};
> +
> +static int icc_device_init(DeviceState *dev)
> +{
> +    ICCDevice *id = ICC_DEVICE(dev);
> +    ICCDeviceClass *k = ICC_DEVICE_GET_CLASS(id);
> +
> +    return k->init(id);
> +}
> +
> +static void icc_device_class_init(ObjectClass *klass, void *data)
> +{
> +    DeviceClass *k = DEVICE_CLASS(klass);
> +
> +    k->init = icc_device_init;

Please use k->realize for any new base device type or type derived from
SysBusDevice.

If there's nothing to do at this level, you can just provide a dummy
realize function here to override Device's and override it (without
saving the parent's dummy function) in IOAPIC or wherever you need it.

> +    k->bus_type = TYPE_ICC_BUS;
> +}
> +
> +static const TypeInfo icc_device_info = {
> +    .name = TYPE_ICC_DEVICE,
> +    .parent = TYPE_DEVICE,
> +    .abstract = true,
> +    .instance_size = sizeof(ICCDevice),
> +    .class_size = sizeof(ICCDeviceClass),
> +    .class_init = icc_device_class_init,
> +};
> +
> +typedef struct ICCBridgeState {
> +    SysBusDevice busdev;

parent_obj please - makes it clear that it is a QOM struct and flags
accidental uses of obsolete SysBus macros.

> +} ICCBridgeState;
> +#define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
> +
> +
> +static void icc_bridge_initfn(Object *obj)
> +{
> +    qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus");

qbus_create_inplace()?

> +}
> +
> +static const TypeInfo icc_bridge_info = {
> +    .name  = "icc-bridge",

TYPE_ICC_BRIDGE?

> +    .parent = TYPE_SYS_BUS_DEVICE,
> +    .instance_init  = icc_bridge_initfn,
> +    .instance_size  = sizeof(ICCBridgeState),
> +};
> +
> +static void icc_bus_register_types(void)
> +{
> +    type_register_static(&icc_bus_info);
> +    type_register_static(&icc_device_info);
> +    type_register_static(&icc_bridge_info);
> +}
> +
> +type_init(icc_bus_register_types)

BTW Anthony recently suggested to drop the "fn" where there's no init
vs. initfn name conflicts; I don't mind either way.

> diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
> index cff8013..d8c3e1a 100644
> --- a/hw/i386/pc_piix.c
> +++ b/hw/i386/pc_piix.c
> @@ -37,6 +37,7 @@
>  #include "hw/kvm/clock.h"
>  #include "sysemu/sysemu.h"
>  #include "hw/sysbus.h"
> +#include "hw/i386/icc_bus.h"
>  #include "sysemu/arch_init.h"
>  #include "sysemu/blockdev.h"
>  #include "hw/i2c/smbus.h"
> @@ -84,8 +85,13 @@ static void pc_init1(MemoryRegion *system_memory,
>      MemoryRegion *ram_memory;
>      MemoryRegion *pci_memory;
>      MemoryRegion *rom_memory;
> +    DeviceState *icc_bridge;
>      void *fw_cfg = NULL;
>  
> +    icc_bridge = qdev_create(NULL, TYPE_ICC_BRIDGE);
> +    object_property_add_child(qdev_get_machine(), "icc-bridge",
> +                              OBJECT(icc_bridge), NULL);
> +
>      pc_cpus_init(cpu_model);
>      pc_acpi_init("acpi-dsdt.aml");
>  
> @@ -160,6 +166,7 @@ static void pc_init1(MemoryRegion *system_memory,
>      if (pci_enabled) {
>          ioapic_init_gsi(gsi_state, "i440fx");
>      }
> +    qdev_init_nofail(icc_bridge);
>  
>      pc_register_ferr_irq(gsi[13]);
>  
> diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
> index 6ac1a89..9055145 100644
> --- a/hw/i386/pc_q35.c
> +++ b/hw/i386/pc_q35.c
> @@ -41,6 +41,7 @@
>  #include "hw/ide/pci.h"
>  #include "hw/ide/ahci.h"
>  #include "hw/usb.h"
> +#include "hw/i386/icc_bus.h"
>  
>  /* ICH9 AHCI has 6 ports */
>  #define MAX_SATA_PORTS     6
> @@ -85,6 +86,11 @@ static void pc_q35_init(QEMUMachineInitArgs *args)
>      ICH9LPCState *ich9_lpc;
>      PCIDevice *ahci;
>      qemu_irq *cmos_s3;
> +    DeviceState *icc_bridge;
> +
> +    icc_bridge = qdev_create(NULL, TYPE_ICC_BRIDGE);
> +    object_property_add_child(qdev_get_machine(), "icc-bridge",
> +                              OBJECT(icc_bridge), NULL);
>  
>      pc_cpus_init(cpu_model);
>      pc_acpi_init("q35-acpi-dsdt.aml");
> @@ -168,6 +174,7 @@ static void pc_q35_init(QEMUMachineInitArgs *args)
>      if (pci_enabled) {
>          ioapic_init_gsi(gsi_state, NULL);
>      }
> +    qdev_init_nofail(icc_bridge);
>  
>      pc_register_ferr_irq(gsi[13]);
>  
> diff --git a/include/hw/i386/icc_bus.h b/include/hw/i386/icc_bus.h
> new file mode 100644
> index 0000000..aa30e0c
> --- /dev/null
> +++ b/include/hw/i386/icc_bus.h
> @@ -0,0 +1,53 @@
> +/* icc_bus.h
> + * emulate x86 ICC(INTERRUPT CONTROLLER COMMUNICATIONS) bus
> + *
> + * Copyright (c) 2013 Red Hat, Inc
> + *
> + * Authors:
> + *     Igor Mammedov <imammedo@redhat.com>
> + *
> + * This library is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2 of the License, or (at your option) any later version.
> + *
> + * This library is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with this library; if not, see <http://www.gnu.org/licenses/>
> + */
> +#ifndef ICC_BUS_H
> +#define ICC_BUS_H
> +
> +#include "hw/qdev-core.h"
> +
> +#define TYPE_ICC_BUS "icc-bus"
> +
> +#ifndef CONFIG_USER_ONLY
> +typedef struct ICCBus {

/*< private >*/

> +    BusState qbus;

parent_obj

/*< public >*/

...since it is in a header.

> +} ICCBus;
> +#define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
> +
> +typedef struct ICCDevice {
> +    DeviceState qdev;

dito

> +} ICCDevice;
> +
> +typedef struct ICCDeviceClass {
> +    DeviceClass parent_class;
> +    int (*init)(ICCDevice *dev);

Please read the realize documentation in qdev-core.h - instead of having
the device provide a specialized init hook, it should override (and if
necessary, save) the parent's realize callback.

> +} ICCDeviceClass;
> +#define TYPE_ICC_DEVICE "icc-device"
> +#define ICC_DEVICE(obj) OBJECT_CHECK(ICCDevice, (obj), TYPE_ICC_DEVICE)
> +#define ICC_DEVICE_CLASS(klass) \
> +     OBJECT_CLASS_CHECK(ICCDeviceClass, (klass), TYPE_ICC_DEVICE)
> +#define ICC_DEVICE_GET_CLASS(obj) \
> +     OBJECT_GET_CLASS(ICCDeviceClass, (obj), TYPE_ICC_DEVICE)
> +
> +#define TYPE_ICC_BRIDGE "icc-bridge"
> +
> +#endif /* CONFIG_USER_ONLY */
> +#endif

General concept looks fine to me. Maybe add a bit more whiteline spacing
between struct and macros and between different QOM types in .c.

Who becomes the MAINTAINER of ICC?

Regards,
Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

[Andreas Färber]

Am 22.04.2013 13:45, schrieb Igor Mammedov:
> On Mon, 22 Apr 2013 13:59:17 +0300
> Gleb Natapov <gleb@redhat.com> wrote:
> 
>> On Mon, Apr 22, 2013 at 12:54:04PM +0200, Igor Mammedov wrote:
>>> On Mon, 22 Apr 2013 13:40:02 +0300
>>> Gleb Natapov <gleb@redhat.com> wrote:
>>>
>>>> On Tue, Apr 16, 2013 at 12:12:43AM +0200, Igor Mammedov wrote:
>>>>> ... also add stub for it, to make possible to use it in qom/cpu.c,
>>>>> which is shared with user emulators.
>>>>>
>>>>> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
>>>>> ---
>>>>> Note:
>>>>>   it will be re-used from qom/cpu.c by hotpluged CPU
>>>>> ---
>>>>>  cpus.c              | 11 ++++++++---
>>>>>  include/qom/cpu.h   |  9 +++++++++
>>>>>  stubs/Makefile.objs |  1 +
>>>>>  stubs/cpus.c        |  5 +++++
>>>>>  4 files changed, 23 insertions(+), 3 deletions(-)
>>>>>  create mode 100644 stubs/cpus.c
>>>>>
>>>>> diff --git a/cpus.c b/cpus.c
>>>>> index c15ff6c..c347db5 100644
>>>>> --- a/cpus.c
>>>>> +++ b/cpus.c
>>>>> @@ -993,6 +993,13 @@ void pause_all_vcpus(void)
>>>>>      }
>>>>>  }
>>>>>  
>>>>> +void resume_vcpu(CPUState *cpu)
>>>>> +{
>>>>> +    cpu->stop = false;
>>>>> +    cpu->stopped = false;
>>>>> +    qemu_cpu_kick(cpu);
>>>>> +}
>>>>> +
>>>>>  void resume_all_vcpus(void)
>>>>>  {
>>>>>      CPUArchState *penv = first_cpu;
>>>>> @@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
>>>>>      qemu_clock_enable(vm_clock, true);
>>>>>      while (penv) {
>>>>>          CPUState *pcpu = ENV_GET_CPU(penv);
>>>>> -        pcpu->stop = false;
>>>>> -        pcpu->stopped = false;
>>>>> -        qemu_cpu_kick(pcpu);
>>>>> +        resume_vcpu(pcpu);
>>>>>          penv = penv->next_cpu;
>>>>>      }
>>>>>  }
>>>>> diff --git a/include/qom/cpu.h b/include/qom/cpu.h
>>>>> index 3664a1b..6ce6f10 100644
>>>>> --- a/include/qom/cpu.h
>>>>> +++ b/include/qom/cpu.h
>>>>> @@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
>>>>>   */
>>>>>  void cpu_reset_interrupt(CPUState *cpu, int mask);
>>>>>  
>>>>> +/**
>>>>> + * resume_vcpu:
>>>>> + * @cpu: The CPU to resume.
>>>>> + *
>>>>> + * Resumes CPU, i.e. puts CPU into runable state.
>>>>> + * KVM requires extra call cpu_synchronize_post_init() to make
>>>>> + * KVM's CPU instance runable.
>>>>> + */
>>>> Can you elaborate on this a bit. Why is it needed and why is it relevant
>>>> for this function?
>>> If running in KVM mode and resume_vcpu() is called without
>>> cpu_synchronize_post_init(), then CPU instance in KVM will be
>>> left in UNCONFIGURED state. I've added comment here to document 
>>> that resume_vcpu() isn't enough to make CPU run-able in case of KVM.
>>>
>> But this really relevant only during vcpu creation after that
>> resume_vcpu() should just work. My be have launch_vcpu() function that
> Agreed, it's applicable only during CPU creation.
> 
>> does the right thing and add assert to resume_vcpu() if it is called on
>> vcpu that is not launched yet. In other words made the dependency clear
>> in the code, not comment.
> Would dropping this comment, suffice
> and re-factoring to use launch_vcpu() done later?

Either way I had request cpu_... naming for CPUState functions on IRC.
Could be a follow-up cleanup.

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 01/16 v2] cpu: make kvm-stub.o a part of CPU library

[Andreas Färber]

Am 18.04.2013 16:24, schrieb Eduardo Habkost:
> On Thu, Apr 18, 2013 at 06:23:50AM +0200, Igor Mammedov wrote:
>> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> 
> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>

Looks good to me, too.

Paolo and Gleb, are you okay with this disguised v5?

Andreas

>> ---
>> v3:
>>   put ifndef around msi.h in kvm-stub.c instead of in msi.h
>>
>> v2:
>>   remove unnecessary kvm-stub.o from "System emulator target" section
>> ---
>>  Makefile.target      | 14 +++++++-------
>>  include/sysemu/kvm.h |  4 ++--
>>  kvm-stub.c           |  7 ++++++-
>>  3 files changed, 15 insertions(+), 10 deletions(-)
>>
>> diff --git a/Makefile.target b/Makefile.target
>> index 2bd6d14..9bde8e5 100644
>> --- a/Makefile.target
>> +++ b/Makefile.target
>> @@ -64,6 +64,12 @@ all: $(PROGS) stap
>>  # Dummy command so that make thinks it has done something
>>  	@true
>>  
>> +CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
>> +CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
>> +CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
>> +CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
>> +CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
>> +
>>  #########################################################
>>  # cpu emulator library
>>  obj-y = exec.o translate-all.o cpu-exec.o
>> @@ -74,6 +80,7 @@ obj-y += fpu/softfloat.o
>>  obj-y += target-$(TARGET_BASE_ARCH)/
>>  obj-y += disas.o
>>  obj-$(CONFIG_GDBSTUB_XML) += gdbstub-xml.o
>> +obj-$(CONFIG_NO_KVM) += kvm-stub.o
>>  
>>  #########################################################
>>  # Linux user emulator target
>> @@ -102,18 +109,11 @@ endif #CONFIG_BSD_USER
>>  #########################################################
>>  # System emulator target
>>  ifdef CONFIG_SOFTMMU
>> -CONFIG_NO_PCI = $(if $(subst n,,$(CONFIG_PCI)),n,y)
>> -CONFIG_NO_KVM = $(if $(subst n,,$(CONFIG_KVM)),n,y)
>> -CONFIG_NO_XEN = $(if $(subst n,,$(CONFIG_XEN)),n,y)
>> -CONFIG_NO_GET_MEMORY_MAPPING = $(if $(subst n,,$(CONFIG_HAVE_GET_MEMORY_MAPPING)),n,y)
>> -CONFIG_NO_CORE_DUMP = $(if $(subst n,,$(CONFIG_HAVE_CORE_DUMP)),n,y)
>> -
>>  obj-y += arch_init.o cpus.o monitor.o gdbstub.o balloon.o ioport.o
>>  obj-y += qtest.o
>>  obj-y += hw/
>>  obj-$(CONFIG_FDT) += device_tree.o
>>  obj-$(CONFIG_KVM) += kvm-all.o
>> -obj-$(CONFIG_NO_KVM) += kvm-stub.o
>>  obj-y += memory.o savevm.o cputlb.o
>>  obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
>>  obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
>> diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
>> index 495e6f8..93cef28 100644
>> --- a/include/sysemu/kvm.h
>> +++ b/include/sysemu/kvm.h
>> @@ -144,10 +144,10 @@ int kvm_cpu_exec(CPUArchState *env);
>>  #if !defined(CONFIG_USER_ONLY)
>>  void *kvm_vmalloc(ram_addr_t size);
>>  void *kvm_arch_vmalloc(ram_addr_t size);
>> -void kvm_setup_guest_memory(void *start, size_t size);
>> +#endif
>>  
>> +void kvm_setup_guest_memory(void *start, size_t size);
>>  void kvm_flush_coalesced_mmio_buffer(void);
>> -#endif
>>  
>>  int kvm_insert_breakpoint(CPUArchState *current_env, target_ulong addr,
>>                            target_ulong len, int type);
>> diff --git a/kvm-stub.c b/kvm-stub.c
>> index 3e5c201..e5d2906 100644
>> --- a/kvm-stub.c
>> +++ b/kvm-stub.c
>> @@ -12,11 +12,14 @@
>>  
>>  #include "qemu-common.h"
>>  #include "hw/hw.h"
>> -#include "hw/pci/msi.h"
>>  #include "cpu.h"
>>  #include "exec/gdbstub.h"
>>  #include "sysemu/kvm.h"
>>  
>> +#ifndef CONFIG_USER_ONLY
>> +#include "hw/pci/msi.h"
>> +#endif
>> +
>>  KVMState *kvm_state;
>>  bool kvm_kernel_irqchip;
>>  bool kvm_async_interrupts_allowed;
>> @@ -122,6 +125,7 @@ int kvm_on_sigbus(int code, void *addr)
>>      return 1;
>>  }
>>  
>> +#ifndef CONFIG_USER_ONLY
>>  int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
>>  {
>>      return -ENOSYS;
>> @@ -145,3 +149,4 @@ int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq)
>>  {
>>      return -ENOSYS;
>>  }
>> +#endif
>> -- 
>> 1.8.2
>>
> 


-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 02/16] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

[Andreas Färber]

Am 16.04.2013 00:12, schrieb Igor Mammedov:
> ... to synchronize CPU state to KVM
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
> v2:
>   * linking kvm-stub.o to *-user target moved in separate patch
> ---
>  include/sysemu/kvm.h | 18 ++++++++++--------
>  kvm-all.c            |  1 +
>  kvm-stub.c           |  1 +
>  qom/cpu.c            |  4 ++++
>  vl.c                 |  1 -
>  5 files changed, 16 insertions(+), 9 deletions(-)

Looks okay to me. Gleb, can you ack?

Andreas

> diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
> index 93cef28..8fb2489 100644
> --- a/include/sysemu/kvm.h
> +++ b/include/sysemu/kvm.h
> @@ -250,8 +250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension);
>  uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
>                                        uint32_t index, int reg);
>  void kvm_cpu_synchronize_state(CPUArchState *env);
> -void kvm_cpu_synchronize_post_reset(CPUState *cpu);
> -void kvm_cpu_synchronize_post_init(CPUState *cpu);
>  
>  /* generic hooks - to be moved/refactored once there are more users */
>  
> @@ -262,6 +260,16 @@ static inline void cpu_synchronize_state(CPUArchState *env)
>      }
>  }
>  
> +#if !defined(CONFIG_USER_ONLY)
> +int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
> +                                       hwaddr *phys_addr);
> +#endif
> +
> +#endif /* NEED_CPU_H */
> +
> +void kvm_cpu_synchronize_post_reset(CPUState *cpu);
> +void kvm_cpu_synchronize_post_init(CPUState *cpu);
> +
>  static inline void cpu_synchronize_post_reset(CPUState *cpu)
>  {
>      if (kvm_enabled()) {
> @@ -277,12 +285,6 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
>  }
>  
>  
> -#if !defined(CONFIG_USER_ONLY)
> -int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
> -                                       hwaddr *phys_addr);
> -#endif
> -
> -#endif
>  int kvm_set_ioeventfd_mmio(int fd, uint32_t adr, uint32_t val, bool assign,
>                             uint32_t size);
>  
> diff --git a/kvm-all.c b/kvm-all.c
> index fc4e17c..1d17128 100644
> --- a/kvm-all.c
> +++ b/kvm-all.c
> @@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
>  bool kvm_irqfds_allowed;
>  bool kvm_msi_via_irqfd_allowed;
>  bool kvm_gsi_routing_allowed;
> +bool kvm_allowed;
>  
>  static const KVMCapabilityInfo kvm_required_capabilites[] = {
>      KVM_CAP_INFO(USER_MEMORY),
> diff --git a/kvm-stub.c b/kvm-stub.c
> index a131305..4282e09 100644
> --- a/kvm-stub.c
> +++ b/kvm-stub.c
> @@ -23,6 +23,7 @@ bool kvm_async_interrupts_allowed;
>  bool kvm_irqfds_allowed;
>  bool kvm_msi_via_irqfd_allowed;
>  bool kvm_gsi_routing_allowed;
> +bool kvm_allowed;
>  
>  int kvm_init_vcpu(CPUState *cpu)
>  {
> diff --git a/qom/cpu.c b/qom/cpu.c
> index e242dcb..0c76712 100644
> --- a/qom/cpu.c
> +++ b/qom/cpu.c
> @@ -20,6 +20,7 @@
>  
>  #include "qom/cpu.h"
>  #include "qemu-common.h"
> +#include "sysemu/kvm.h"
>  
>  void cpu_reset_interrupt(CPUState *cpu, int mask)
>  {
> @@ -57,6 +58,9 @@ static ObjectClass *cpu_common_class_by_name(const char *cpu_model)
>  
>  static void cpu_common_realizefn(DeviceState *dev, Error **errp)
>  {
> +    if (dev->hotplugged) {
> +        cpu_synchronize_post_init(CPU(dev));
> +    }
>  }
>  
>  static void cpu_class_init(ObjectClass *klass, void *data)
> diff --git a/vl.c b/vl.c
> index dd94549..bc9c016 100644
> --- a/vl.c
> +++ b/vl.c
> @@ -267,7 +267,6 @@ static NotifierList machine_init_done_notifiers =
>      NOTIFIER_LIST_INITIALIZER(machine_init_done_notifiers);
>  
>  static bool tcg_allowed = true;
> -bool kvm_allowed;
>  bool xen_allowed;
>  uint32_t xen_domid;
>  enum xen_mode xen_mode = XEN_EMULATE;
> 


-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 10/16] target-i386: introduce apic-id property

[Andreas Färber]

Am 16.04.2013 00:12, schrieb Igor Mammedov:
> ... and use it from board level to set APIC ID for CPUs
> it creates.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
> Note:
>   * pc_new_cpu() function will be reused later in CPU hot-plug hook.

Suggest as main commit message to avoid the "...":

The property is used from board level to set APIC ID for CPUs it
creates. Do so in a new pc_new_cpu() helper, to be reused for hot-plug.

> 
> v3:
>   * user error_propagate() in property setter
> v2:
>   * use generic cpu_exists() instead of custom one
>   * make apic-id dynamic property, so it won't be possible to use it
>     as global property, since it's instance specific
> ---
>  hw/i386/pc.c      | 25 ++++++++++++++++++++++++-
>  target-i386/cpu.c | 42 ++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 66 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index dc1a78b..cb57878 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -889,9 +889,29 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
>      }
>  }
>  
> +static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
> +{
> +    X86CPU *cpu;
> +
> +    cpu = cpu_x86_create(cpu_model, errp);
> +    if (!cpu) {
> +        return;
> +    }
> +
> +    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
> +    object_property_set_bool(OBJECT(cpu), true, "realized", errp);
> +
> +    if (error_is_set(errp)) {

Please use a local Error* variable with error_propagate, otherwise this
is fragile.

> +        if (cpu != NULL) {
> +            object_unref(OBJECT(cpu));
> +        }
> +    }
> +}
> +
>  void pc_cpus_init(const char *cpu_model)
>  {
>      int i;
> +    Error *error = NULL;
>  
>      /* init CPUs */
>      if (cpu_model == NULL) {
> @@ -903,7 +923,10 @@ void pc_cpus_init(const char *cpu_model)
>      }
>  
>      for (i = 0; i < smp_cpus; i++) {
> -        if (!cpu_x86_init(cpu_model)) {
> +        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
> +        if (error) {
> +            fprintf(stderr, "%s\n", error_get_pretty(error));
> +            error_free(error);
>              exit(1);
>          }
>      }
> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> index a415fa3..6d6c527 100644
> --- a/target-i386/cpu.c
> +++ b/target-i386/cpu.c
> @@ -1271,6 +1271,45 @@ static void x86_cpuid_set_tsc_freq(Object *obj, Visitor *v, void *opaque,
>      cpu->env.tsc_khz = value / 1000;
>  }
>  
> +static void x86_cpuid_get_apic_id(Object *obj, Visitor *v, void *opaque,
> +                                  const char *name, Error **errp)
> +{
> +    X86CPU *cpu = X86_CPU(obj);
> +    int64_t value = cpu->env.cpuid_apic_id;
> +
> +    visit_type_int(v, &value, name, errp);
> +}
> +
> +static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
> +                                  const char *name, Error **errp)
> +{
> +    X86CPU *cpu = X86_CPU(obj);
> +    const int64_t min = 0;
> +    const int64_t max = UINT32_MAX;
> +    Error *error = NULL;
> +    int64_t value;
> +
> +    visit_type_int(v, &value, name, &error);
> +    if (error) {
> +        error_propagate(errp, error);
> +        return;
> +    }

> +    if (value < min || value > max) {
> +        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
> +                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
> +                   object_get_typename(obj), name, value, min, max);
> +        error_propagate(errp, error);
> +        return;
> +    }
> +
> +    if (cpu_exists(value)) {
> +        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
> +        error_propagate(errp, error);

You could do both error_setg()s directly on errp.

> +        return;
> +    }
> +    cpu->env.cpuid_apic_id = value;
> +}
> +
>  static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, const char *name)
>  {
>      x86_def_t *def;
> @@ -2259,6 +2298,9 @@ static void x86_cpu_initfn(Object *obj)
>      object_property_add(obj, "tsc-frequency", "int",
>                          x86_cpuid_get_tsc_freq,
>                          x86_cpuid_set_tsc_freq, NULL, NULL, NULL);
> +    object_property_add(obj, "apic-id", "int",
> +                        x86_cpuid_get_apic_id,
> +                        x86_cpuid_set_apic_id, NULL, NULL, NULL);
>  
>      env->cpuid_apic_id = x86_cpu_apic_id_from_index(cs->cpu_index);
>  

Otherwise I like this x86-specific version now!

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 01/16 v2] cpu: make kvm-stub.o a part of CPU library

[Paolo Bonzini]

Il 22/04/2013 15:45, Andreas Färber ha scritto:
> Am 18.04.2013 16:24, schrieb Eduardo Habkost:
>> > On Thu, Apr 18, 2013 at 06:23:50AM +0200, Igor Mammedov wrote:
>>> >> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
>> > 
>> > Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
> Looks good to me, too.
> 
> Paolo and Gleb, are you okay with this disguised v5?

I am fine with it too.

Paolo

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

[Igor Mammedov]

On Mon, 22 Apr 2013 15:42:09 +0200
Andreas Färber <afaerber@suse.de> wrote:

> Am 22.04.2013 13:45, schrieb Igor Mammedov:
> > On Mon, 22 Apr 2013 13:59:17 +0300
> > Gleb Natapov <gleb@redhat.com> wrote:
> > 
> >> On Mon, Apr 22, 2013 at 12:54:04PM +0200, Igor Mammedov wrote:
> >>> On Mon, 22 Apr 2013 13:40:02 +0300
> >>> Gleb Natapov <gleb@redhat.com> wrote:
> >>>
> >>>> On Tue, Apr 16, 2013 at 12:12:43AM +0200, Igor Mammedov wrote:
> >>>>> ... also add stub for it, to make possible to use it in qom/cpu.c,
> >>>>> which is shared with user emulators.
> >>>>>
> >>>>> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> >>>>> ---
> >>>>> Note:
> >>>>>   it will be re-used from qom/cpu.c by hotpluged CPU
> >>>>> ---
> >>>>>  cpus.c              | 11 ++++++++---
> >>>>>  include/qom/cpu.h   |  9 +++++++++
> >>>>>  stubs/Makefile.objs |  1 +
> >>>>>  stubs/cpus.c        |  5 +++++
> >>>>>  4 files changed, 23 insertions(+), 3 deletions(-)
> >>>>>  create mode 100644 stubs/cpus.c
> >>>>>
> >>>>> diff --git a/cpus.c b/cpus.c
> >>>>> index c15ff6c..c347db5 100644
> >>>>> --- a/cpus.c
> >>>>> +++ b/cpus.c
> >>>>> @@ -993,6 +993,13 @@ void pause_all_vcpus(void)
> >>>>>      }
> >>>>>  }
> >>>>>  
> >>>>> +void resume_vcpu(CPUState *cpu)
> >>>>> +{
> >>>>> +    cpu->stop = false;
> >>>>> +    cpu->stopped = false;
> >>>>> +    qemu_cpu_kick(cpu);
> >>>>> +}
> >>>>> +
> >>>>>  void resume_all_vcpus(void)
> >>>>>  {
> >>>>>      CPUArchState *penv = first_cpu;
> >>>>> @@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
> >>>>>      qemu_clock_enable(vm_clock, true);
> >>>>>      while (penv) {
> >>>>>          CPUState *pcpu = ENV_GET_CPU(penv);
> >>>>> -        pcpu->stop = false;
> >>>>> -        pcpu->stopped = false;
> >>>>> -        qemu_cpu_kick(pcpu);
> >>>>> +        resume_vcpu(pcpu);
> >>>>>          penv = penv->next_cpu;
> >>>>>      }
> >>>>>  }
> >>>>> diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> >>>>> index 3664a1b..6ce6f10 100644
> >>>>> --- a/include/qom/cpu.h
> >>>>> +++ b/include/qom/cpu.h
> >>>>> @@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
> >>>>>   */
> >>>>>  void cpu_reset_interrupt(CPUState *cpu, int mask);
> >>>>>  
> >>>>> +/**
> >>>>> + * resume_vcpu:
> >>>>> + * @cpu: The CPU to resume.
> >>>>> + *
> >>>>> + * Resumes CPU, i.e. puts CPU into runable state.
> >>>>> + * KVM requires extra call cpu_synchronize_post_init() to make
> >>>>> + * KVM's CPU instance runable.
> >>>>> + */
> >>>> Can you elaborate on this a bit. Why is it needed and why is it
> >>>> relevant for this function?
> >>> If running in KVM mode and resume_vcpu() is called without
> >>> cpu_synchronize_post_init(), then CPU instance in KVM will be
> >>> left in UNCONFIGURED state. I've added comment here to document 
> >>> that resume_vcpu() isn't enough to make CPU run-able in case of KVM.
> >>>
> >> But this really relevant only during vcpu creation after that
> >> resume_vcpu() should just work. My be have launch_vcpu() function that
> > Agreed, it's applicable only during CPU creation.
> > 
> >> does the right thing and add assert to resume_vcpu() if it is called on
> >> vcpu that is not launched yet. In other words made the dependency clear
> >> in the code, not comment.
> > Would dropping this comment, suffice
> > and re-factoring to use launch_vcpu() done later?
> 
> Either way I had request cpu_... naming for CPUState functions on IRC.
> Could be a follow-up cleanup.
How about cpu_resume(), is it suitable?

> 
> Andreas
> 

Re: [Qemu-devel] [PATCH 06/16] target-i386: pc: update rtc_cmos on CPU hot-plug

[Andreas Färber]

Am 16.04.2013 00:12, schrieb Igor Mammedov:
> ... so that on reboot BIOS could read current available CPU count
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
>  hw/i386/pc.c                   | 20 ++++++++++++++++++++
>  hw/timer/mc146818rtc.c         |  7 +++++++
>  include/hw/timer/mc146818rtc.h |  1 +
>  3 files changed, 28 insertions(+)
> 
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index 8d75b34..dc1a78b 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -337,6 +337,21 @@ static void pc_cmos_init_late(void *opaque)
>      qemu_unregister_reset(pc_cmos_init_late, opaque);
>  }
>  
> +typedef struct rtc_cpu_hotplug_arg {
> +    ISADevice *rtc_state;
> +    Notifier cpu_added_notifier;
> +} rtc_cpu_hotplug_arg;

Are the arguments (rtc_state) intentionally placed before the common part?

> +
> +static void rtc_notify_cpu_added(Notifier *notifier, void *data)
> +{
> +    rtc_cpu_hotplug_arg *arg = container_of(notifier, rtc_cpu_hotplug_arg,
> +                                           cpu_added_notifier);
> +    ISADevice *s = arg->rtc_state;
> +
> +    /* increment the number of CPUs */
> +    rtc_set_memory(s, 0x5f, rtc_get_memory(s, 0x5f) + 1);
> +}

I think this proves Gleb's point of reordering the notifier and CPU
resumption since the first instruction executed might theoretically be
reading the RTC memory.

Otherwise looks good with one minor nit...

> +
>  void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
>                    const char *boot_device,
>                    ISADevice *floppy, BusState *idebus0, BusState *idebus1,
> @@ -345,6 +360,7 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
>      int val, nb, i;
>      FDriveType fd_type[2] = { FDRIVE_DRV_NONE, FDRIVE_DRV_NONE };
>      static pc_cmos_init_late_arg arg;
> +    static rtc_cpu_hotplug_arg cpu_hotplug_cb;
>  
>      /* various important CMOS locations needed by PC/Bochs bios */
>  
> @@ -383,6 +399,10 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
>  
>      /* set the number of CPU */
>      rtc_set_memory(s, 0x5f, smp_cpus - 1);
> +    /* init CPU hotplug notifier */
> +    cpu_hotplug_cb.rtc_state = s;
> +    cpu_hotplug_cb.cpu_added_notifier.notify = rtc_notify_cpu_added;
> +    qemu_register_cpu_added_notifier(&cpu_hotplug_cb.cpu_added_notifier);
>  
>      /* set boot devices, and disable floppy signature check if requested */
>      if (set_boot_dev(s, boot_device, fd_bootchk)) {
> diff --git a/hw/timer/mc146818rtc.c b/hw/timer/mc146818rtc.c
> index 69e6844..e639942 100644
> --- a/hw/timer/mc146818rtc.c
> +++ b/hw/timer/mc146818rtc.c
> @@ -677,6 +677,13 @@ void rtc_set_memory(ISADevice *dev, int addr, int val)
>          s->cmos_data[addr] = val;
>  }
>  
> +int rtc_get_memory(ISADevice *dev, int addr)
> +{
> +    RTCState *s = DO_UPCAST(RTCState, dev, dev);

If we apply my mc146818rtc.c QOM'ification patch to qom-cpu first, this
can become MC146818_RTC(dev). Anthony just ack'ed that, but qemu.git is
currently broken for testing, so I'm still holding off changes.

Andreas

> +    assert(addr >= 0 && addr <= 127);
> +    return s->cmos_data[addr];
> +}
> +
>  static void rtc_set_date_from_host(ISADevice *dev)
>  {
>      RTCState *s = DO_UPCAST(RTCState, dev, dev);
> diff --git a/include/hw/timer/mc146818rtc.h b/include/hw/timer/mc146818rtc.h
> index 854ea3f..09f37b7 100644
> --- a/include/hw/timer/mc146818rtc.h
> +++ b/include/hw/timer/mc146818rtc.h
> @@ -6,6 +6,7 @@
>  
>  ISADevice *rtc_init(ISABus *bus, int base_year, qemu_irq intercept_irq);
>  void rtc_set_memory(ISADevice *dev, int addr, int val);
> +int rtc_get_memory(ISADevice *dev, int addr);
>  void rtc_set_date(ISADevice *dev, const struct tm *tm);
>  
>  #endif /* !MC146818RTC_H */

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 03/16] introduce resume_vcpu(), for single CPU

[Andreas Färber]

Am 22.04.2013 16:50, schrieb Igor Mammedov:
> On Mon, 22 Apr 2013 15:42:09 +0200
> Andreas Färber <afaerber@suse.de> wrote:
> 
>> Am 22.04.2013 13:45, schrieb Igor Mammedov:
>>> On Mon, 22 Apr 2013 13:59:17 +0300
>>> Gleb Natapov <gleb@redhat.com> wrote:
>>>
>>>> On Mon, Apr 22, 2013 at 12:54:04PM +0200, Igor Mammedov wrote:
>>>>> On Mon, 22 Apr 2013 13:40:02 +0300
>>>>> Gleb Natapov <gleb@redhat.com> wrote:
>>>>>
>>>>>> On Tue, Apr 16, 2013 at 12:12:43AM +0200, Igor Mammedov wrote:
>>>>>>> ... also add stub for it, to make possible to use it in qom/cpu.c,
>>>>>>> which is shared with user emulators.
>>>>>>>
>>>>>>> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
>>>>>>> ---
>>>>>>> Note:
>>>>>>>   it will be re-used from qom/cpu.c by hotpluged CPU
>>>>>>> ---
>>>>>>>  cpus.c              | 11 ++++++++---
>>>>>>>  include/qom/cpu.h   |  9 +++++++++
>>>>>>>  stubs/Makefile.objs |  1 +
>>>>>>>  stubs/cpus.c        |  5 +++++
>>>>>>>  4 files changed, 23 insertions(+), 3 deletions(-)
>>>>>>>  create mode 100644 stubs/cpus.c
>>>>>>>
>>>>>>> diff --git a/cpus.c b/cpus.c
>>>>>>> index c15ff6c..c347db5 100644
>>>>>>> --- a/cpus.c
>>>>>>> +++ b/cpus.c
>>>>>>> @@ -993,6 +993,13 @@ void pause_all_vcpus(void)
>>>>>>>      }
>>>>>>>  }
>>>>>>>  
>>>>>>> +void resume_vcpu(CPUState *cpu)
>>>>>>> +{
>>>>>>> +    cpu->stop = false;
>>>>>>> +    cpu->stopped = false;
>>>>>>> +    qemu_cpu_kick(cpu);
>>>>>>> +}
>>>>>>> +
>>>>>>>  void resume_all_vcpus(void)
>>>>>>>  {
>>>>>>>      CPUArchState *penv = first_cpu;
>>>>>>> @@ -1000,9 +1007,7 @@ void resume_all_vcpus(void)
>>>>>>>      qemu_clock_enable(vm_clock, true);
>>>>>>>      while (penv) {
>>>>>>>          CPUState *pcpu = ENV_GET_CPU(penv);
>>>>>>> -        pcpu->stop = false;
>>>>>>> -        pcpu->stopped = false;
>>>>>>> -        qemu_cpu_kick(pcpu);
>>>>>>> +        resume_vcpu(pcpu);
>>>>>>>          penv = penv->next_cpu;
>>>>>>>      }
>>>>>>>  }
>>>>>>> diff --git a/include/qom/cpu.h b/include/qom/cpu.h
>>>>>>> index 3664a1b..6ce6f10 100644
>>>>>>> --- a/include/qom/cpu.h
>>>>>>> +++ b/include/qom/cpu.h
>>>>>>> @@ -256,5 +256,14 @@ void cpu_interrupt(CPUState *cpu, int mask);
>>>>>>>   */
>>>>>>>  void cpu_reset_interrupt(CPUState *cpu, int mask);
>>>>>>>  
>>>>>>> +/**
>>>>>>> + * resume_vcpu:
>>>>>>> + * @cpu: The CPU to resume.
>>>>>>> + *
>>>>>>> + * Resumes CPU, i.e. puts CPU into runable state.
>>>>>>> + * KVM requires extra call cpu_synchronize_post_init() to make
>>>>>>> + * KVM's CPU instance runable.
>>>>>>> + */
>>>>>> Can you elaborate on this a bit. Why is it needed and why is it
>>>>>> relevant for this function?
>>>>> If running in KVM mode and resume_vcpu() is called without
>>>>> cpu_synchronize_post_init(), then CPU instance in KVM will be
>>>>> left in UNCONFIGURED state. I've added comment here to document 
>>>>> that resume_vcpu() isn't enough to make CPU run-able in case of KVM.
>>>>>
>>>> But this really relevant only during vcpu creation after that
>>>> resume_vcpu() should just work. My be have launch_vcpu() function that
>>> Agreed, it's applicable only during CPU creation.
>>>
>>>> does the right thing and add assert to resume_vcpu() if it is called on
>>>> vcpu that is not launched yet. In other words made the dependency clear
>>>> in the code, not comment.
>>> Would dropping this comment, suffice
>>> and re-factoring to use launch_vcpu() done later?
>>
>> Either way I had request cpu_... naming for CPUState functions on IRC.
>> Could be a follow-up cleanup.
> How about cpu_resume(), is it suitable?

Sure, that matches the scheme.

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 12/16] target-i386: cpu: attach ICC bus to CPU on its creation

[Andreas Färber]

Am 16.04.2013 00:12, schrieb Igor Mammedov:
> ... during startup,  so it would be possible to unplug it later
> and set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add
> attach hotplugged CPU to ICC bus.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>

Reviewed-by: Andreas Färber <afaerber@suse.de>

But still one question...

> ---
>  target-i386/cpu.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> index 6d6c527..3b5f90b 100644
> --- a/target-i386/cpu.c
> +++ b/target-i386/cpu.c
> @@ -41,6 +41,7 @@
>  #endif
>  
>  #include "sysemu/sysemu.h"
> +#include "hw/i386/icc_bus.h"
>  #ifndef CONFIG_USER_ONLY
>  #include "hw/xen/xen.h"
>  #include "hw/sysbus.h"
> @@ -1609,6 +1610,7 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
>      gchar **model_pieces;
>      char *name, *features;
>      Error *error = NULL;
> +    Object *icc_bus = object_resolve_path_type("icc-bus", TYPE_ICC_BUS, NULL);
>  
>      model_pieces = g_strsplit(cpu_model, ",", 2);
>      if (!model_pieces[0]) {
> @@ -1619,6 +1621,10 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error **errp)
>      features = model_pieces[1];
>  
>      cpu = X86_CPU(object_new(TYPE_X86_CPU));
> +    if (icc_bus) {
> +        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
> +        object_unref(OBJECT(cpu));
> +    }
>      env = &cpu->env;
>      env->cpu_model_str = cpu_model;
>  

You seem to be avoiding making lack of icc-bus an error although you add
it for both PC and q35 PC - which non-ICC use cases are you thinking of?

Andreas

> @@ -2330,6 +2336,7 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
>  
>      xcc->parent_realize = dc->realize;
>      dc->realize = x86_cpu_realizefn;
> +    dc->bus_type = TYPE_ICC_BUS;
>  
>      xcc->parent_reset = cc->reset;
>      cc->reset = x86_cpu_reset;
> 


-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 13/16] target-i386: replace MSI_SPACE_SIZE with APIC_SPACE_SIZE

[Andreas Färber]

Am 16.04.2013 00:12, schrieb Igor Mammedov:
> ... and put APIC_SPACE_SIZE in public header so that it could be
> reused later elsewhere.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>

Reviewed-by: Andreas Färber <afaerber@suse.de>

Andreas

> ---
>  hw/i386/kvm/apic.c              | 2 +-
>  hw/intc/apic.c                  | 2 +-
>  hw/xen/xen_apic.c               | 2 +-
>  include/hw/i386/apic_internal.h | 2 --
>  target-i386/cpu.h               | 1 +
>  5 files changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/hw/i386/kvm/apic.c b/hw/i386/kvm/apic.c
> index c6ff982..8f80425 100644
> --- a/hw/i386/kvm/apic.c
> +++ b/hw/i386/kvm/apic.c
> @@ -174,7 +174,7 @@ static const MemoryRegionOps kvm_apic_io_ops = {
>  static void kvm_apic_init(APICCommonState *s)
>  {
>      memory_region_init_io(&s->io_memory, &kvm_apic_io_ops, s, "kvm-apic-msi",
> -                          MSI_SPACE_SIZE);
> +                          APIC_SPACE_SIZE);
>  
>      if (kvm_has_gsi_routing()) {
>          msi_supported = true;
> diff --git a/hw/intc/apic.c b/hw/intc/apic.c
> index 2d79a9e..756dff0 100644
> --- a/hw/intc/apic.c
> +++ b/hw/intc/apic.c
> @@ -874,7 +874,7 @@ static const MemoryRegionOps apic_io_ops = {
>  static void apic_init(APICCommonState *s)
>  {
>      memory_region_init_io(&s->io_memory, &apic_io_ops, s, "apic-msi",
> -                          MSI_SPACE_SIZE);
> +                          APIC_SPACE_SIZE);
>  
>      s->timer = qemu_new_timer_ns(vm_clock, apic_timer, s);
>      local_apics[s->idx] = s;
> diff --git a/hw/xen/xen_apic.c b/hw/xen/xen_apic.c
> index a2eb8a1..a958782 100644
> --- a/hw/xen/xen_apic.c
> +++ b/hw/xen/xen_apic.c
> @@ -39,7 +39,7 @@ static const MemoryRegionOps xen_apic_io_ops = {
>  static void xen_apic_init(APICCommonState *s)
>  {
>      memory_region_init_io(&s->io_memory, &xen_apic_io_ops, s, "xen-apic-msi",
> -                          MSI_SPACE_SIZE);
> +                          APIC_SPACE_SIZE);
>  
>  #if defined(CONFIG_XEN_CTRL_INTERFACE_VERSION) \
>      && CONFIG_XEN_CTRL_INTERFACE_VERSION >= 420
> diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
> index 578241f..aac6290 100644
> --- a/include/hw/i386/apic_internal.h
> +++ b/include/hw/i386/apic_internal.h
> @@ -66,8 +66,6 @@
>  
>  #define MAX_APICS 255
>  
> -#define MSI_SPACE_SIZE                  0x100000
> -
>  typedef struct APICCommonState APICCommonState;
>  
>  #define TYPE_APIC_COMMON "apic-common"
> diff --git a/target-i386/cpu.h b/target-i386/cpu.h
> index cf1b05c..65cf97d 100644
> --- a/target-i386/cpu.h
> +++ b/target-i386/cpu.h
> @@ -1269,5 +1269,6 @@ uint32_t x86_cpu_apic_id_from_index(unsigned int cpu_index);
>  void enable_compat_apic_id_mode(void);
>  
>  #define APIC_DEFAULT_ADDRESS 0xfee00000
> +#define APIC_SPACE_SIZE      0x100000
>  
>  #endif /* CPU_I386_H */
> 


-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 11/16] introduce ICC bus/device/bridge

[Igor Mammedov]

On Mon, 22 Apr 2013 15:22:30 +0200
Andreas Färber <afaerber@suse.de> wrote:

> Am 16.04.2013 00:12, schrieb Igor Mammedov:
> > ... to provide hotplug-able bus.
> > 
> > * icc-bridge will serve as a parent for icc-bus and provide
> >   mmio mapping services to child icc-devices.
> > * icc-device will replace SysBusDevice as a parent of APIC
> >   and IOAPIC devices.
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > ---
> > v2:
> >   * Rebase on top the last HW reorganization series.
> >   * Move hw/icc_bus.c into hw/cpu/ and hw/icc_bus.h include/hw/i386/
> 
> http://wiki.qemu.org/QOMConventions
> 
> Comments below...
> 
[...]
> > +
> > +static void icc_device_class_init(ObjectClass *klass, void *data)
> > +{
> > +    DeviceClass *k = DEVICE_CLASS(klass);
> > +
> > +    k->init = icc_device_init;
> 
> Please use k->realize for any new base device type or type derived from
> SysBusDevice.
> 
> If there's nothing to do at this level, you can just provide a dummy
> realize function here to override Device's and override it (without
> saving the parent's dummy function) in IOAPIC or wherever you need it.
>
I'll try it, and if changes isn't big, I'll merge into this patch.

> > +    k->bus_type = TYPE_ICC_BUS;
> > +}
> > +
> > +static const TypeInfo icc_device_info = {
> > +    .name = TYPE_ICC_DEVICE,
> > +    .parent = TYPE_DEVICE,
> > +    .abstract = true,
> > +    .instance_size = sizeof(ICCDevice),
> > +    .class_size = sizeof(ICCDeviceClass),
> > +    .class_init = icc_device_class_init,
> > +};
> > +
> > +typedef struct ICCBridgeState {
> > +    SysBusDevice busdev;
> 
> parent_obj please - makes it clear that it is a QOM struct and flags
> accidental uses of obsolete SysBus macros.
done

> 
> > +} ICCBridgeState;
> > +#define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj),
> > TYPE_ICC_BRIDGE) +
> > +
> > +static void icc_bridge_initfn(Object *obj)
> > +{
> > +    qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus");
> 
> qbus_create_inplace()?
done
 
[...]

> 
> BTW Anthony recently suggested to drop the "fn" where there's no init
> vs. initfn name conflicts; I don't mind either way.
changed to init
 
[...]
> 
> /*< private >*/
> 
> > +    BusState qbus;
> 
> parent_obj
done

> 
> /*< public >*/
> 
> ...since it is in a header.
> 
> > +} ICCBus;
> > +#define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
> > +
> > +typedef struct ICCDevice {
> > +    DeviceState qdev;
> 
> dito
s/qdev/parent_obj/ will affect APIC and IOAPIC patches make-ing them even
bigger and more difficult to review distracting from real changes.

it would be better to send follow-up patch, that does trivial QOM conversions
afterwards.

I'm not sure that making ICCDevice.qdev private would be good performance wise
since it would require using DEVICE() dynamic cast in apic_check_pic().

I'd like to keep this as is for now and put cleanup patch on TODO list for
1.6 to get rid of DO_UPCAST() macro in apic/ioapic files.

> 
> Who becomes the MAINTAINER of ICC?
Perhaps me??

> 
> Regards,
> Andreas
> 

Re: [Qemu-devel] [PATCH 06/16] target-i386: pc: update rtc_cmos on CPU hot-plug

[Igor Mammedov]

On Mon, 22 Apr 2013 16:56:59 +0200
Andreas Färber <afaerber@suse.de> wrote:

> Am 16.04.2013 00:12, schrieb Igor Mammedov:
> > ... so that on reboot BIOS could read current available CPU count
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > ---
> >  hw/i386/pc.c                   | 20 ++++++++++++++++++++
> >  hw/timer/mc146818rtc.c         |  7 +++++++
> >  include/hw/timer/mc146818rtc.h |  1 +
> >  3 files changed, 28 insertions(+)
> > 
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index 8d75b34..dc1a78b 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -337,6 +337,21 @@ static void pc_cmos_init_late(void *opaque)
> >      qemu_unregister_reset(pc_cmos_init_late, opaque);
> >  }
> >  
> > +typedef struct rtc_cpu_hotplug_arg {
> > +    ISADevice *rtc_state;
> > +    Notifier cpu_added_notifier;
> > +} rtc_cpu_hotplug_arg;
> 
> Are the arguments (rtc_state) intentionally placed before the common part?
no, but it's not QOM type so it doesn't matter.
I'll reorder it on next respin.

> 
> > +
> > +static void rtc_notify_cpu_added(Notifier *notifier, void *data)
> > +{
> > +    rtc_cpu_hotplug_arg *arg = container_of(notifier,
> > rtc_cpu_hotplug_arg,
> > +                                           cpu_added_notifier);
> > +    ISADevice *s = arg->rtc_state;
> > +
> > +    /* increment the number of CPUs */
> > +    rtc_set_memory(s, 0x5f, rtc_get_memory(s, 0x5f) + 1);
> > +}
> 
> I think this proves Gleb's point of reordering the notifier and CPU
> resumption since the first instruction executed might theoretically be
> reading the RTC memory.
I've already changed notifier clall place according to Gleb's suggestions.

> 
> Otherwise looks good with one minor nit...
> 
> > +
> >  void pc_cmos_init(ram_addr_t ram_size, ram_addr_t above_4g_mem_size,
> >                    const char *boot_device,
> >                    ISADevice *floppy, BusState *idebus0, BusState
> > *idebus1, @@ -345,6 +360,7 @@ void pc_cmos_init(ram_addr_t ram_size,
> > ram_addr_t above_4g_mem_size, int val, nb, i;
> >      FDriveType fd_type[2] = { FDRIVE_DRV_NONE, FDRIVE_DRV_NONE };
> >      static pc_cmos_init_late_arg arg;
> > +    static rtc_cpu_hotplug_arg cpu_hotplug_cb;
> >  
> >      /* various important CMOS locations needed by PC/Bochs bios */
> >  
> > @@ -383,6 +399,10 @@ void pc_cmos_init(ram_addr_t ram_size, ram_addr_t
> > above_4g_mem_size, 
> >      /* set the number of CPU */
> >      rtc_set_memory(s, 0x5f, smp_cpus - 1);
> > +    /* init CPU hotplug notifier */
> > +    cpu_hotplug_cb.rtc_state = s;
> > +    cpu_hotplug_cb.cpu_added_notifier.notify = rtc_notify_cpu_added;
> > +    qemu_register_cpu_added_notifier(&cpu_hotplug_cb.cpu_added_notifier);
> >  
> >      /* set boot devices, and disable floppy signature check if requested
> > */ if (set_boot_dev(s, boot_device, fd_bootchk)) {
> > diff --git a/hw/timer/mc146818rtc.c b/hw/timer/mc146818rtc.c
> > index 69e6844..e639942 100644
> > --- a/hw/timer/mc146818rtc.c
> > +++ b/hw/timer/mc146818rtc.c
> > @@ -677,6 +677,13 @@ void rtc_set_memory(ISADevice *dev, int addr, int
> > val) s->cmos_data[addr] = val;
> >  }
> >  
> > +int rtc_get_memory(ISADevice *dev, int addr)
> > +{
> > +    RTCState *s = DO_UPCAST(RTCState, dev, dev);
> 
> If we apply my mc146818rtc.c QOM'ification patch to qom-cpu first, this
> can become MC146818_RTC(dev). Anthony just ack'ed that, but qemu.git is
> currently broken for testing, so I'm still holding off changes.
if you push it into your qom-cpu tree now, I'll rebase on top of it.

> 
> Andreas
> 
> > +    assert(addr >= 0 && addr <= 127);
> > +    return s->cmos_data[addr];
> > +}
> > +
> >  static void rtc_set_date_from_host(ISADevice *dev)
> >  {
> >      RTCState *s = DO_UPCAST(RTCState, dev, dev);
> > diff --git a/include/hw/timer/mc146818rtc.h
> > b/include/hw/timer/mc146818rtc.h index 854ea3f..09f37b7 100644
> > --- a/include/hw/timer/mc146818rtc.h
> > +++ b/include/hw/timer/mc146818rtc.h
> > @@ -6,6 +6,7 @@
> >  
> >  ISADevice *rtc_init(ISABus *bus, int base_year, qemu_irq intercept_irq);
> >  void rtc_set_memory(ISADevice *dev, int addr, int val);
> > +int rtc_get_memory(ISADevice *dev, int addr);
> >  void rtc_set_date(ISADevice *dev, const struct tm *tm);
> >  
> >  #endif /* !MC146818RTC_H */
> 

Re: [Qemu-devel] [PATCH 14/16] target-i386: move APIC to ICC bus

[Andreas Färber]

Am 16.04.2013 00:12, schrieb Igor Mammedov:
> ... to allow it to be hotplugged
> 
>  * map APIC's mmio at board level if it is present
>  * do not register mmio region for each APIC, since
>    only one is used/mapped
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
>  hw/cpu/icc_bus.c                | 15 ++++++++++++++-
>  hw/i386/kvmvapic.c              |  1 +
>  hw/i386/pc.c                    | 20 +++++++++++++++++---
>  hw/intc/apic_common.c           | 17 ++++++++++++-----
>  include/hw/i386/apic_internal.h |  6 +++---
>  include/hw/i386/icc_bus.h       |  2 ++
>  target-i386/cpu.c               | 16 +++-------------
>  7 files changed, 52 insertions(+), 25 deletions(-)
> 
> diff --git a/hw/cpu/icc_bus.c b/hw/cpu/icc_bus.c
> index 00b9be3..5078c38 100644
> --- a/hw/cpu/icc_bus.c
> +++ b/hw/cpu/icc_bus.c
> @@ -62,13 +62,26 @@ static const TypeInfo icc_device_info = {
>  
>  typedef struct ICCBridgeState {
>      SysBusDevice busdev;
> +    MemoryRegion apic_container;
>  } ICCBridgeState;
>  #define ICC_BRIGDE(obj) OBJECT_CHECK(ICCBridgeState, (obj), TYPE_ICC_BRIDGE)
>  
>  
>  static void icc_bridge_initfn(Object *obj)
>  {
> -    qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus");
> +    ICCBridgeState *s = ICC_BRIGDE(obj);
> +    SysBusDevice *sb = SYS_BUS_DEVICE(obj);
> +    ICCBus *ibus;
> +
> +    ibus = ICC_BUS(qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus"));

qbus_create_inplace() as suggested on ICC patch would allow to access it
through state field.

> +
> +    /* Do not change order of registering regions,
> +     * APIC must be first registered region, board maps it by 0 index
> +     */
> +    memory_region_init(&s->apic_container, "icc-apic-container",
> +                       APIC_SPACE_SIZE);
> +    sysbus_init_mmio(sb, &s->apic_container);
> +    ibus->apic_address_space = &s->apic_container;
>  }
>  
>  static const TypeInfo icc_bridge_info = {
> diff --git a/hw/i386/kvmvapic.c b/hw/i386/kvmvapic.c
> index 3a10c07..5b558aa 100644
> --- a/hw/i386/kvmvapic.c
> +++ b/hw/i386/kvmvapic.c
> @@ -12,6 +12,7 @@
>  #include "sysemu/cpus.h"
>  #include "sysemu/kvm.h"
>  #include "hw/i386/apic_internal.h"
> +#include "hw/sysbus.h"
>  
>  #define VAPIC_IO_PORT           0x7e
>  

I take it, this is necessary due to dropping hw/sysbus.h include in
cpu.c? Might be split out then since patch is rather large already.

> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index cb57878..31b5294 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -52,6 +52,7 @@
>  #include "sysemu/arch_init.h"
>  #include "qemu/bitmap.h"
>  #include "qemu/config-file.h"
> +#include "hw/i386/icc_bus.h"
>  
>  /* debug PC/ISA interrupts */
>  //#define DEBUG_IRQ

> @@ -889,13 +890,13 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level)
>      }
>  }
>  
> -static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
> +static X86CPU *pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
>  {
>      X86CPU *cpu;
>  
>      cpu = cpu_x86_create(cpu_model, errp);
>      if (!cpu) {
> -        return;
> +        return cpu;
>      }
>  
>      object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
> @@ -904,14 +905,18 @@ static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error **errp)
>      if (error_is_set(errp)) {
>          if (cpu != NULL) {
>              object_unref(OBJECT(cpu));
> +            cpu = NULL;
>          }
>      }
> +    return cpu;
>  }
>  

Squash this into the pc_new_cpu() introduction?

>  void pc_cpus_init(const char *cpu_model)
>  {
>      int i;
> +    X86CPU *cpu = NULL;
>      Error *error = NULL;
> +    SysBusDevice *ib;

icc maybe? Anything that more's comprehensible. ;)

>  
>      /* init CPUs */
>      if (cpu_model == NULL) {
> @@ -922,14 +927,23 @@ void pc_cpus_init(const char *cpu_model)
>  #endif
>      }
>  
> +    ib = SYS_BUS_DEVICE(object_resolve_path_type("icc-bridge",
> +                                                 TYPE_ICC_BRIDGE, NULL));
> +
>      for (i = 0; i < smp_cpus; i++) {
> -        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
> +        cpu = pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);

(Here it would've been nice too but would lead to assigned-but-not-used
warnings.)

>          if (error) {
>              fprintf(stderr, "%s\n", error_get_pretty(error));
>              error_free(error);
>              exit(1);
>          }
>      }
> +
> +    /* map APIC MMIO area if CPU has APIC */
> +    if (cpu && cpu->env.apic_state) {
> +        /* XXX: what if the base changes? */
> +        sysbus_mmio_map_overlap(ib, 0, APIC_DEFAULT_ADDRESS, 0x1000);
> +    }
>  }
>  
>  void pc_acpi_init(const char *default_dsdt)
> diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
> index e0ae07a..39396f1 100644
> --- a/hw/intc/apic_common.c
> +++ b/hw/intc/apic_common.c
> @@ -21,6 +21,8 @@
>  #include "hw/i386/apic_internal.h"
>  #include "trace.h"
>  #include "sysemu/kvm.h"
> +#include "hw/qdev.h"
> +#include "hw/sysbus.h"
>  
>  static int apic_irq_delivered;
>  bool apic_report_tpr_access;
> @@ -282,12 +284,14 @@ static int apic_load_old(QEMUFile *f, void *opaque, int version_id)
>      return 0;
>  }
>  
> -static int apic_init_common(SysBusDevice *dev)
> +static int apic_init_common(ICCDevice *dev)

Instead of mechanically replacing SysBusDevice with ICCDevice in qdev
init functions, please change to realize. Could be its own prepended
patch if you prefer.

>  {
>      APICCommonState *s = APIC_COMMON(dev);
> +    DeviceState *d = DEVICE(dev);
>      APICCommonClass *info;
>      static DeviceState *vapic;
>      static int apic_no;
> +    static bool mmio_registered;
>  
>      if (apic_no >= MAX_APICS) {
>          return -1;
> @@ -296,8 +300,11 @@ static int apic_init_common(SysBusDevice *dev)
>  
>      info = APIC_COMMON_GET_CLASS(s);
>      info->init(s);
> -
> -    sysbus_init_mmio(dev, &s->io_memory);
> +    if (!mmio_registered) {
> +        MemoryRegion *as = ICC_BUS(d->parent_bus)->apic_address_space;
> +        memory_region_add_subregion(as, 0, &s->io_memory);
> +        mmio_registered = true;
> +    }
>  
>      /* Note: We need at least 1M to map the VAPIC option ROM */
>      if (!vapic && s->vapic_control & VAPIC_ENABLE_MASK &&
> @@ -375,7 +382,7 @@ static Property apic_properties_common[] = {
>  
>  static void apic_common_class_init(ObjectClass *klass, void *data)
>  {
> -    SysBusDeviceClass *sc = SYS_BUS_DEVICE_CLASS(klass);
> +    ICCDeviceClass *sc = ICC_DEVICE_CLASS(klass);

idc please. :)

>      DeviceClass *dc = DEVICE_CLASS(klass);
>  
>      dc->vmsd = &vmstate_apic_common;
> @@ -387,7 +394,7 @@ static void apic_common_class_init(ObjectClass *klass, void *data)
>  
>  static const TypeInfo apic_common_type = {
>      .name = TYPE_APIC_COMMON,
> -    .parent = TYPE_SYS_BUS_DEVICE,
> +    .parent = TYPE_ICC_DEVICE,
>      .instance_size = sizeof(APICCommonState),
>      .class_size = sizeof(APICCommonClass),
>      .class_init = apic_common_class_init,
> diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
> index aac6290..05acf4b 100644
> --- a/include/hw/i386/apic_internal.h
> +++ b/include/hw/i386/apic_internal.h
> @@ -21,7 +21,7 @@
>  #define QEMU_APIC_INTERNAL_H
>  
>  #include "exec/memory.h"
> -#include "hw/sysbus.h"
> +#include "hw/i386/icc_bus.h"
>  #include "qemu/timer.h"
>  
>  /* APIC Local Vector Table */
> @@ -78,7 +78,7 @@ typedef struct APICCommonState APICCommonState;
>  
>  typedef struct APICCommonClass
>  {
> -    SysBusDeviceClass parent_class;
> +    ICCDeviceClass parent_class;
>  
>      void (*init)(APICCommonState *s);
>      void (*set_base)(APICCommonState *s, uint64_t val);
> @@ -92,7 +92,7 @@ typedef struct APICCommonClass
>  } APICCommonClass;
>  
>  struct APICCommonState {
> -    SysBusDevice busdev;
> +    ICCDevice busdev;

parent_obj please, while at it.

>  
>      MemoryRegion io_memory;
>      X86CPU *cpu;
> diff --git a/include/hw/i386/icc_bus.h b/include/hw/i386/icc_bus.h
> index aa30e0c..1b07619 100644
> --- a/include/hw/i386/icc_bus.h
> +++ b/include/hw/i386/icc_bus.h
> @@ -22,6 +22,7 @@
>  #ifndef ICC_BUS_H
>  #define ICC_BUS_H
>  
> +#include "exec/memory.h"
>  #include "hw/qdev-core.h"
>  
>  #define TYPE_ICC_BUS "icc-bus"
> @@ -29,6 +30,7 @@
>  #ifndef CONFIG_USER_ONLY
>  typedef struct ICCBus {
>      BusState qbus;
> +    MemoryRegion *apic_address_space;
>  } ICCBus;
>  #define ICC_BUS(obj) OBJECT_CHECK(ICCBus, (obj), TYPE_ICC_BUS)
>  
> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> index 3b5f90b..bdac6ea 100644
> --- a/target-i386/cpu.c
> +++ b/target-i386/cpu.c
> @@ -41,10 +41,10 @@
>  #endif
>  
>  #include "sysemu/sysemu.h"
> +#include "hw/qdev-properties.h"
>  #include "hw/i386/icc_bus.h"
>  #ifndef CONFIG_USER_ONLY
>  #include "hw/xen/xen.h"
> -#include "hw/sysbus.h"
>  #include "hw/i386/apic_internal.h"
>  #endif
>  
> @@ -2111,6 +2111,7 @@ static void mce_init(X86CPU *cpu)
>  static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
>  {
>      CPUX86State *env = &cpu->env;
> +    DeviceState *dev = DEVICE(cpu);
>      APICCommonState *apic;
>      const char *apic_type = "apic";
>  
> @@ -2120,7 +2121,7 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
>          apic_type = "xen-apic";
>      }
>  
> -    env->apic_state = qdev_try_create(NULL, apic_type);
> +    env->apic_state = qdev_try_create(dev->parent_bus, apic_type);

qdev_get_parent_bus() according to header comment.

>      if (env->apic_state == NULL) {
>          error_setg(errp, "APIC device '%s' could not be created", apic_type);
>          return;
> @@ -2137,7 +2138,6 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
>  static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
>  {
>      CPUX86State *env = &cpu->env;
> -    static int apic_mapped;
>  
>      if (env->apic_state == NULL) {
>          return;
> @@ -2148,16 +2148,6 @@ static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
>                     object_get_typename(OBJECT(env->apic_state)));
>          return;
>      }
> -
> -    /* XXX: mapping more APICs at the same memory location */
> -    if (apic_mapped == 0) {
> -        /* NOTE: the APIC is directly connected to the CPU - it is not
> -           on the global memory bus. */
> -        /* XXX: what if the base changes? */
> -        sysbus_mmio_map_overlap(SYS_BUS_DEVICE(env->apic_state), 0,
> -                                APIC_DEFAULT_ADDRESS, 0x1000);
> -        apic_mapped = 1;
> -    }
>  }
>  #else
>  static void x86_cpu_apic_realize(X86CPU *cpu, Error **errp)
> 

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 12/16] target-i386: cpu: attach ICC bus to CPU on its creation

[Igor Mammedov]

On Mon, 22 Apr 2013 17:02:33 +0200
Andreas Färber <afaerber@suse.de> wrote:

> Am 16.04.2013 00:12, schrieb Igor Mammedov:
> > ... during startup,  so it would be possible to unplug it later
> > and set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add
> > attach hotplugged CPU to ICC bus.
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> 
> Reviewed-by: Andreas Färber <afaerber@suse.de>
> 
> But still one question...
> 
> > ---
> >  target-i386/cpu.c | 7 +++++++
> >  1 file changed, 7 insertions(+)
> > 
> > diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> > index 6d6c527..3b5f90b 100644
> > --- a/target-i386/cpu.c
> > +++ b/target-i386/cpu.c
> > @@ -41,6 +41,7 @@
> >  #endif
> >  
> >  #include "sysemu/sysemu.h"
> > +#include "hw/i386/icc_bus.h"
> >  #ifndef CONFIG_USER_ONLY
> >  #include "hw/xen/xen.h"
> >  #include "hw/sysbus.h"
> > @@ -1609,6 +1610,7 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error
> > **errp) gchar **model_pieces;
> >      char *name, *features;
> >      Error *error = NULL;
> > +    Object *icc_bus = object_resolve_path_type("icc-bus", TYPE_ICC_BUS,
> > NULL); 
> >      model_pieces = g_strsplit(cpu_model, ",", 2);
> >      if (!model_pieces[0]) {
> > @@ -1619,6 +1621,10 @@ X86CPU *cpu_x86_create(const char *cpu_model,
> > Error **errp) features = model_pieces[1];
> >  
> >      cpu = X86_CPU(object_new(TYPE_X86_CPU));
> > +    if (icc_bus) {
> > +        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
> > +        object_unref(OBJECT(cpu));
> > +    }
> >      env = &cpu->env;
> >      env->cpu_model_str = cpu_model;
> >  
> 
> You seem to be avoiding making lack of icc-bus an error although you add
> it for both PC and q35 PC - which non-ICC use cases are you thinking of?
*-user targets, CPU is bus-less there.

> 
> Andreas
> 
> > @@ -2330,6 +2336,7 @@ static void x86_cpu_common_class_init(ObjectClass
> > *oc, void *data) 
> >      xcc->parent_realize = dc->realize;
> >      dc->realize = x86_cpu_realizefn;
> > +    dc->bus_type = TYPE_ICC_BUS;
> >  
> >      xcc->parent_reset = cc->reset;
> >      cc->reset = x86_cpu_reset;
> > 
> 
> 

Re: [Qemu-devel] [PATCH 12/16] target-i386: cpu: attach ICC bus to CPU on its creation

[Andreas Färber]

Am 22.04.2013 17:20, schrieb Igor Mammedov:
> On Mon, 22 Apr 2013 17:02:33 +0200
> Andreas Färber <afaerber@suse.de> wrote:
> 
>> Am 16.04.2013 00:12, schrieb Igor Mammedov:
>>> ... during startup,  so it would be possible to unplug it later
>>> and set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add
>>> attach hotplugged CPU to ICC bus.
>>>
>>> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
>>
>> Reviewed-by: Andreas Färber <afaerber@suse.de>
>>
>> But still one question...
>>
>>> ---
>>>  target-i386/cpu.c | 7 +++++++
>>>  1 file changed, 7 insertions(+)
>>>
>>> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
>>> index 6d6c527..3b5f90b 100644
>>> --- a/target-i386/cpu.c
>>> +++ b/target-i386/cpu.c
>>> @@ -41,6 +41,7 @@
>>>  #endif
>>>  
>>>  #include "sysemu/sysemu.h"
>>> +#include "hw/i386/icc_bus.h"
>>>  #ifndef CONFIG_USER_ONLY
>>>  #include "hw/xen/xen.h"
>>>  #include "hw/sysbus.h"
>>> @@ -1609,6 +1610,7 @@ X86CPU *cpu_x86_create(const char *cpu_model, Error
>>> **errp) gchar **model_pieces;
>>>      char *name, *features;
>>>      Error *error = NULL;
>>> +    Object *icc_bus = object_resolve_path_type("icc-bus", TYPE_ICC_BUS,
>>> NULL); 
>>>      model_pieces = g_strsplit(cpu_model, ",", 2);
>>>      if (!model_pieces[0]) {
>>> @@ -1619,6 +1621,10 @@ X86CPU *cpu_x86_create(const char *cpu_model,
>>> Error **errp) features = model_pieces[1];
>>>  
>>>      cpu = X86_CPU(object_new(TYPE_X86_CPU));
>>> +    if (icc_bus) {
>>> +        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
>>> +        object_unref(OBJECT(cpu));
>>> +    }
>>>      env = &cpu->env;
>>>      env->cpu_model_str = cpu_model;
>>>  
>>
>> You seem to be avoiding making lack of icc-bus an error although you add
>> it for both PC and q35 PC - which non-ICC use cases are you thinking of?
> *-user targets, CPU is bus-less there.

Would you be opposed to #ifdef'ing it then? In particular I am asking
because of the NULL error argument to the resolve function - for softmmu
I would prefer to make lack of icc-bus a fatal error.

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 14/16] target-i386: move APIC to ICC bus

[Igor Mammedov]

On Mon, 22 Apr 2013 17:18:58 +0200
Andreas Färber <afaerber@suse.de> wrote:

> Am 16.04.2013 00:12, schrieb Igor Mammedov:
[...]

> >  static void icc_bridge_initfn(Object *obj)
> >  {
> > -    qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus");
> > +    ICCBridgeState *s = ICC_BRIGDE(obj);
> > +    SysBusDevice *sb = SYS_BUS_DEVICE(obj);
> > +    ICCBus *ibus;
> > +
> > +    ibus = ICC_BUS(qbus_create(TYPE_ICC_BUS, DEVICE(obj), "icc-bus"));
> 
> qbus_create_inplace() as suggested on ICC patch would allow to access it
> through state field.
done

> 
> > +
> > +    /* Do not change order of registering regions,
> > +     * APIC must be first registered region, board maps it by 0 index
> > +     */
> > +    memory_region_init(&s->apic_container, "icc-apic-container",
> > +                       APIC_SPACE_SIZE);
> > +    sysbus_init_mmio(sb, &s->apic_container);
> > +    ibus->apic_address_space = &s->apic_container;
> >  }
> >  
> >  static const TypeInfo icc_bridge_info = {
> > diff --git a/hw/i386/kvmvapic.c b/hw/i386/kvmvapic.c
> > index 3a10c07..5b558aa 100644
> > --- a/hw/i386/kvmvapic.c
> > +++ b/hw/i386/kvmvapic.c
> > @@ -12,6 +12,7 @@
> >  #include "sysemu/cpus.h"
> >  #include "sysemu/kvm.h"
> >  #include "hw/i386/apic_internal.h"
> > +#include "hw/sysbus.h"
> >  
> >  #define VAPIC_IO_PORT           0x7e
> >  
> 
> I take it, this is necessary due to dropping hw/sysbus.h include in
> cpu.c? Might be split out then since patch is rather large already.
It's due to dropping  hw/sysbus.h from include/hw/i386/apic_internal.h
It's hard to split in meaningful separate patch.

> 
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index cb57878..31b5294 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -52,6 +52,7 @@
> >  #include "sysemu/arch_init.h"
> >  #include "qemu/bitmap.h"
> >  #include "qemu/config-file.h"
> > +#include "hw/i386/icc_bus.h"
> >  
> >  /* debug PC/ISA interrupts */
> >  //#define DEBUG_IRQ
> 
> > @@ -889,13 +890,13 @@ void pc_acpi_smi_interrupt(void *opaque, int irq,
> > int level) }
> >  }
> >  
> > -static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error
> > **errp) +static X86CPU *pc_new_cpu(const char *cpu_model, int64_t
> > apic_id, Error **errp) {
> >      X86CPU *cpu;
> >  
> >      cpu = cpu_x86_create(cpu_model, errp);
> >      if (!cpu) {
> > -        return;
> > +        return cpu;
> >      }
> >  
> >      object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
> > @@ -904,14 +905,18 @@ static void pc_new_cpu(const char *cpu_model,
> > int64_t apic_id, Error **errp) if (error_is_set(errp)) {
> >          if (cpu != NULL) {
> >              object_unref(OBJECT(cpu));
> > +            cpu = NULL;
> >          }
> >      }
> > +    return cpu;
> >  }
> >  
> 
> Squash this into the pc_new_cpu() introduction?
sure, it's less obvious why it would be needed but makes this patch smaller.

> 
> >  void pc_cpus_init(const char *cpu_model)
> >  {
> >      int i;
> > +    X86CPU *cpu = NULL;
> >      Error *error = NULL;
> > +    SysBusDevice *ib;
> 
> icc maybe? Anything that more's comprehensible. ;)
icc_bridge then.

[...]
> > diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
> > index e0ae07a..39396f1 100644
> > --- a/hw/intc/apic_common.c
> > +++ b/hw/intc/apic_common.c
> > @@ -21,6 +21,8 @@
> >  #include "hw/i386/apic_internal.h"
> >  #include "trace.h"
> >  #include "sysemu/kvm.h"
> > +#include "hw/qdev.h"
> > +#include "hw/sysbus.h"
> >  
> >  static int apic_irq_delivered;
> >  bool apic_report_tpr_access;
> > @@ -282,12 +284,14 @@ static int apic_load_old(QEMUFile *f, void *opaque,
> > int version_id) return 0;
> >  }
> >  
> > -static int apic_init_common(SysBusDevice *dev)
> > +static int apic_init_common(ICCDevice *dev)
> 
> Instead of mechanically replacing SysBusDevice with ICCDevice in qdev
> init functions, please change to realize. Could be its own prepended
> patch if you prefer.

IRC with Andreas, agreed to do it only for ICCDevice. And do re-factoring
of init() in APICs/IOAPIC later.

> 
> >  {
> >      APICCommonState *s = APIC_COMMON(dev);
> > +    DeviceState *d = DEVICE(dev);
> >      APICCommonClass *info;
> >      static DeviceState *vapic;
> >      static int apic_no;
> > +    static bool mmio_registered;
> >  
> >      if (apic_no >= MAX_APICS) {
> >          return -1;
[...]

> > @@ -375,7 +382,7 @@ static Property apic_properties_common[] = {
> >  
> >  static void apic_common_class_init(ObjectClass *klass, void *data)
> >  {
> > -    SysBusDeviceClass *sc = SYS_BUS_DEVICE_CLASS(klass);
> > +    ICCDeviceClass *sc = ICC_DEVICE_CLASS(klass);
> 
> idc please. :)
done

> 
> >      DeviceClass *dc = DEVICE_CLASS(klass);
> >  
> >      dc->vmsd = &vmstate_apic_common;
> > @@ -387,7 +394,7 @@ static void apic_common_class_init(ObjectClass
> > *klass, void *data) 
> >  static const TypeInfo apic_common_type = {
[...]

> > @@ -2120,7 +2121,7 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error
> > **errp) apic_type = "xen-apic";
> >      }
> >  
> > -    env->apic_state = qdev_try_create(NULL, apic_type);
> > +    env->apic_state = qdev_try_create(dev->parent_bus, apic_type);
> 
> qdev_get_parent_bus() according to header comment.
done

> 
> >      if (env->apic_state == NULL) {
[...]
> Andreas
> 

Re: [Qemu-devel] [PATCH 01/16 v2] cpu: make kvm-stub.o a part of CPU library

[Gleb Natapov]

On Mon, Apr 22, 2013 at 04:35:13PM +0200, Paolo Bonzini wrote:
> Il 22/04/2013 15:45, Andreas Färber ha scritto:
> > Am 18.04.2013 16:24, schrieb Eduardo Habkost:
> >> > On Thu, Apr 18, 2013 at 06:23:50AM +0200, Igor Mammedov wrote:
> >>> >> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> >> > 
> >> > Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
> > Looks good to me, too.
> > 
> > Paolo and Gleb, are you okay with this disguised v5?
> 
> I am fine with it too.
> 
Same here.

--
			Gleb.

Re: [Qemu-devel] [PATCH 02/16] cpu: call cpu_synchronize_post_init() from CPUClass.realize() if hotplugged

[Gleb Natapov]

On Mon, Apr 22, 2013 at 03:46:37PM +0200, Andreas Färber wrote:
> Am 16.04.2013 00:12, schrieb Igor Mammedov:
> > ... to synchronize CPU state to KVM
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > ---
> > v2:
> >   * linking kvm-stub.o to *-user target moved in separate patch
> > ---
> >  include/sysemu/kvm.h | 18 ++++++++++--------
> >  kvm-all.c            |  1 +
> >  kvm-stub.c           |  1 +
> >  qom/cpu.c            |  4 ++++
> >  vl.c                 |  1 -
> >  5 files changed, 16 insertions(+), 9 deletions(-)
> 
> Looks okay to me. Gleb, can you ack?
> 
Yes, looks good to me.

> Andreas
> 
> > diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
> > index 93cef28..8fb2489 100644
> > --- a/include/sysemu/kvm.h
> > +++ b/include/sysemu/kvm.h
> > @@ -250,8 +250,6 @@ int kvm_check_extension(KVMState *s, unsigned int extension);
> >  uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
> >                                        uint32_t index, int reg);
> >  void kvm_cpu_synchronize_state(CPUArchState *env);
> > -void kvm_cpu_synchronize_post_reset(CPUState *cpu);
> > -void kvm_cpu_synchronize_post_init(CPUState *cpu);
> >  
> >  /* generic hooks - to be moved/refactored once there are more users */
> >  
> > @@ -262,6 +260,16 @@ static inline void cpu_synchronize_state(CPUArchState *env)
> >      }
> >  }
> >  
> > +#if !defined(CONFIG_USER_ONLY)
> > +int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
> > +                                       hwaddr *phys_addr);
> > +#endif
> > +
> > +#endif /* NEED_CPU_H */
> > +
> > +void kvm_cpu_synchronize_post_reset(CPUState *cpu);
> > +void kvm_cpu_synchronize_post_init(CPUState *cpu);
> > +
> >  static inline void cpu_synchronize_post_reset(CPUState *cpu)
> >  {
> >      if (kvm_enabled()) {
> > @@ -277,12 +285,6 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
> >  }
> >  
> >  
> > -#if !defined(CONFIG_USER_ONLY)
> > -int kvm_physical_memory_addr_from_host(KVMState *s, void *ram_addr,
> > -                                       hwaddr *phys_addr);
> > -#endif
> > -
> > -#endif
> >  int kvm_set_ioeventfd_mmio(int fd, uint32_t adr, uint32_t val, bool assign,
> >                             uint32_t size);
> >  
> > diff --git a/kvm-all.c b/kvm-all.c
> > index fc4e17c..1d17128 100644
> > --- a/kvm-all.c
> > +++ b/kvm-all.c
> > @@ -109,6 +109,7 @@ bool kvm_async_interrupts_allowed;
> >  bool kvm_irqfds_allowed;
> >  bool kvm_msi_via_irqfd_allowed;
> >  bool kvm_gsi_routing_allowed;
> > +bool kvm_allowed;
> >  
> >  static const KVMCapabilityInfo kvm_required_capabilites[] = {
> >      KVM_CAP_INFO(USER_MEMORY),
> > diff --git a/kvm-stub.c b/kvm-stub.c
> > index a131305..4282e09 100644
> > --- a/kvm-stub.c
> > +++ b/kvm-stub.c
> > @@ -23,6 +23,7 @@ bool kvm_async_interrupts_allowed;
> >  bool kvm_irqfds_allowed;
> >  bool kvm_msi_via_irqfd_allowed;
> >  bool kvm_gsi_routing_allowed;
> > +bool kvm_allowed;
> >  
> >  int kvm_init_vcpu(CPUState *cpu)
> >  {
> > diff --git a/qom/cpu.c b/qom/cpu.c
> > index e242dcb..0c76712 100644
> > --- a/qom/cpu.c
> > +++ b/qom/cpu.c
> > @@ -20,6 +20,7 @@
> >  
> >  #include "qom/cpu.h"
> >  #include "qemu-common.h"
> > +#include "sysemu/kvm.h"
> >  
> >  void cpu_reset_interrupt(CPUState *cpu, int mask)
> >  {
> > @@ -57,6 +58,9 @@ static ObjectClass *cpu_common_class_by_name(const char *cpu_model)
> >  
> >  static void cpu_common_realizefn(DeviceState *dev, Error **errp)
> >  {
> > +    if (dev->hotplugged) {
> > +        cpu_synchronize_post_init(CPU(dev));
> > +    }
> >  }
> >  
> >  static void cpu_class_init(ObjectClass *klass, void *data)
> > diff --git a/vl.c b/vl.c
> > index dd94549..bc9c016 100644
> > --- a/vl.c
> > +++ b/vl.c
> > @@ -267,7 +267,6 @@ static NotifierList machine_init_done_notifiers =
> >      NOTIFIER_LIST_INITIALIZER(machine_init_done_notifiers);
> >  
> >  static bool tcg_allowed = true;
> > -bool kvm_allowed;
> >  bool xen_allowed;
> >  uint32_t xen_domid;
> >  enum xen_mode xen_mode = XEN_EMULATE;
> > 
> 
> 
> -- 
> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

--
			Gleb.

Re: [Qemu-devel] [PATCH 10/16] target-i386: introduce apic-id property

[Igor Mammedov]

On Mon, 22 Apr 2013 16:05:34 +0200
Andreas Färber <afaerber@suse.de> wrote:

> Am 16.04.2013 00:12, schrieb Igor Mammedov:
> > ... and use it from board level to set APIC ID for CPUs
> > it creates.
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
> > ---
> > Note:
> >   * pc_new_cpu() function will be reused later in CPU hot-plug hook.
> 
> Suggest as main commit message to avoid the "...":
> 
> The property is used from board level to set APIC ID for CPUs it
> creates. Do so in a new pc_new_cpu() helper, to be reused for hot-plug.
I'll do on next respin.

> > 
> > v3:
> >   * user error_propagate() in property setter
> > v2:
> >   * use generic cpu_exists() instead of custom one
> >   * make apic-id dynamic property, so it won't be possible to use it
> >     as global property, since it's instance specific
> > ---
> >  hw/i386/pc.c      | 25 ++++++++++++++++++++++++-
> >  target-i386/cpu.c | 42 ++++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 66 insertions(+), 1 deletion(-)
> > 
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index dc1a78b..cb57878 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -889,9 +889,29 @@ void pc_acpi_smi_interrupt(void *opaque, int irq,
> > int level) }
> >  }
> >  
> > +static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error
> > **errp) +{
> > +    X86CPU *cpu;
> > +
> > +    cpu = cpu_x86_create(cpu_model, errp);
> > +    if (!cpu) {
> > +        return;
> > +    }
> > +
> > +    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
> > +    object_property_set_bool(OBJECT(cpu), true, "realized", errp);
> > +
> > +    if (error_is_set(errp)) {
> 
> Please use a local Error* variable with error_propagate, otherwise this
> is fragile.
done

> 
> > +        if (cpu != NULL) {
> > +            object_unref(OBJECT(cpu));
> > +        }
> > +    }
> > +}
> > +
> >  void pc_cpus_init(const char *cpu_model)
> >  {
> >      int i;
> > +    Error *error = NULL;
> >  
> >      /* init CPUs */
> >      if (cpu_model == NULL) {
> > @@ -903,7 +923,10 @@ void pc_cpus_init(const char *cpu_model)
> >      }
> >  
> >      for (i = 0; i < smp_cpus; i++) {
> > -        if (!cpu_x86_init(cpu_model)) {
> > +        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
> > +        if (error) {
> > +            fprintf(stderr, "%s\n", error_get_pretty(error));
> > +            error_free(error);
> >              exit(1);
> >          }
> >      }
> > diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> > index a415fa3..6d6c527 100644
> > --- a/target-i386/cpu.c
> > +++ b/target-i386/cpu.c
> > @@ -1271,6 +1271,45 @@ static void x86_cpuid_set_tsc_freq(Object *obj,
> > Visitor *v, void *opaque, cpu->env.tsc_khz = value / 1000;
> >  }
> >  
> > +static void x86_cpuid_get_apic_id(Object *obj, Visitor *v, void *opaque,
> > +                                  const char *name, Error **errp)
> > +{
> > +    X86CPU *cpu = X86_CPU(obj);
> > +    int64_t value = cpu->env.cpuid_apic_id;
> > +
> > +    visit_type_int(v, &value, name, errp);
> > +}
> > +
> > +static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
> > +                                  const char *name, Error **errp)
> > +{
> > +    X86CPU *cpu = X86_CPU(obj);
> > +    const int64_t min = 0;
> > +    const int64_t max = UINT32_MAX;
> > +    Error *error = NULL;
> > +    int64_t value;
> > +
> > +    visit_type_int(v, &value, name, &error);
> > +    if (error) {
> > +        error_propagate(errp, error);
> > +        return;
> > +    }
> 
> > +    if (value < min || value > max) {
> > +        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
> > +                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
> > +                   object_get_typename(obj), name, value, min, max);
> > +        error_propagate(errp, error);
> > +        return;
> > +    }
> > +
> > +    if (cpu_exists(value)) {
> > +        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
> > +        error_propagate(errp, error);
> 
> You could do both error_setg()s directly on errp.
it was so before, but I was requested to use local error here:
http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg01606.html

> 
> > +        return;
> > +    }
> > +    cpu->env.cpuid_apic_id = value;
> > +}
> > +
> >  static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, const char *name)
> >  {
> >      x86_def_t *def;
> > @@ -2259,6 +2298,9 @@ static void x86_cpu_initfn(Object *obj)
> >      object_property_add(obj, "tsc-frequency", "int",
> >                          x86_cpuid_get_tsc_freq,
> >                          x86_cpuid_set_tsc_freq, NULL, NULL, NULL);
> > +    object_property_add(obj, "apic-id", "int",
> > +                        x86_cpuid_get_apic_id,
> > +                        x86_cpuid_set_apic_id, NULL, NULL, NULL);
> >  
> >      env->cpuid_apic_id = x86_cpu_apic_id_from_index(cs->cpu_index);
> >  
> 
> Otherwise I like this x86-specific version now!
> 
> Andreas
> 

Re: [Qemu-devel] [PATCH 07/16] cpu: introduce get_arch_id() method and override it for target-i386

[Andreas Färber]

Am 16.04.2013 00:12, schrieb Igor Mammedov:
> get_arch_id() adds possibility for generic code to get guest visible
> CPU ID without accessing CPUArchState. If target doesn't override it,
> it will return cpu_index.
> 
> Override it on target-i386 to return APIC ID.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> ---
>  * it will be used later by new cpu_exists() generic function and
>    acpi_piix.
>  * s/cpu_firmware_id/cpu_arch_id/
> ---
>  include/qom/cpu.h |  2 ++
>  qom/cpu.c         |  6 ++++++
>  target-i386/cpu.c | 10 ++++++++++
>  3 files changed, 18 insertions(+)

Eduardo and me had discussed on IRC that apparently the only reason we
need this generalized "arch_id" is that acpi_piix4 is shared between x86
and mips. Did you investigate whether we can just access the x86 apic-id
property instead for x86 and leave mips as-is without hotplug support?

Andreas

> 
> diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> index 6ce6f10..428aaf0 100644
> --- a/include/qom/cpu.h
> +++ b/include/qom/cpu.h
> @@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
>   * instantiatable CPU type.
>   * @reset: Callback to reset the #CPUState to its initial state.
>   * @do_interrupt: Callback for interrupt handling.
> + * @get_arch_id: Callback for getting architecture depended CPU ID
>   * @vmsd: State description for migration.
>   *
>   * Represents a CPU family or model.
> @@ -58,6 +59,7 @@ typedef struct CPUClass {
>  
>      void (*reset)(CPUState *cpu);
>      void (*do_interrupt)(CPUState *cpu);
> +    int64_t (*get_arch_id)(CPUState *cpu);
>  
>      const struct VMStateDescription *vmsd;
>  } CPUClass;
> diff --git a/qom/cpu.c b/qom/cpu.c
> index 819986e..a13ddda 100644
> --- a/qom/cpu.c
> +++ b/qom/cpu.c
> @@ -76,6 +76,11 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
>      }
>  }
>  
> +static int64_t cpu_common_get_arch_id(CPUState *cpu)
> +{
> +    return cpu->cpu_index;
> +}
> +
>  static void cpu_class_init(ObjectClass *klass, void *data)
>  {
>      DeviceClass *dc = DEVICE_CLASS(klass);
> @@ -83,6 +88,7 @@ static void cpu_class_init(ObjectClass *klass, void *data)
>  
>      k->class_by_name = cpu_common_class_by_name;
>      k->reset = cpu_common_reset;
> +    k->get_arch_id = cpu_common_get_arch_id;
>      dc->realize = cpu_common_realizefn;
>      dc->no_user = 1;
>  }
> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> index e2302d8..a415fa3 100644
> --- a/target-i386/cpu.c
> +++ b/target-i386/cpu.c
> @@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
>      }
>  }
>  
> +static int64_t x86_cpu_get_arch_id(CPUState *cpu)
> +{
> +    X86CPU *x86cpu = X86_CPU(cpu);
> +    CPUX86State *env = &x86cpu->env;
> +
> +    return env->cpuid_apic_id;
> +}
> +
>  static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
>  {
>      X86CPUClass *xcc = X86_CPU_CLASS(oc);
> @@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
>  
>      cc->do_interrupt = x86_cpu_do_interrupt;
>      cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
> +
> +    cc->get_arch_id = x86_cpu_get_arch_id;
>  }
>  
>  static const TypeInfo x86_cpu_type_info = {
> 


-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 07/16] cpu: introduce get_arch_id() method and override it for target-i386

[Igor Mammedov]

On Mon, 22 Apr 2013 18:33:48 +0200
Andreas Färber <afaerber@suse.de> wrote:

> Am 16.04.2013 00:12, schrieb Igor Mammedov:
> > get_arch_id() adds possibility for generic code to get guest visible
> > CPU ID without accessing CPUArchState. If target doesn't override it,
> > it will return cpu_index.
> > 
> > Override it on target-i386 to return APIC ID.
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > ---
> >  * it will be used later by new cpu_exists() generic function and
> >    acpi_piix.
> >  * s/cpu_firmware_id/cpu_arch_id/
> > ---
> >  include/qom/cpu.h |  2 ++
> >  qom/cpu.c         |  6 ++++++
> >  target-i386/cpu.c | 10 ++++++++++
> >  3 files changed, 18 insertions(+)
> 
> Eduardo and me had discussed on IRC that apparently the only reason we
> need this generalized "arch_id" is that acpi_piix4 is shared between x86
> and mips. Did you investigate whether we can just access the x86 apic-id
> property instead for x86 and leave mips as-is without hotplug support?
Yep, one possible way is to used so unloved ifdefs in piix4.c and inclusion
of cpu.h in it, which is not nice as well.

BTW:
 * if ACPI will be someday used for on ARM then using cpu_arch_id() when
   building ACPI tables is also justified.
 * it could reused in s360 hotplug patches as well, if you looked at my
   review comment on its CPU hot-plug patch.


> Andreas
> 
> > 
> > diff --git a/include/qom/cpu.h b/include/qom/cpu.h
> > index 6ce6f10..428aaf0 100644
> > --- a/include/qom/cpu.h
> > +++ b/include/qom/cpu.h
> > @@ -45,6 +45,7 @@ typedef struct CPUState CPUState;
> >   * instantiatable CPU type.
> >   * @reset: Callback to reset the #CPUState to its initial state.
> >   * @do_interrupt: Callback for interrupt handling.
> > + * @get_arch_id: Callback for getting architecture depended CPU ID
> >   * @vmsd: State description for migration.
> >   *
> >   * Represents a CPU family or model.
> > @@ -58,6 +59,7 @@ typedef struct CPUClass {
> >  
> >      void (*reset)(CPUState *cpu);
> >      void (*do_interrupt)(CPUState *cpu);
> > +    int64_t (*get_arch_id)(CPUState *cpu);
> >  
> >      const struct VMStateDescription *vmsd;
> >  } CPUClass;
> > diff --git a/qom/cpu.c b/qom/cpu.c
> > index 819986e..a13ddda 100644
> > --- a/qom/cpu.c
> > +++ b/qom/cpu.c
> > @@ -76,6 +76,11 @@ static void cpu_common_realizefn(DeviceState *dev,
> > Error **errp) }
> >  }
> >  
> > +static int64_t cpu_common_get_arch_id(CPUState *cpu)
> > +{
> > +    return cpu->cpu_index;
> > +}
> > +
> >  static void cpu_class_init(ObjectClass *klass, void *data)
> >  {
> >      DeviceClass *dc = DEVICE_CLASS(klass);
> > @@ -83,6 +88,7 @@ static void cpu_class_init(ObjectClass *klass, void
> > *data) 
> >      k->class_by_name = cpu_common_class_by_name;
> >      k->reset = cpu_common_reset;
> > +    k->get_arch_id = cpu_common_get_arch_id;
> >      dc->realize = cpu_common_realizefn;
> >      dc->no_user = 1;
> >  }
> > diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> > index e2302d8..a415fa3 100644
> > --- a/target-i386/cpu.c
> > +++ b/target-i386/cpu.c
> > @@ -2272,6 +2272,14 @@ static void x86_cpu_initfn(Object *obj)
> >      }
> >  }
> >  
> > +static int64_t x86_cpu_get_arch_id(CPUState *cpu)
> > +{
> > +    X86CPU *x86cpu = X86_CPU(cpu);
> > +    CPUX86State *env = &x86cpu->env;
> > +
> > +    return env->cpuid_apic_id;
> > +}
> > +
> >  static void x86_cpu_common_class_init(ObjectClass *oc, void *data)
> >  {
> >      X86CPUClass *xcc = X86_CPU_CLASS(oc);
> > @@ -2286,6 +2294,8 @@ static void x86_cpu_common_class_init(ObjectClass
> > *oc, void *data) 
> >      cc->do_interrupt = x86_cpu_do_interrupt;
> >      cpu_class_set_vmsd(cc, &vmstate_x86_cpu);
> > +
> > +    cc->get_arch_id = x86_cpu_get_arch_id;
> >  }
> >  
> >  static const TypeInfo x86_cpu_type_info = {
> > 
> 
> 

Re: [Qemu-devel] [PATCH 12/16] target-i386: cpu: attach ICC bus to CPU on its creation

[Igor Mammedov]

On Mon, 22 Apr 2013 17:27:29 +0200
Andreas Färber <afaerber@suse.de> wrote:

> Am 22.04.2013 17:20, schrieb Igor Mammedov:
> > On Mon, 22 Apr 2013 17:02:33 +0200
> > Andreas Färber <afaerber@suse.de> wrote:
> > 
> >> Am 16.04.2013 00:12, schrieb Igor Mammedov:
> >>> ... during startup,  so it would be possible to unplug it later
> >>> and set bus_type to TYPE_ICC_BUS for X86CPU type to make device_add
> >>> attach hotplugged CPU to ICC bus.
> >>>
> >>> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> >>
> >> Reviewed-by: Andreas Färber <afaerber@suse.de>
> >>
> >> But still one question...
> >>
> >>> ---
> >>>  target-i386/cpu.c | 7 +++++++
> >>>  1 file changed, 7 insertions(+)
> >>>
> >>> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> >>> index 6d6c527..3b5f90b 100644
> >>> --- a/target-i386/cpu.c
> >>> +++ b/target-i386/cpu.c
> >>> @@ -41,6 +41,7 @@
> >>>  #endif
> >>>  
> >>>  #include "sysemu/sysemu.h"
> >>> +#include "hw/i386/icc_bus.h"
> >>>  #ifndef CONFIG_USER_ONLY
> >>>  #include "hw/xen/xen.h"
> >>>  #include "hw/sysbus.h"
> >>> @@ -1609,6 +1610,7 @@ X86CPU *cpu_x86_create(const char *cpu_model,
> >>> Error **errp) gchar **model_pieces;
> >>>      char *name, *features;
> >>>      Error *error = NULL;
> >>> +    Object *icc_bus = object_resolve_path_type("icc-bus", TYPE_ICC_BUS,
> >>> NULL); 
> >>>      model_pieces = g_strsplit(cpu_model, ",", 2);
> >>>      if (!model_pieces[0]) {
> >>> @@ -1619,6 +1621,10 @@ X86CPU *cpu_x86_create(const char *cpu_model,
> >>> Error **errp) features = model_pieces[1];
> >>>  
> >>>      cpu = X86_CPU(object_new(TYPE_X86_CPU));
> >>> +    if (icc_bus) {
> >>> +        qdev_set_parent_bus(DEVICE(cpu), BUS(icc_bus));
> >>> +        object_unref(OBJECT(cpu));
> >>> +    }
> >>>      env = &cpu->env;
> >>>      env->cpu_model_str = cpu_model;
> >>>  
> >>
> >> You seem to be avoiding making lack of icc-bus an error although you add
> >> it for both PC and q35 PC - which non-ICC use cases are you thinking of?
> > *-user targets, CPU is bus-less there.
> 
> Would you be opposed to #ifdef'ing it then? In particular I am asking
> because of the NULL error argument to the resolve function - for softmmu
> I would prefer to make lack of icc-bus a fatal error.
I'm fine with ifdef-ing there, if there is not objections.
I'll look at making it fatal error on softmmu

> 
> Andreas
> 

Re: [Qemu-devel] [PATCH 05/16] introduce CPU hot-plug notifier

[Igor Mammedov]

On Mon, 22 Apr 2013 14:24:00 +0300
Gleb Natapov <gleb@redhat.com> wrote:

> On Mon, Apr 22, 2013 at 01:09:19PM +0200, Igor Mammedov wrote:
> > On Mon, 22 Apr 2013 14:00:01 +0300
> > Gleb Natapov <gleb@redhat.com> wrote:
> > 
> > > On Tue, Apr 16, 2013 at 12:12:45AM +0200, Igor Mammedov wrote:
> > > > hot-added CPU will be distributed to acpi_piix4 and rtc_cmos
> > > > 
> > > > Signed-off-by: Igor Mammedov <imammedo@redhat.com>
> > > > Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
> > > > Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
> > > > ---
> > > > v2:
> > > >   * move notifier to qom/cpu.c and call it from CPUClass.realize() on
> > > > hotplug
> > > >   * remove get_firmware_id() since it belong to other patch
> > > > ---
> > > >  include/sysemu/sysemu.h |  3 +++
> > > >  qom/cpu.c               | 12 ++++++++++++
> > > >  2 files changed, 15 insertions(+)
> > > > 
> > > > diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> > > > index 6578782..a8c3de1 100644
> > > > --- a/include/sysemu/sysemu.h
> > > > +++ b/include/sysemu/sysemu.h
> > > > @@ -152,6 +152,9 @@ void do_pci_device_hot_remove(Monitor *mon, const
> > > > QDict *qdict); /* generic hotplug */
> > > >  void drive_hot_add(Monitor *mon, const QDict *qdict);
> > > >  
> > > > +/* CPU hotplug */
> > > > +void qemu_register_cpu_added_notifier(Notifier *notifier);
> > > > +
> > > >  /* pcie aer error injection */
> > > >  void pcie_aer_inject_error_print(Monitor *mon, const QObject *data);
> > > >  int do_pcie_aer_inject_error(Monitor *mon,
> > > > diff --git a/qom/cpu.c b/qom/cpu.c
> > > > index 40a4259..819986e 100644
> > > > --- a/qom/cpu.c
> > > > +++ b/qom/cpu.c
> > > > @@ -21,6 +21,17 @@
> > > >  #include "qom/cpu.h"
> > > >  #include "qemu-common.h"
> > > >  #include "sysemu/kvm.h"
> > > > +#include "qemu/notify.h"
> > > > +#include "sysemu/sysemu.h"
> > > > +
> > > > +/* CPU hot-plug notifiers */
> > > > +static NotifierList cpu_added_notifiers =
> > > > +    NOTIFIER_LIST_INITIALIZER(cpu_add_notifiers);
> > > > +
> > > > +void qemu_register_cpu_added_notifier(Notifier *notifier)
> > > > +{
> > > > +    notifier_list_add(&cpu_added_notifiers, notifier);
> > > > +}
> > > >  
> > > >  void cpu_reset_interrupt(CPUState *cpu, int mask)
> > > >  {
> > > > @@ -61,6 +72,7 @@ static void cpu_common_realizefn(DeviceState *dev,
> > > > Error **errp) if (dev->hotplugged) {
> > > >          cpu_synchronize_post_init(CPU(dev));
> > > >          resume_vcpu(CPU(dev));
> > > > +        notifier_list_notify(&cpu_added_notifiers, dev);
> > > Why do it after vcpu is running? May be notifier want to do something
> > > that should be done before vcpu is running.
> > currently there is not need for adding_cpu notifier.
> > This one signals that there is a new fully functional CPU available.
> > 
> You are using it to update cmos information for BIOS. It would be strange
> for newly added vcpu to see incorrect number of cpus in cmos if, for
> some reason, it will check it. The "new cpu is added and ready to tun,
> do your last things before it is launched" callback seams more reasonable
> then "new cpu is running wild" one.
IRC,
I'm moving it between cpu_synchronize_post_init() and resume_vcpu(), since
cpu_synchronize_post_init() pushes apic_state into kvm.

> --
> 			Gleb.

Re: [Qemu-devel] [PATCH 16/16] add cpu-add qmp command and implement CPU hot-add for target-i386

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 2203 bytes --]

On 04/16/2013 02:04 PM, Igor Mammedov wrote:

>>> +#
>>> +# Returns: Nothing on success
>>> +##
>>> +{ 'command': 'cpu-add', 'data': {'id': 'int'} }
>>> +
>>
>> Should be usable from libvirt's perspective, even if hot-plugging more
>> than one cpu requires more than one QMP call.  Do we have a counterpart
>> QMP call to easily determine which cpu ids can still be hotplugged?  If
>> so, should we mention that in the documentation of this command?
> We do not have it yet. Despite interface allowing to plug arbitrary CPU,
> libvirt shouldn't do it in order not to break migration support. Since
> migration target should be started with all CPU from source (including
> hot-plugged ones). And current command line doesn't have means for this.
> 
> I'd propose do implement in libvirt something like:
> 
> hotplug_id = current_cpu_count
> { "execute": "cpu-add", "arguments": { "id": hotplug_id } }
> current_cpu_count += 1
> 
> until arbitrary CPU hotplug and interface for enumerating possible CPUs
> settle down.

Yes, that's pretty much how libvirt already handles hotplug for xen:
libvirt currently maintains its own count of plugged cpus, and
hot[un]plugs the next id so that the online cpus are always contiguous
ids starting from 0.  We are discussing on the libvirt lists ways of
improving libvirt API to expose hotplug of arbitrary ids (after all, a
4-cpu bare metal machine can hotunplug cpu 1 while still leaving 2 and 3
online) - but that can come later, perhaps as qemu 1.6 gains better
support for NUMA layout with better control of what ids should be used.

> 
> There was a proposal for enumerating possible CPUs in previous version,
> but it was target specific, and I was convinced to drop it for 1.5 and
> aim for generic way to expose this information later.
> http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg02286.html
> Opinion from libvirt POV would be nice to have though.

Agreed that deferring more powerful id enumeration for a later patch
series is fine, and shouldn't hold up getting basic hotplug into 1.5.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]

Re: [Qemu-devel] [PATCH 10/16] target-i386: introduce apic-id property

[Andreas Färber]

Am 22.04.2013 18:30, schrieb Igor Mammedov:
> On Mon, 22 Apr 2013 16:05:34 +0200
> Andreas Färber <afaerber@suse.de> wrote:
> 
>> Am 16.04.2013 00:12, schrieb Igor Mammedov:
>>> ... and use it from board level to set APIC ID for CPUs
>>> it creates.
>>>
>>> Signed-off-by: Igor Mammedov <imammedo@redhat.com>
>>> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
>>> ---
>>> Note:
>>>   * pc_new_cpu() function will be reused later in CPU hot-plug hook.
>>
>> Suggest as main commit message to avoid the "...":
>>
>> The property is used from board level to set APIC ID for CPUs it
>> creates. Do so in a new pc_new_cpu() helper, to be reused for hot-plug.
> I'll do on next respin.
> 
>>>
>>> v3:
>>>   * user error_propagate() in property setter
>>> v2:
>>>   * use generic cpu_exists() instead of custom one
>>>   * make apic-id dynamic property, so it won't be possible to use it
>>>     as global property, since it's instance specific
>>> ---
>>>  hw/i386/pc.c      | 25 ++++++++++++++++++++++++-
>>>  target-i386/cpu.c | 42 ++++++++++++++++++++++++++++++++++++++++++
>>>  2 files changed, 66 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
>>> index dc1a78b..cb57878 100644
>>> --- a/hw/i386/pc.c
>>> +++ b/hw/i386/pc.c
>>> @@ -889,9 +889,29 @@ void pc_acpi_smi_interrupt(void *opaque, int irq,
>>> int level) }
>>>  }
>>>  
>>> +static void pc_new_cpu(const char *cpu_model, int64_t apic_id, Error
>>> **errp) +{
>>> +    X86CPU *cpu;
>>> +
>>> +    cpu = cpu_x86_create(cpu_model, errp);
>>> +    if (!cpu) {
>>> +        return;
>>> +    }
>>> +
>>> +    object_property_set_int(OBJECT(cpu), apic_id, "apic-id", errp);
>>> +    object_property_set_bool(OBJECT(cpu), true, "realized", errp);
>>> +
>>> +    if (error_is_set(errp)) {
>>
>> Please use a local Error* variable with error_propagate, otherwise this
>> is fragile.
> done
> 
>>
>>> +        if (cpu != NULL) {
>>> +            object_unref(OBJECT(cpu));
>>> +        }
>>> +    }
>>> +}
>>> +
>>>  void pc_cpus_init(const char *cpu_model)
>>>  {
>>>      int i;
>>> +    Error *error = NULL;
>>>  
>>>      /* init CPUs */
>>>      if (cpu_model == NULL) {
>>> @@ -903,7 +923,10 @@ void pc_cpus_init(const char *cpu_model)
>>>      }
>>>  
>>>      for (i = 0; i < smp_cpus; i++) {
>>> -        if (!cpu_x86_init(cpu_model)) {
>>> +        pc_new_cpu(cpu_model, x86_cpu_apic_id_from_index(i), &error);
>>> +        if (error) {
>>> +            fprintf(stderr, "%s\n", error_get_pretty(error));
>>> +            error_free(error);
>>>              exit(1);
>>>          }
>>>      }
>>> diff --git a/target-i386/cpu.c b/target-i386/cpu.c
>>> index a415fa3..6d6c527 100644
>>> --- a/target-i386/cpu.c
>>> +++ b/target-i386/cpu.c
>>> @@ -1271,6 +1271,45 @@ static void x86_cpuid_set_tsc_freq(Object *obj,
>>> Visitor *v, void *opaque, cpu->env.tsc_khz = value / 1000;
>>>  }
>>>  
>>> +static void x86_cpuid_get_apic_id(Object *obj, Visitor *v, void *opaque,
>>> +                                  const char *name, Error **errp)
>>> +{
>>> +    X86CPU *cpu = X86_CPU(obj);
>>> +    int64_t value = cpu->env.cpuid_apic_id;
>>> +
>>> +    visit_type_int(v, &value, name, errp);
>>> +}
>>> +
>>> +static void x86_cpuid_set_apic_id(Object *obj, Visitor *v, void *opaque,
>>> +                                  const char *name, Error **errp)
>>> +{
>>> +    X86CPU *cpu = X86_CPU(obj);
>>> +    const int64_t min = 0;
>>> +    const int64_t max = UINT32_MAX;
>>> +    Error *error = NULL;
>>> +    int64_t value;
>>> +
>>> +    visit_type_int(v, &value, name, &error);
>>> +    if (error) {
>>> +        error_propagate(errp, error);
>>> +        return;
>>> +    }
>>
>>> +    if (value < min || value > max) {
>>> +        error_setg(&error, "Property %s.%s doesn't take value %" PRId64
>>> +                   " (minimum: %" PRId64 ", maximum: %" PRId64 ")" ,
>>> +                   object_get_typename(obj), name, value, min, max);
>>> +        error_propagate(errp, error);
>>> +        return;
>>> +    }
>>> +
>>> +    if (cpu_exists(value)) {
>>> +        error_setg(&error, "CPU with APIC ID %" PRIi64 " exists", value);
>>> +        error_propagate(errp, error);
>>
>> You could do both error_setg()s directly on errp.
> it was so before, but I was requested to use local error here:
> http://lists.gnu.org/archive/html/qemu-devel/2013-04/msg01606.html

The only thing Paolo requested there (and which I +1) is not to do
error_is_set(errp). That is independent of error_setg() though, which is
followed by a return, so has no functional difference.

Andreas

> 
>>
>>> +        return;
>>> +    }
>>> +    cpu->env.cpuid_apic_id = value;
>>> +}
>>> +
>>>  static int cpu_x86_find_by_name(x86_def_t *x86_cpu_def, const char *name)
>>>  {
>>>      x86_def_t *def;
>>> @@ -2259,6 +2298,9 @@ static void x86_cpu_initfn(Object *obj)
>>>      object_property_add(obj, "tsc-frequency", "int",
>>>                          x86_cpuid_get_tsc_freq,
>>>                          x86_cpuid_set_tsc_freq, NULL, NULL, NULL);
>>> +    object_property_add(obj, "apic-id", "int",
>>> +                        x86_cpuid_get_apic_id,
>>> +                        x86_cpuid_set_apic_id, NULL, NULL, NULL);
>>>  
>>>      env->cpuid_apic_id = x86_cpu_apic_id_from_index(cs->cpu_index);
>>>  
>>
>> Otherwise I like this x86-specific version now!
>>
>> Andreas
>>
> 
> 


-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg