[Qemu-devel] [PATCH v4 00/13] pc: q35: x2APIC support in kvm_apic mode

[Igor Mammedov <imammedo@redhat.com>]

Changes since v3:
  - fix endianness issues when filling MADT/SRAT entries (Eduardo)
  - squash "acpi: cphp: support x2APIC entry in cpu._MAT" into
    "pc: acpi: x2APIC support for MADT table and _MAT method" (Eduardo)
  - keep assert() as it doesn't affect x2APIC cpus in bochs_bios_init() (Eduardo)
  - restore kvm_has_x2apic_api() and use it to avoid side-effects
    of kvm_enable_x2apic(). x2APIC API will be enabled by iommu
    if it's present or not enabled at all. (Radim)
  - add intel iommu example to error message (Eduardo)

Changes since v2:
  - rebase on top of EIM fixes
  - drop kvm_has_x2apic_ids() and reuse kvm_enable_x2apic()
    from Radim's EIM fixes
  - fix hang on reboot in BIOS due to not updated 'etc/boot-cpus' fwcfg file
    after CPU hotplug
  - drop not used anymore pc_present_cpus_count() and incrementally count
    present VCPUs as they are added/removed at (un)plug callbacks time

Changes since v1:
  - rebase on top of 2.7
  - drop add 2.8 machine and linux headers update patches
  - drop numa related patches (will post separately as unrelated)
  - change default mc->maxcpus only for q35

Changes since RFC:
  - use new KVM_CAP_X2APIC_API to detect x2APIC IDs support
  - rebase on top of 2.7-rc1, since many deps were merged
  - fix etc/boot-cpus to account for -device provided cpus
  - include not yet merged _PXM fix as prereq
  - add 2.8 machine type and bump up maxcpus count since it

Series extends current CPU/kvm_apic/Q35 machine
code to support x2APIC and upto 288 VCPUs when QEMU
is used with KVM's lapic.

Due to FW_CFG_MAX_CPUS (which is actually apic_id_limit)
being limited to uint16_t, the max possible APIC ID is
limitted to 2^16 with this series but that should
be sufficient for bumping VCPUs number for quite a while.

Not yet fixed kernel x2APIC issues:
CPU hotplug doesn't work for CPUs where APIC ID > 254
(guest has to be rebooted to pickup hotplugged CPUs)
due to kernel_irqchip loosing directed IPIs (INIT/SIPI)
to APICs above 254 as a hotplugged CPU is in after power-on
state (i.e. not in x2APIC mode).
Radim's going to post KVM patch to fix it and on top of it
I'll post a followup QEMU sanity check patch to detect if host
supports directed IPIs to CPUs with APIC IDs above 254 in
after power-on state.


Tested with following CLI:
 QEMU -M q35 -enable-kvm \
      -device intel-iommu,intremap=on,eim=on -machine kernel_irqchip=split \
      -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 \
      -device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0       \
      -bios x2apic_bios.bin

v3 for reference:
[PATCH v3 00/13] pc: q35: x2APIC support in kvm_apic mode
https://lists.gnu.org/archive/html/qemu-devel/2016-10/msg02865.html

git gree for testing:
https://github.com/imammedo/qemu.git x2apic_v4

To play with the feature, one would also need x2apic enabled
seabios counterpart:
https://github.com/imammedo/seabios.git x2apic_v4

Cc: kraxel@redhat.com
Cc: ehabkost@redhat.com
Cc: liuxiaojian6@huawei.com
Cc: mst@redhat.com
Cc: rkrcmar@redhat.com
Cc: peterx@redhat.com
Cc: kevin@koconnor.net
Cc: pbonzini@redhat.com
Cc: lersek@redhat.com
Cc: chao.gao@intel.com

Igor Mammedov (13):
  pc: acpi: x2APIC support for MADT table and _MAT method
  pc: acpi: x2APIC support for SRAT table
  acpi: cphp: force switch to modern cpu hotplug if APIC ID > 254
  pc: leave max apic_id_limit only in legacy cpu hotplug code
  pc: apic_common: extend APIC ID property to 32bit
  pc: apic_common: restore APIC ID to initial ID on reset
  pc: apic_common: reset APIC ID to initial ID when switching into
    x2APIC mode
  pc: kvm_apic: pass APIC ID depending on xAPIC/x2APIC mode
  pc: clarify FW_CFG_MAX_CPUS usage comment
  increase MAX_CPUMASK_BITS from 255 to 288
  pc: add 'etc/boot-cpus' fw_cfg file for machine with more than 255
    CPUs
  pc: require IRQ remapping and EIM if there could be x2APIC CPUs
  pc: q35: bump max_cpus to 288

 include/hw/acpi/acpi-defs.h     |  29 +++++++++++
 include/hw/i386/apic_internal.h |   3 +-
 include/hw/i386/pc.h            |   2 +
 include/sysemu/sysemu.h         |   2 +-
 target-i386/cpu.h               |   1 +
 target-i386/kvm_i386.h          |   1 +
 hw/acpi/cpu.c                   |   5 ++
 hw/acpi/cpu_hotplug.c           |  17 ++++--
 hw/arm/virt.c                   |   2 +-
 hw/i386/acpi-build.c            | 112 ++++++++++++++++++++++++++++------------
 hw/i386/kvm/apic.c              |  12 ++++-
 hw/i386/pc.c                    |  74 +++++++++++++++-----------
 hw/i386/pc_q35.c                |   2 +
 hw/intc/apic_common.c           |  52 ++++++++++++++++++-
 hw/ppc/spapr.c                  |   2 +-
 target-i386/cpu.c               |   2 +-
 target-i386/kvm.c               |  13 +++--
 17 files changed, 250 insertions(+), 81 deletions(-)

-- 
2.7.4