[Qemu-devel] [RFC PATCH v2 00/23] CPU and Memory hotplug for PowerPC sPAPR guests

[Bharata B Rao <bharata@linux.vnet.ibm.com>]

Hi,

This is the version 2 of the patchset that provides CPU and memory hotplug
support for PowerPC sPAPR guests.

These patches apply against spapr-hotplug-pci-v7 branch of Michael Roth's
PCI hotplug tree (git://github.com/mdroth/qemu). I am basing against
Michael's tree because that has the DR connector base code that is required
to do any hotplug in sPAPR.

I have switched to socket level semantics as suggested by Andreas Farber
(http://lists.gnu.org/archive/html/qemu-devel/2015-01/msg04410.html). What
this means is that I now add full sockets (consisting of cores and threads)
at once in response to device_add command for CPU hotplug. This is enabled
by borrowing one patch from Andreas' WIP tree and building PowerPC parts
on top of it.
(https://github.com/afaerber/qemu-cpu/commit/d33c7caa6471507266d02208ff98f72d4990092c)
I would have ideally liked to post this v2 after Andreas formally
posts his socket level abstraction patchset. But I thought may be I could
get some more review while waiting for his post.

I don't expect anyone to try this out yet, but here is my git tree
for the bravehearts :)

spapr-hotplug branch at https://github.com/bharata/qemu/

Major changes in this patchset (v2)
-----------------------------------
- Switched to socket level semantics that is being proposed by Andreas.
- Reorganized CPU device tree generation code for sPAPR so that same
  code is used in the normal and hotplug path.
- Common CPU init code shared between both bootpath and hotplug path.
- Added documentation about new device tree nodes that are being
  added for hotplug.
- CPU hotplug on LE guest now works.
- Hotplugging of more memory than minimum size (256MB) at once works.
- Enforced alignment requirements for memory hotplug.
- Fixed generic CPU enumeration code to enable proper hot removal
  of CPUs.
- Fixed a crash that was happening when a VM which undergone CPU
  removal is rebooted.
- Not mixing sPAPR code with generic ppc code now.
- Addressed most of the review comments from v1 except a few.

v1: http://lists.gnu.org/archive/html/qemu-devel/2015-01/msg00611.html
v0: http://lists.nongnu.org/archive/html/qemu-devel/2014-09/msg00752.html

TODOs
-----
- Share code between pc_dimm_plug() and spapr_memory_plug().
- Make the algorithm that looks up the NUMA node given the physical address
  more efficient.
- Test/enable migration after hotplug.
- Address a few object reference leaks.
- And of course, much more wider testing.

Example Usage
-------------
CPU hotplug:

Cmdline: -smp 16,maxcpus=32,sockets=4,cores=2,threads=2
Monitor: (qemu) device_add powerpc64-cpu-socket,id=sock5

Memory hotplug: (same semantics as x86)

Cmdline: -m 2G,slots=4,maxmem=4G
Monitor: (qemu) object_add memory-backend-ram,id=ram1,size=512M
         (qemu) device_add pc-dimm,id=dimm1,memdev=ram1

After the above steps, the hotplug action needs to be completed by
using rtas_errd and drmgr utilities (part of powerpc-utils package).
With some changes, I am able to get both memory and hotplug parts
working in powerpc-utils, but I expect Nathan Fontenot to take care
of these parts more properly. Nathan has RFC patches to the guest
kernel that complete the hotplug action for both CPU and memory
completely within the guest kernel. When that is available, these user
space tools will not be needed.

Andreas Färber (1):
  cpu: Prepare Socket container type

Bharata B Rao (20):
  spapr: Add DRC dt entries for CPUs
  spapr: Consider max_cpus during xics initialization
  spapr: Support ibm,lrdr-capacity device tree property
  spapr: Reorganize CPU dt generation code
  spapr: Consolidate cpu init code into a routine
  ppc: Prepare CPU socket/core abstraction
  spapr: Add CPU hotplug handler
  ppc: Update cpu_model in MachineState
  ppc: Create sockets and cores for CPUs
  spapr: CPU hotplug support
  cpus: Add Error argument to cpu_exec_init()
  cpus: Convert cpu_index into a bitmap
  ppc: Move cpu_exec_init() call to realize function
  xics_kvm: Don't enable KVM_CAP_IRQ_XICS if already enabled
  xics_kvm: Add cpu_destroy method to XICS
  spapr: CPU hot unplug support
  spapr: Remove vCPU objects after CPU hot unplug
  spapr: Initialize hotplug memory address space
  spapr: Support ibm,dynamic-reconfiguration-memory
  spapr: Memory hotplug support

Gu Zheng (1):
  cpus: Reclaim vCPU objects

Michael Roth (1):
  spapr: enable PHB/CPU/LMB hotplug for pseries-2.3

 cpus.c                            |   44 ++
 default-configs/ppc64-softmmu.mak |    1 +
 docs/specs/ppc-spapr-hotplug.txt  |   66 +++
 exec.c                            |   39 +-
 hw/cpu/Makefile.objs              |    2 +-
 hw/cpu/socket.c                   |   21 +
 hw/intc/xics.c                    |   12 +
 hw/intc/xics_kvm.c                |   19 +
 hw/ppc/Makefile.objs              |    1 +
 hw/ppc/cpu-core.c                 |   63 +++
 hw/ppc/cpu-socket.c               |   62 +++
 hw/ppc/mac_newworld.c             |   10 +-
 hw/ppc/mac_oldworld.c             |    7 +-
 hw/ppc/ppc440_bamboo.c            |    7 +-
 hw/ppc/prep.c                     |    7 +-
 hw/ppc/spapr.c                    | 1014 +++++++++++++++++++++++++++++--------
 hw/ppc/spapr_events.c             |   11 +-
 hw/ppc/spapr_hcall.c              |   51 +-
 hw/ppc/spapr_rtas.c               |   29 +-
 hw/ppc/virtex_ml507.c             |    7 +-
 include/exec/exec-all.h           |    2 +-
 include/hw/cpu/socket.h           |   14 +
 include/hw/ppc/cpu-core.h         |   32 ++
 include/hw/ppc/cpu-socket.h       |   32 ++
 include/hw/ppc/spapr.h            |   37 +-
 include/hw/ppc/xics.h             |    3 +
 include/qom/cpu.h                 |   19 +
 include/sysemu/kvm.h              |    1 +
 kvm-all.c                         |   57 ++-
 kvm-stub.c                        |    5 +
 linux-headers/linux/kvm.h         |    1 +
 target-alpha/cpu.c                |    2 +-
 target-arm/cpu.c                  |    2 +-
 target-cris/cpu.c                 |    2 +-
 target-i386/cpu.c                 |    2 +-
 target-lm32/cpu.c                 |    2 +-
 target-m68k/cpu.c                 |    2 +-
 target-microblaze/cpu.c           |    2 +-
 target-mips/cpu.c                 |    2 +-
 target-moxie/cpu.c                |    2 +-
 target-openrisc/cpu.c             |    2 +-
 target-ppc/cpu.h                  |    1 +
 target-ppc/kvm.c                  |    7 +
 target-ppc/kvm_ppc.h              |    6 +
 target-ppc/translate_init.c       |   55 +-
 target-s390x/cpu.c                |    2 +-
 target-sh4/cpu.c                  |    2 +-
 target-sparc/cpu.c                |    2 +-
 target-tricore/cpu.c              |    2 +-
 target-unicore32/cpu.c            |    2 +-
 target-xtensa/cpu.c               |    2 +-
 51 files changed, 1503 insertions(+), 274 deletions(-)
 create mode 100644 hw/cpu/socket.c
 create mode 100644 hw/ppc/cpu-core.c
 create mode 100644 hw/ppc/cpu-socket.c
 create mode 100644 include/hw/cpu/socket.h
 create mode 100644 include/hw/ppc/cpu-core.h
 create mode 100644 include/hw/ppc/cpu-socket.h

-- 
2.1.0