Re: [Qemu-devel] CPU hotplug

[Bharata B Rao <bharata@linux.vnet.ibm.com>]

On Mon, Feb 01, 2016 at 04:35:17PM +1100, David Gibson wrote:
> Hi,
> 
> It seems to me we're getting rather bogged down in how to proceed with
> an improved CPU hotplug (and hot unplug) interface, both generically
> and for ppc in particular.
> 
> So here's a somewhat more concrete suggestion of a way forward, to see
> if we can get some consensus.
> 
> The biggest difficulty I think we're grappling with is that device-add
> is actually *not* a great interface to cpu hotplug.  Or rather, it's
> not great as the _only_ interface: in order to represent the many
> different constraints on how cpus can be plugged on various platforms,
> it's natural to use a heirarchy of cpu core / socket / package types
> specific to the specific platform or real-world cpu package being
> modeled.  However, for the normal case of a regular homogenous (and at
> least slightly para-virtualized) server, that interface is nasty for
> management layers because they have to know the right type to
> instantiate.
> 
> To address this, I'm proposing this two layer interface:
> 
> Layer 1: Low-level, device-add based
> 
>     * a new, generic cpu-package QOM type represents a group of 1 or
>       more cpu threads which can be hotplugged as a unit
>     * cpu-package is abstract and can't be instantiated directly
>     * archs and/or individual platforms have specific subtypes of
>       cpu-package which can be instantiated
>     * for platforms attempting to be faithful representations of real
>       hardware these subtypes would match the specific characteristics
>       of the real hardware devices.  In addition to the cpu threads,
>       they may have other on chip devices as sub-objects.
>     * for platforms which are paravirtual - or which have existing
>       firmware abstractions for cpu cores/sockets/packages/whatever -
>       these could be more abstract, but would still be tied to that
>       platform's constraints
>     * Depending on the platform the cpu-package object could have
>       further internal structure (e.g. a package object representing a
>       socket contains package objects representing each core, which in
>       turn contain cpu objects for each thread)
>         * Some crazy platform that has multiple daughterboards each with
>           several multi-chip-modules each with several chips, each
> 	  with several cores each with several threads could represent
> 	  that too.
> 
> What would be common to all the cpu-package subtypes is:
>     * A boolean "present" attribute ("realized" might already be
>       suitable, but I'm not certain)
>     * A generic means of determining the number of cpu threads in the
>       package, and enumerating those
>     * A generic means of determining if the package is hotpluggable or
>       not
>     * They'd get listed in a standard place in the QOM tree
> 
> This interface is suitable if you want complete control over
> constructing the system, including weird cases like heterogeneous
> machines (either totally different cpu types, or just different
> numbers of threads in different packages).
> 
> The intention is that these objects would never look at the global cpu
> type or sockets/cores/threads numbers.  The next level up would
> instead configure the packages to match those for the common case.
> 
> Layer 2: Higher-level
> 
>     * not all machine types need support this model, but I'd expect
>       all future versions of machine types designed for production use
>       to do so
>     * machine types don't construct cpu objects directly
>     * instead they create enough cpu-package objects - of a subtype
>       suitable for this machine - to provide maxcpus threads
>     * the machine type would set the "present" bit on enough of the
>       cpu packages to provide the base number of cpu threads

In the generic cpu-core RFC that I posted last year
(https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg01526.html),
I did have backend objects (which I called them sockets) into which
the generic cpu-core device would fit it and I used the QOM links to
bring out the notion of cpu-core device populating the socket.

I had the sockets as backend objects and created as many of them as needed
upfront to fit the max_cpus. These objects weren't exposed them to the user,
but instead the cpu-core device was exposed to the user.

However, I like the current proposal where Layer 2 interface is exposed to the
user and letting archs build up the CPU topology underneath in the manner
that they deem fit for the arch.

Regards,
Bharata.