From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35774) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YaPu2-00054h-T5 for qemu-devel@nongnu.org; Tue, 24 Mar 2015 10:34:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YaPty-0007hT-7f for qemu-devel@nongnu.org; Tue, 24 Mar 2015 10:33:58 -0400 Received: from e06smtp13.uk.ibm.com ([195.75.94.109]:37394) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YaPtx-0007hB-Qs for qemu-devel@nongnu.org; Tue, 24 Mar 2015 10:33:54 -0400 Received: from /spool/local by e06smtp13.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 24 Mar 2015 14:33:50 -0000 Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by d06dlp03.portsmouth.uk.ibm.com (Postfix) with ESMTP id 9B69D1B08061 for ; Tue, 24 Mar 2015 14:34:14 +0000 (GMT) Received: from d06av11.portsmouth.uk.ibm.com (d06av11.portsmouth.uk.ibm.com [9.149.37.252]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id t2OEXmMk9568682 for ; Tue, 24 Mar 2015 14:33:48 GMT Received: from d06av11.portsmouth.uk.ibm.com (localhost [127.0.0.1]) by d06av11.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id t2OEXmFh004415 for ; Tue, 24 Mar 2015 08:33:48 -0600 Message-ID: <551175CB.5040105@de.ibm.com> Date: Tue, 24 Mar 2015 15:33:47 +0100 From: Christian Borntraeger MIME-Version: 1.0 References: <1427131923-4670-1-git-send-email-afaerber@suse.de> In-Reply-To: <1427131923-4670-1-git-send-email-afaerber@suse.de> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Subject: Re: [Qemu-devel] [PATCH RFC 0/4] target-i386: PC socket/core/thread modeling, part 1 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: =?UTF-8?B?QW5kcmVhcyBGw6RyYmVy?= , qemu-devel@nongnu.org Cc: Peter Maydell , Eduardo Habkost , Paolo Bonzini , "Jason J. Herne" , Bharata B Rao , Cornelia Huck , Igor Mammedov Am 23.03.2015 um 18:31 schrieb Andreas Färber: > Hello, > > This long-postponed series proposes a hierarchical QOM model of socket > and core objects for the x86 PC machines. Just some comments from the s390 side as we probably want more than status quo in the future as well. Traditionally all CPUs were equal from the software side, but hardware was already organized in a node like fashion. Starting with z10 (2008) the cpu topology was exposed to software (not the memory topology!) which allowed Linux to optimize scheduling based on locality. So on an LPAR you might see something like $ lscpu -e CPU BOOK SOCKET CORE L1d:L1i:L2 ONLINE CONFIGURED POLARIZATION ADDRESS 0 0 0 0 0:0:0 ja ja horizontal 0 1 0 0 1 1:1:1 ja ja horizontal 1 2 0 0 2 2:2:2 ja ja horizontal 2 3 0 1 3 3:3:3 ja ja horizontal 3 4 0 1 4 4:4:4 ja ja horizontal 4 5 0 1 5 5:5:5 ja ja horizontal 5 6 0 2 6 6:6:6 ja ja horizontal 6 7 0 2 7 7:7:7 ja ja horizontal 7 8 0 2 8 8:8:8 ja ja horizontal 8 9 0 3 9 9:9:9 ja ja horizontal 9 10 0 3 10 10:10:10 ja ja horizontal 10 11 0 3 11 11:11:11 ja ja horizontal 11 12 1 4 12 12:12:12 ja ja horizontal 12 13 1 4 13 13:13:13 ja ja horizontal 13 14 1 4 14 14:14:14 ja ja horizontal 14 15 1 5 15 15:15:15 ja ja horizontal 15 Under z/VM the topology is flat (each CPU on a separate node) # lscpu -e CPU BOOK SOCKET CORE L1d:L1i:L2d:L2i ONLINE CONFIGURED POLARIZATION ADDRESS 0 0 0 0 0:0:0:0 ja ja horizontal 0 1 1 1 1 1:1:1:1 ja ja horizontal 1 2 2 2 2 2:2:2:2 ja ja horizontal 2 Thats what we have today in KVM as well. Now... > > Background is that due to qdev limitations we had to introduce an ICC bus > to be able to hot-add CPUs and their APICs. By now this limitation could be > resolved via a QOM hotplug handler interface. > > However, the QOM hotplug model is associated with having link<> properties. > Given that physically we cannot hot-add hyperthreads but full CPU sockets, > this series prepares the underbelly for having those link properties be of > the new type X86CPUSocket rather than X86CPU. > > As final step in this series, the X86CPU allocation model is changed to be > init'ed in-place, as part of an "atomic" socket object. A follow-up will be > to attempt the same in-place allocation model for the APIC; one difficulty > there is that several places do a NULL check for the APIC pointer as quick > way of telling whether an APIC is present or not. > > NOT IN THIS SERIES is converting cpu-add to the same socket/core/thread model > and initializing them in-place. The current PoC implementation assumes that > CPUs get added sequentially and that the preceding CPU can be used to obtain > the core via unclean QOM parent accesses. > IIUC that must be changed so that an arbitrary thread added via cpu-add > creates the full socket and core(s). That would work best if indexed link<> > properties could be used. That's an undecided design question of whether > we want to have /machine/cpu-socket[n] or whether it makes sense to integrate > NUMA modeling while at it and rather have /machine/node[n]/socket[m]. ...looking into the future, we probably want to be able to model a cpu node topology as newer machines are getting bigger and bigger and the locality gets more important. So maybe we want to have a KVM-guest with 40 vCPUs and pin 20 vCPUs on host book (node) 0 and 20 vcpu on host book 1. - so node is of interest. Regarding threads: z13 has SMT2. Right now with upstream code, we can fan out at the lpar level so that each thread becomes a host CPU and that is used to back guest vCPUs. The alternative to pass a full host core to the guest and do the fan out in the guest is currently not supported. Regarding cpu hotplug: This work is currently halted due to other things but we plan to continue to work on that. I will have a look at your code. Thanks for working on this. > > Note that this socket modeling is not only PC-specific in the softmmu sense > but also in that not every X86CPU must be on a socket (e.g., Quark X1000). > Therefore it was a concious decision to not label some things target-i386 > and to place code in pc.c rather than cpu.c. > > Further note that it is ignored here that -smp enforces that AMD CPUs don't > have hyperthreads, i.e. AMD X86CPUs will have only one thread[n] child<>. > > Context: > > qemu.git master > "pc: Ensure non-zero CPU ref count after attaching to ICC bus" > -> this series adding socket/core objects > cpu-add conversion > APIC cleanups > > Available for testing here: > git://github.com/afaerber/qemu-cpu.git qom-cpu-x86-sockets-1.v1 > https://github.com/afaerber/qemu-cpu/commits/qom-cpu-x86-sockets-1.v1 > > Regards, > Andreas > > Cc: Eduardo Habkost > Cc: Igor Mammedov > Cc: Paolo Bonzini > Cc: Peter Maydell > Cc: Bharata B Rao > Cc: Christian Borntraeger > > Andreas Färber (4): > cpu: Prepare Socket container type > target-i386: Prepare CPU socket/core abstraction > pc: Create sockets and cores for CPUs > pc: Create initial CPUs in-place > > hw/cpu/Makefile.objs | 2 +- > hw/cpu/socket.c | 21 ++++++++++ > hw/i386/Makefile.objs | 1 + > hw/i386/cpu-core.c | 45 +++++++++++++++++++++ > hw/i386/cpu-socket.c | 45 +++++++++++++++++++++ > hw/i386/pc.c | 95 ++++++++++++++++++++++++++++++++++++++++---- > include/hw/cpu/socket.h | 14 +++++++ > include/hw/i386/cpu-core.h | 29 ++++++++++++++ > include/hw/i386/cpu-socket.h | 29 ++++++++++++++ > 9 files changed, 272 insertions(+), 9 deletions(-) > create mode 100644 hw/cpu/socket.c > create mode 100644 hw/i386/cpu-core.c > create mode 100644 hw/i386/cpu-socket.c > create mode 100644 include/hw/cpu/socket.h > create mode 100644 include/hw/i386/cpu-core.h > create mode 100644 include/hw/i386/cpu-socket.h >