From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33167) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1adUM9-0002Eh-Gv for qemu-devel@nongnu.org; Tue, 08 Mar 2016 22:00:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1adUM7-0007Zr-KJ for qemu-devel@nongnu.org; Tue, 08 Mar 2016 22:00:13 -0500 Date: Wed, 9 Mar 2016 13:58:53 +1100 From: David Gibson Message-ID: <20160309025853.GJ22546@voom.fritz.box> References: <1457074461-14285-1-git-send-email-bharata@linux.vnet.ibm.com> <1457074461-14285-9-git-send-email-bharata@linux.vnet.ibm.com> <20160307034906.GF22546@voom.fritz.box> <20160307062942.GF5054@in.ibm.com> <20160308042739.GW22546@voom.fritz.box> <20160308103708.718a1d69@nial.brq.redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Or8LRHETumScJgaM" Content-Disposition: inline In-Reply-To: <20160308103708.718a1d69@nial.brq.redhat.com> Subject: Re: [Qemu-devel] [RFC PATCH v1 08/10] spapr: CPU hotplug support List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Igor Mammedov Cc: mjrosato@linux.vnet.ibm.com, agraf@suse.de, thuth@redhat.com, pkrempa@redhat.com, ehabkost@redhat.com, aik@ozlabs.ru, armbru@redhat.com, qemu-devel@nongnu.org, borntraeger@de.ibm.com, qemu-ppc@nongnu.org, Bharata B Rao , pbonzini@redhat.com, afaerber@suse.de, mdroth@linux.vnet.ibm.com --Or8LRHETumScJgaM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Mar 08, 2016 at 10:37:08AM +0100, Igor Mammedov wrote: > On Tue, 8 Mar 2016 15:27:39 +1100 > David Gibson wrote: >=20 > > On Mon, Mar 07, 2016 at 11:59:42AM +0530, Bharata B Rao wrote: > > > On Mon, Mar 07, 2016 at 02:49:06PM +1100, David Gibson wrote: =20 > > > > On Fri, Mar 04, 2016 at 12:24:19PM +0530, Bharata B Rao wrote: =20 > > > > > Set up device tree entries for the hotplugged CPU core and use the > > > > > exising EPOW event infrastructure to send CPU hotplug notificatio= n to > > > > > the guest. > > > > >=20 > > > > > Signed-off-by: Bharata B Rao > > > > > --- > > > > > hw/ppc/spapr.c | 73 +++++++++++++++++++++++++++= +++++++++++++- > > > > > hw/ppc/spapr_cpu_core.c | 60 +++++++++++++++++++++++++++= ++++++ > > > > > hw/ppc/spapr_events.c | 3 ++ > > > > > hw/ppc/spapr_rtas.c | 24 ++++++++++++++ > > > > > include/hw/ppc/spapr.h | 4 +++ > > > > > include/hw/ppc/spapr_cpu_core.h | 2 ++ > > > > > 6 files changed, 165 insertions(+), 1 deletion(-) > > > > >=20 > > > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c > > > > > index 5acb612..6c4ac50 100644 > > > > > --- a/hw/ppc/spapr.c > > > > > +++ b/hw/ppc/spapr.c > > > > > @@ -603,6 +603,18 @@ static void spapr_populate_cpu_dt(CPUState *= cs, void *fdt, int offset, > > > > > size_t page_sizes_prop_size; > > > > > uint32_t vcpus_per_socket =3D smp_threads * smp_cores; > > > > > uint32_t pft_size_prop[] =3D {0, cpu_to_be32(spapr->htab_shi= ft)}; > > > > > + sPAPRMachineClass *smc =3D SPAPR_MACHINE_GET_CLASS(qdev_get_= machine()); > > > > > + sPAPRDRConnector *drc; > > > > > + sPAPRDRConnectorClass *drck; > > > > > + int drc_index; > > > > > + > > > > > + if (smc->dr_cpu_enabled) { > > > > > + drc =3D spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE= _CPU, index); > > > > > + g_assert(drc); > > > > > + drck =3D SPAPR_DR_CONNECTOR_GET_CLASS(drc); > > > > > + drc_index =3D drck->get_index(drc); > > > > > + _FDT((fdt_setprop_cell(fdt, offset, "ibm,my-drc-index", = drc_index))); > > > > > + } > > > > > =20 > > > > > /* Note: we keep CI large pages off for now because a 64K ca= pable guest > > > > > * provisioned with large pages might otherwise try to map a= qemu > > > > > @@ -987,6 +999,16 @@ static void spapr_finalize_fdt(sPAPRMachineS= tate *spapr, > > > > > _FDT(spapr_drc_populate_dt(fdt, 0, NULL, SPAPR_DR_CONNEC= TOR_TYPE_LMB)); > > > > > } > > > > > =20 > > > > > + if (smc->dr_cpu_enabled) { > > > > > + int offset =3D fdt_path_offset(fdt, "/cpus"); > > > > > + ret =3D spapr_drc_populate_dt(fdt, offset, NULL, > > > > > + SPAPR_DR_CONNECTOR_TYPE_CPU); > > > > > + if (ret < 0) { > > > > > + error_report("Couldn't set up CPU DR device tree pro= perties"); > > > > > + exit(1); > > > > > + } > > > > > + } > > > > > + > > > > > _FDT((fdt_pack(fdt))); > > > > > =20 > > > > > if (fdt_totalsize(fdt) > FDT_MAX_SIZE) { > > > > > @@ -1181,7 +1203,7 @@ static void ppc_spapr_reset(void) > > > > > =20 > > > > > } > > > > > =20 > > > > > -static void spapr_cpu_reset(void *opaque) > > > > > +void spapr_cpu_reset(void *opaque) > > > > > { > > > > > sPAPRMachineState *spapr =3D SPAPR_MACHINE(qdev_get_machine(= )); > > > > > PowerPCCPU *cpu =3D opaque; > > > > > @@ -1622,6 +1644,8 @@ static void spapr_boot_set(void *opaque, co= nst char *boot_device, > > > > > void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, E= rror **errp) > > > > > { > > > > > CPUPPCState *env =3D &cpu->env; > > > > > + CPUState *cs =3D CPU(cpu); > > > > > + int i; > > > > > =20 > > > > > /* Set time-base frequency to 512 MHz */ > > > > > cpu_ppc_tb_init(env, TIMEBASE_FREQ); > > > > > @@ -1646,6 +1670,14 @@ void spapr_cpu_init(sPAPRMachineState *spa= pr, PowerPCCPU *cpu, Error **errp) > > > > > } > > > > > } > > > > > =20 > > > > > + /* Set NUMA node for the added CPUs */ > > > > > + for (i =3D 0; i < nb_numa_nodes; i++) { > > > > > + if (test_bit(cs->cpu_index, numa_info[i].node_cpu)) { > > > > > + cs->numa_node =3D i; > > > > > + break; > > > > > + } > > > > > + } > > > > > + =20 > > > >=20 > > > > This hunk seems like it belongs in a different patch. =20 > > >=20 > > > It appears that this would be needed by other archs also to set the > > > NUMA node for the hot-plugged CPU. How about make an API out of this > > > and use this something like below ? Igor ? =20 > >=20 > > Is there a way we could put this in the the CPU thread initialization > > itself? Rather than requiring every platform to call a helper. > I'd suggest hotplugable CPU entity to have 'node' property, like we have > in pc-dimm. Ok. Do you think that makes sense for the base core type (which will sometimes be a hotpluggable CPU entity and sometimes might not be)? > However machine owns numa mapping, so setting it from thread > initialization seems to be wrong. Ah.. good point. > Could that be done from machine's plug() handler (spapr_core_plug)? Yes, I think so. The core might need a "set node" method, but we should be able to control this from the plug() handler. > Also I notice that there is in no way to check/set 'address' properties > at machine level before calling cpu->realize(), which makes us > to attempt checking them inside cpu->realize() or at post realize time > in HotplugHandler(machine)->plug() hook with following full rollback > in case of failure. > Maybe we need add an HotplugHandler->pre_realize() hook to allow machine > verify/set properties of CPU before it's realized. I don't really see what the problem is with checking inside realize(). --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --Or8LRHETumScJgaM Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJW35FtAAoJEGw4ysog2bOSQS0QAL9/DH2WC9M16S/OeyiXvL4P nS1GnFE8K2rzO80iQlku5kijGGJp2qbq7Z1+ZSTFbBYb9k0Fcq10s7dwgwkSmgKW 70budHa4NEsg1IiwmjpRf5wvYatY7Eyz4BdB26/AI63zpa19z/Hjw9/Ll8r7NKy3 QR+rOpq036Vr4SQzWuh8dxush3ZOxhzZwBKZPKbHWP04GSIoTzilSm1CH+pj+VpT gZnYOITN5bh4g9/G9HAToiJ5o6e6b+2So2SAaIbPJmm6CPImxS3KBiRPlJcDQDMU ToA0M8GDCYC2VidaDH7g+dPOkMTeL4LdQhIxAkzhnVFgtrSYIzx/BL1Tx2mguimv MyrpzFOCryuwLu2LV1nCJ1P0UPKfVduhpKEBUi67mkh/V1aDMkxYf/7DVvxPOsyI mx2PD4lOQK4Q/RmooGWR8TH1CpxNWcczkRxJVXsMZ+5OlfMzILimjp3e/NRkXcK6 j/ekY5+XJPektsPbM1ykSW5fku8nzlr9rX7VDysJ+qdM8Uzw4cAza4P9c1cQCsVq cpBu9JjNz27ncomsNdCStwZJJhAO//ylvl0CWhC/GV0Bh+DKH3iVNCKPCvhvOZjI xliU/py4vPDxee8Sh/i6p1u5a+NmEFopXoccNUqcrKrQP6bpr1CeUC6155GPEqTN wSfNIo5uow6LOSpb3LZb =Tmx7 -----END PGP SIGNATURE----- --Or8LRHETumScJgaM--