From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43808) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bMRnH-0004LZ-JP for qemu-devel@nongnu.org; Sun, 10 Jul 2016 23:22:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bMRnE-00031I-8H for qemu-devel@nongnu.org; Sun, 10 Jul 2016 23:22:03 -0400 Date: Mon, 11 Jul 2016 13:23:39 +1000 From: David Gibson Message-ID: <20160711032339.GK16355@voom.fritz.box> References: <1467903025-13383-1-git-send-email-bharata@linux.vnet.ibm.com> <1467903025-13383-4-git-send-email-bharata@linux.vnet.ibm.com> <20160707181131.3d9926fc@bahia.lan> <20160708052533.GN14675@voom.fritz.box> <20160708094647.6267415c@bahia.lan> <20160708075907.GS14675@voom.fritz.box> <20160708172424.0f37f508@bahia.lan> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6K2R/cS9K4qvcBNq" Content-Disposition: inline In-Reply-To: <20160708172424.0f37f508@bahia.lan> Subject: Re: [Qemu-devel] [RFC PATCH v2 3/5] spapr: Set stable_cpu_id for threads of CPU cores List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Greg Kurz Cc: Bharata B Rao , qemu-devel@nongnu.org, qemu-ppc@nongnu.org, imammedo@redhat.com, nikunj@linux.vnet.ibm.com, pbonzini@redhat.com --6K2R/cS9K4qvcBNq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jul 08, 2016 at 05:24:24PM +0200, Greg Kurz wrote: > On Fri, 8 Jul 2016 17:59:07 +1000 > David Gibson wrote: >=20 > > On Fri, Jul 08, 2016 at 09:46:47AM +0200, Greg Kurz wrote: > > > On Fri, 8 Jul 2016 15:25:33 +1000 > > > David Gibson wrote: > > > =20 > > > > On Thu, Jul 07, 2016 at 06:11:31PM +0200, Greg Kurz wrote: =20 > > > > > On Thu, 7 Jul 2016 20:20:23 +0530 > > > > > Bharata B Rao wrote: > > > > > =20 > > > > > > Conditonally set stable_cpu_id for CPU threads that are created= as part > > > > > > of spapr CPU cores. The use of stable_cpu_id is enabled for pse= ries-2.7 > > > > > > onwards. > > > > > > =20 > > > > >=20 > > > > > The last sentence is a bit confusing since the enablement actuall= y happens > > > > > in patch 5/5. > > > > > =20 > > > > > > Signed-off-by: Bharata B Rao > > > > > > --- > > > > > > hw/ppc/spapr_cpu_core.c | 7 +++++++ > > > > > > 1 file changed, 7 insertions(+) > > > > > >=20 > > > > > > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c > > > > > > index b104778..0ec3513 100644 > > > > > > --- a/hw/ppc/spapr_cpu_core.c > > > > > > +++ b/hw/ppc/spapr_cpu_core.c > > > > > > @@ -293,8 +293,15 @@ static void spapr_cpu_core_realize(DeviceS= tate *dev, Error **errp) > > > > > > for (i =3D 0; i < cc->nr_threads; i++) { > > > > > > char id[32]; > > > > > > obj =3D sc->threads + i * size; > > > > > > + CPUState *cs; > > > > > > =20 > > > > > > object_initialize(obj, size, typename); > > > > > > + cs =3D CPU(obj); > > > > > > + > > > > > > + /* Use core_id (which is actually cpu_dt_id) as stable= CPU id */ =20 > > > > >=20 > > > > > It isn't what I had in mind. More something like below: > > > > >=20 > > > > > In ppc_spapr_init(): > > > > >=20 > > > > > for (i =3D 0; i < spapr_max_cores; i++) { > > > > > spapr->cores[i]->stable_id =3D i * smp_threads; > > > > > } > > > > >=20 > > > > >=20 > > > > > In spapr_cpu_core_realize(): > > > > >=20 > > > > > for (j =3D 0; j < cc->nr_threads; j++) { > > > > > stable_cpu_id =3D cc->stable_id + j; > > > > > } > > > > >=20 > > > > > So we need to introduce cc->stable_id. =20 > > > >=20 > > > > No, we don't. Cores have had a stable ID since they were introduce= d. > > > > =20 > > >=20 > > > I agree core_dt_id is stable but it is a DT concept. =20 > >=20 > > There is no core_dt_id. There's just core-id, which is machine > > assigned (via the query hotpluggable cpus interface) and stable. > >=20 > > > static void ppc_spapr_init(MachineState *machine) > > > { > > > [...] > > > for (i =3D 0; i < spapr_max_cores; i++) { > > > int core_dt_id =3D i * smt; =20 > >=20 > > ..uh, ok, except for that poorly named variable. But that's because > > this is in the machine type, and it knows it's going to use the same > > ids to give to the core object and to put in the device tree. > >=20 >=20 > It is core_id everywhere else. >=20 > $ git grep core_id hw/ppc/ > hw/ppc/spapr.c: cpu_props->has_core_id =3D true; > hw/ppc/spapr.c: cpu_props->core_id =3D i * smt; > hw/ppc/spapr_cpu_core.c: spapr->cores[cc->core_id / smt] =3D NULL; > hw/ppc/spapr_cpu_core.c: drc =3D spapr_dr_connector_by_id(SPAPR_DR_CON= NECTOR_TYPE_CPU, cc->core_id); > hw/ppc/spapr_cpu_core.c: index =3D cc->core_id / smt; > hw/ppc/spapr_cpu_core.c: if (cc->core_id % smt) { > hw/ppc/spapr_cpu_core.c: error_setg(&local_err, "invalid core id %= d\n", cc->core_id); > hw/ppc/spapr_cpu_core.c: index =3D cc->core_id / smt; > hw/ppc/spapr_cpu_core.c: error_setg(&local_err, "core id %d out of= range", cc->core_id); > hw/ppc/spapr_cpu_core.c: error_setg(&local_err, "core %d already p= opulated", cc->core_id); >=20 > $ git grep core_dt_id hw/ppc/ > hw/ppc/spapr.c: int core_dt_id =3D i * smt; > hw/ppc/spapr.c: SPAPR_DR_CONNECTOR_= TYPE_CPU, core_dt_id); > hw/ppc/spapr.c: object_property_set_int(core, core_dt_id, = CPU_CORE_PROP_CORE_ID, >=20 > I got confused because the current code still puts cpu_dt_id of thread0 i= n the > device tree. And since cpu_dt_id is still being computed on cpu_index, it= is > a different beast (which needs to be killed since it even crashes simple > hotplug/unplug scenarios). >=20 > > > [...] > > > object_property_set_int(core, core_dt_id, CPU_CORE_PR= OP_CORE_ID, > > > &error_fatal); > > >=20 > > > This patch produces stable_cpu_id in the [0...smt * smp_cores) range.= I find it > > > awkward it depends on the host setup. =20 > >=20 > > True. Possibly we should set these as i * (maximum plausible number > > of threads). > >=20 > > The gotcha is that currently we're using the same "dt_id" to control > > KVM's cpu id and that in turn controls the SMT level. That's a poor > > interface on the kernel side (my bad), but we have to live with it > > now. However we could de-couple that KVM id from the core-id. It'd > > no doubt cause some complications with kvm-xics, but we can probably > > handle it. > >=20 > > > I'm suggesting we introduce cc->stable_id to be able to compute a sim= ple > > > stable_cpu_id in the range [0...max_cpus), like x86 and ARM. =20 > >=20 > > I really don't see what properties this is supposed to have that are > > different from the existing core-id. > >=20 >=20 > Simplicity of always having CPU0, 1, 2, 3... max_cpus in QEMU, and try > to hide the "poor interface on the kernel side" from the code that > does not need it... but maybe that is not that important. Merely having contiguous values seems like a very poor trade off for having two different IDs for the same object to get confused between. --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --6K2R/cS9K4qvcBNq Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJXgxE7AAoJEGw4ysog2bOSOJwP/2LZEwO4Ia+nyLt6wMWLgfBW mHvBxjxhxAoBYAJM+yA2AweNa7JlEpEq2XhFZ55o9qQ4ZJC2+zoTMyX88wNvenED cN3rXQm+q9bL2M4j1i5yu72ekU+SX03VPac9sAkYnCYFiq5q5S5BjvgvP0uGI8h9 x8P8FaVxkhsTZp1fZrg1HR9wperFtFHx4uA6ZTaWPT/n9ziwch2BzRloUleVpHqy DQ8ZyIp5J6N2F94rFJrNZFq0t/ymopLn3jOWF7j9t9m7kG/cjpb3RqxurecaOrUH DHm4sUiMwF7N/x0d8oHbZtJJeUvzOSf/ifYWIauonNi+dDcfZN3UmOxm4d5Cp+O3 EkY94AoDSsI/MzdpB1hsrWTTQ2qrCLQTdq45Af0nLxpIBDBdcEXQ7lpYg5YFn/XI 3yFrseoQhepmTPohsSCQmT5S3ndDHkn1oZWzt5mn9T3+5nDt5D6mRTKLG/eo0Tlo LNtC/sB0gOc2kGopYFGtERJPDUUxtZrfEZccLtfUEydGRcHFEZx5PeuGR6myRUnC pgYx3+moA0t2tOkBJDjQ2pXUgDSvlk6AwLpf6ZIXDLHFdNVp0osla5trqyW4noeO w5SFNGleRqeLe3vY+TQf5UxLki5W352dMR7NoFzikg+U12kCzEKPsgldTHJWqZId hhkpPyektdX8+hZ2up6L =hqGG -----END PGP SIGNATURE----- --6K2R/cS9K4qvcBNq--