[RFC PATCH 00/10] ppc/pnv: Better big-core model, lpar-per-core, PC unit

[RFC PATCH 00/10] ppc/pnv: Better big-core model, lpar-per-core, PC unit

[Nicholas Piggin]

Primary motivation for this series is to improve big-core support.
Other things like SPR indirect, timebase state, PC xscom, are required
for minimal big core support.

I'm still not 100% happy with the big-core topology model after this.
Maybe one day we add pnv big core and pnv small core structures. But
nothing is completely clean because big core mode still has certain
small core restrictions. I think for now we take a bit of mostly
abstracted ugliness in TCG code for the benefit of not spreading
hacks through pervasive (xscom) core addressing.

After this series, power9 and power10 get through skiboot/Linux boot
in SMT8 big-core mode. Not all big core registers are modeled
exactly (some are not shared between small core halves), but that
mostly doesn't matter for OPAL and it can be improved later.

Thanks,
Nick

Nicholas Piggin (10):
  ppc/pnv: Add pointer from PnvCPUState to PnvCore
  ppc/pnv: Move timebase state into PnvCore
  target/ppc: Improve SPR indirect registers
  ppc/pnv: specialise init for powernv8/9/10 machines
  ppc/pnv: Extend chip_pir class method to TIR as well
  ppc: Add a core_index to CPUPPCState for SMT vCPUs
  target/ppc: Add helpers to check for SMT sibling threads
  ppc/pnv: Invert the design for big-core machine modelling
  ppc/pnv: Implement POWER10 PC xscom registers for direct controls
  ppc/pnv: Add an LPAR per core machine option

 include/hw/core/cpu.h        |   8 +
 include/hw/ppc/pnv.h         |   6 +
 include/hw/ppc/pnv_chip.h    |   3 +-
 include/hw/ppc/pnv_core.h    |  31 ++++
 target/ppc/cpu.h             |  37 ++---
 hw/ppc/pnv.c                 | 297 ++++++++++++++++++++++++++++-------
 hw/ppc/pnv_chiptod.c         |   6 +-
 hw/ppc/pnv_core.c            | 129 +++++++++++++--
 hw/ppc/spapr_cpu_core.c      |   7 +
 system/cpus.c                |  10 ++
 target/ppc/cpu_init.c        |  26 +--
 target/ppc/excp_helper.c     |  16 +-
 target/ppc/misc_helper.c     |  98 ++++++------
 target/ppc/timebase_helper.c |  82 +++++-----
 14 files changed, 548 insertions(+), 208 deletions(-)

-- 
2.43.0

[RFC PATCH 01/10] ppc/pnv: Add pointer from PnvCPUState to PnvCore

[Nicholas Piggin]

This helps move core state from CPU to core structures.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 include/hw/ppc/pnv_core.h | 1 +
 hw/ppc/pnv_core.c         | 3 +++
 2 files changed, 4 insertions(+)

diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
index c6d62fd145..30c1e5b1a3 100644
--- a/include/hw/ppc/pnv_core.h
+++ b/include/hw/ppc/pnv_core.h
@@ -54,6 +54,7 @@ struct PnvCoreClass {
 #define PNV_CORE_TYPE_NAME(cpu_model) cpu_model PNV_CORE_TYPE_SUFFIX
 
 typedef struct PnvCPUState {
+    PnvCore *core;
     Object *intc;
 } PnvCPUState;
 
diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
index f40ab721d6..7b0ea7812b 100644
--- a/hw/ppc/pnv_core.c
+++ b/hw/ppc/pnv_core.c
@@ -225,6 +225,7 @@ static const MemoryRegionOps pnv_core_power10_xscom_ops = {
 static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
                                  int thread_index)
 {
+    PnvCPUState *pnv_cpu = pnv_cpu_state(cpu);
     CPUPPCState *env = &cpu->env;
     int core_hwid;
     ppc_spr_t *pir = &env->spr_cb[SPR_PIR];
@@ -232,6 +233,8 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
     Error *local_err = NULL;
     PnvChipClass *pcc = PNV_CHIP_GET_CLASS(pc->chip);
 
+    pnv_cpu->core = pc;
+
     if (!qdev_realize(DEVICE(cpu), NULL, errp)) {
         return;
     }
-- 
2.43.0

Re: [RFC PATCH 01/10] ppc/pnv: Add pointer from PnvCPUState to PnvCore

[Cédric Le Goater]

On 5/26/24 14:26, Nicholas Piggin wrote:
> This helps move core state from CPU to core structures.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   include/hw/ppc/pnv_core.h | 1 +
>   hw/ppc/pnv_core.c         | 3 +++
>   2 files changed, 4 insertions(+)
> 
> diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
> index c6d62fd145..30c1e5b1a3 100644
> --- a/include/hw/ppc/pnv_core.h
> +++ b/include/hw/ppc/pnv_core.h
> @@ -54,6 +54,7 @@ struct PnvCoreClass {
>   #define PNV_CORE_TYPE_NAME(cpu_model) cpu_model PNV_CORE_TYPE_SUFFIX
>   
>   typedef struct PnvCPUState {
> +    PnvCore *core;
>       Object *intc;
>   } PnvCPUState;
>   
> diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
> index f40ab721d6..7b0ea7812b 100644
> --- a/hw/ppc/pnv_core.c
> +++ b/hw/ppc/pnv_core.c
> @@ -225,6 +225,7 @@ static const MemoryRegionOps pnv_core_power10_xscom_ops = {
>   static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
>                                    int thread_index)
>   {
> +    PnvCPUState *pnv_cpu = pnv_cpu_state(cpu);
>       CPUPPCState *env = &cpu->env;
>       int core_hwid;
>       ppc_spr_t *pir = &env->spr_cb[SPR_PIR];
> @@ -232,6 +233,8 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
>       Error *local_err = NULL;
>       PnvChipClass *pcc = PNV_CHIP_GET_CLASS(pc->chip);
>   
> +    pnv_cpu->core = pc;

I would do the assignment in pnv_core_realize() after cpu->machine_data is
allocated. it's minor.


Reviewed-by: Cédric Le Goater <clg@redhat.com>

Thanks,

C.


>       if (!qdev_realize(DEVICE(cpu), NULL, errp)) {
>           return;
>       }

Re: [RFC PATCH 01/10] ppc/pnv: Add pointer from PnvCPUState to PnvCore

[Harsh Prateek Bora]

On 5/26/24 17:56, Nicholas Piggin wrote:
> This helps move core state from CPU to core structures.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   include/hw/ppc/pnv_core.h | 1 +
>   hw/ppc/pnv_core.c         | 3 +++
>   2 files changed, 4 insertions(+)
> 
> diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
> index c6d62fd145..30c1e5b1a3 100644
> --- a/include/hw/ppc/pnv_core.h
> +++ b/include/hw/ppc/pnv_core.h
> @@ -54,6 +54,7 @@ struct PnvCoreClass {
>   #define PNV_CORE_TYPE_NAME(cpu_model) cpu_model PNV_CORE_TYPE_SUFFIX
>   
>   typedef struct PnvCPUState {
> +    PnvCore *core;

Naming it *pc might be more intuitive with the most of its usage, 
although I see few usage as "pnv_core" as well.

Reviewed-by: Harsh Prateek Bora <harshpb@linux.ibm.com>

>       Object *intc;
>   } PnvCPUState;
>   
> diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
> index f40ab721d6..7b0ea7812b 100644
> --- a/hw/ppc/pnv_core.c
> +++ b/hw/ppc/pnv_core.c
> @@ -225,6 +225,7 @@ static const MemoryRegionOps pnv_core_power10_xscom_ops = {
>   static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
>                                    int thread_index)
>   {
> +    PnvCPUState *pnv_cpu = pnv_cpu_state(cpu);
>       CPUPPCState *env = &cpu->env;
>       int core_hwid;
>       ppc_spr_t *pir = &env->spr_cb[SPR_PIR];
> @@ -232,6 +233,8 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
>       Error *local_err = NULL;
>       PnvChipClass *pcc = PNV_CHIP_GET_CLASS(pc->chip);
>   
> +    pnv_cpu->core = pc;
> +
>       if (!qdev_realize(DEVICE(cpu), NULL, errp)) {
>           return;
>       }

[RFC PATCH 02/10] ppc/pnv: Move timebase state into PnvCore

[Nicholas Piggin]

The timebase state machine is per per-core state and can be driven
by any thread in the core. It is currently implemented as a hack
where the state is in a CPU structure and only thread 0's state is
accessed by the chiptod, which limits programming the timebase
side of the state machine to thread 0 of a core.

Move the state out into PnvCore and share it among all threads.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 include/hw/ppc/pnv_core.h    | 17 ++++++++++++
 target/ppc/cpu.h             | 20 --------------
 hw/ppc/pnv_chiptod.c         |  6 ++--
 target/ppc/timebase_helper.c | 53 ++++++++++++++++++++----------------
 4 files changed, 49 insertions(+), 47 deletions(-)

diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
index 30c1e5b1a3..f434c71547 100644
--- a/include/hw/ppc/pnv_core.h
+++ b/include/hw/ppc/pnv_core.h
@@ -25,6 +25,20 @@
 #include "hw/ppc/pnv.h"
 #include "qom/object.h"
 
+/* ChipTOD and TimeBase State Machine */
+struct pnv_tod_tbst {
+    int tb_ready_for_tod; /* core TB ready to receive TOD from chiptod */
+    int tod_sent_to_tb;   /* chiptod sent TOD to the core TB */
+
+    /*
+     * "Timers" for async TBST events are simulated by mfTFAC because TFAC
+     * is polled for such events. These are just used to ensure firmware
+     * performs the polling at least a few times.
+     */
+    int tb_state_timer;
+    int tb_sync_pulse_timer;
+};
+
 #define TYPE_PNV_CORE "powernv-cpu-core"
 OBJECT_DECLARE_TYPE(PnvCore, PnvCoreClass,
                     PNV_CORE)
@@ -38,6 +52,9 @@ struct PnvCore {
     uint32_t pir;
     uint32_t hwid;
     uint64_t hrmor;
+
+    struct pnv_tod_tbst pnv_tod_tbst;
+
     PnvChip *chip;
 
     MemoryRegion xscom_regs;
diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 2015e603d4..1e86658da6 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -1196,21 +1196,6 @@ DEXCR_ASPECT(SRAPD, 4)
 DEXCR_ASPECT(NPHIE, 5)
 DEXCR_ASPECT(PHIE, 6)
 
-/*****************************************************************************/
-/* PowerNV ChipTOD and TimeBase State Machine */
-struct pnv_tod_tbst {
-    int tb_ready_for_tod; /* core TB ready to receive TOD from chiptod */
-    int tod_sent_to_tb;   /* chiptod sent TOD to the core TB */
-
-    /*
-     * "Timers" for async TBST events are simulated by mfTFAC because TFAC
-     * is polled for such events. These are just used to ensure firmware
-     * performs the polling at least a few times.
-     */
-    int tb_state_timer;
-    int tb_sync_pulse_timer;
-};
-
 /*****************************************************************************/
 /* The whole PowerPC CPU context */
 
@@ -1292,11 +1277,6 @@ struct CPUArchState {
 #define TLB_NEED_LOCAL_FLUSH   0x1
 #define TLB_NEED_GLOBAL_FLUSH  0x2
 
-#if defined(TARGET_PPC64)
-    /* PowerNV chiptod / timebase facility state. */
-    /* Would be nice to put these into PnvCore */
-    struct pnv_tod_tbst pnv_tod_tbst;
-#endif
 #endif
 
     /* Other registers */
diff --git a/hw/ppc/pnv_chiptod.c b/hw/ppc/pnv_chiptod.c
index 3831a72101..3eaddd66f0 100644
--- a/hw/ppc/pnv_chiptod.c
+++ b/hw/ppc/pnv_chiptod.c
@@ -365,7 +365,7 @@ static void pnv_chiptod_xscom_write(void *opaque, hwaddr addr,
                           " TOD_MOVE_TOD_TO_TB_REG with no slave target\n");
         } else {
             PowerPCCPU *cpu = chiptod->slave_pc_target->threads[0];
-            CPUPPCState *env = &cpu->env;
+            PnvCore *pc = pnv_cpu_state(cpu)->core;
 
             /*
              * Moving TOD to TB will set the TB of all threads in a
@@ -377,8 +377,8 @@ static void pnv_chiptod_xscom_write(void *opaque, hwaddr addr,
              * thread 0.
              */
 
-            if (env->pnv_tod_tbst.tb_ready_for_tod) {
-                env->pnv_tod_tbst.tod_sent_to_tb = 1;
+            if (pc->pnv_tod_tbst.tb_ready_for_tod) {
+                pc->pnv_tod_tbst.tod_sent_to_tb = 1;
             } else {
                 qemu_log_mask(LOG_GUEST_ERROR, "pnv_chiptod: xscom write reg"
                               " TOD_MOVE_TOD_TO_TB_REG with TB not ready to"
diff --git a/target/ppc/timebase_helper.c b/target/ppc/timebase_helper.c
index 39d397416e..788c498d63 100644
--- a/target/ppc/timebase_helper.c
+++ b/target/ppc/timebase_helper.c
@@ -19,6 +19,7 @@
 #include "qemu/osdep.h"
 #include "cpu.h"
 #include "hw/ppc/ppc.h"
+#include "hw/ppc/pnv_core.h"
 #include "exec/helper-proto.h"
 #include "exec/exec-all.h"
 #include "qemu/log.h"
@@ -298,8 +299,17 @@ static void write_tfmr(CPUPPCState *env, target_ulong val)
     }
 }
 
+static struct pnv_tod_tbst *cpu_get_tbst(PowerPCCPU *cpu)
+{
+    PnvCore *pc = pnv_cpu_state(cpu)->core;
+
+    return &pc->pnv_tod_tbst;
+}
+
 static void tb_state_machine_step(CPUPPCState *env)
 {
+    PowerPCCPU *cpu = env_archcpu(env);
+    struct pnv_tod_tbst *pnv_tod_tbst = cpu_get_tbst(cpu);
     uint64_t tfmr = env->spr[SPR_TFMR];
     unsigned int tbst = tfmr_get_tb_state(tfmr);
 
@@ -307,15 +317,15 @@ static void tb_state_machine_step(CPUPPCState *env)
         return;
     }
 
-    if (env->pnv_tod_tbst.tb_sync_pulse_timer) {
-        env->pnv_tod_tbst.tb_sync_pulse_timer--;
+    if (pnv_tod_tbst->tb_sync_pulse_timer) {
+        pnv_tod_tbst->tb_sync_pulse_timer--;
     } else {
         tfmr |= TFMR_TB_SYNC_OCCURED;
         write_tfmr(env, tfmr);
     }
 
-    if (env->pnv_tod_tbst.tb_state_timer) {
-        env->pnv_tod_tbst.tb_state_timer--;
+    if (pnv_tod_tbst->tb_state_timer) {
+        pnv_tod_tbst->tb_state_timer--;
         return;
     }
 
@@ -332,20 +342,20 @@ static void tb_state_machine_step(CPUPPCState *env)
     } else if (tfmr & TFMR_MOVE_CHIP_TOD_TO_TB) {
         if (tbst == TBST_SYNC_WAIT) {
             tfmr = tfmr_new_tb_state(tfmr, TBST_GET_TOD);
-            env->pnv_tod_tbst.tb_state_timer = 3;
+            pnv_tod_tbst->tb_state_timer = 3;
         } else if (tbst == TBST_GET_TOD) {
-            if (env->pnv_tod_tbst.tod_sent_to_tb) {
+            if (pnv_tod_tbst->tod_sent_to_tb) {
                 tfmr = tfmr_new_tb_state(tfmr, TBST_TB_RUNNING);
                 tfmr &= ~TFMR_MOVE_CHIP_TOD_TO_TB;
-                env->pnv_tod_tbst.tb_ready_for_tod = 0;
-                env->pnv_tod_tbst.tod_sent_to_tb = 0;
+                pnv_tod_tbst->tb_ready_for_tod = 0;
+                pnv_tod_tbst->tod_sent_to_tb = 0;
             }
         } else {
             qemu_log_mask(LOG_GUEST_ERROR, "TFMR error: MOVE_CHIP_TOD_TO_TB "
                           "state machine in invalid state 0x%x\n", tbst);
             tfmr = tfmr_new_tb_state(tfmr, TBST_TB_ERROR);
             tfmr |= TFMR_FIRMWARE_CONTROL_ERROR;
-            env->pnv_tod_tbst.tb_ready_for_tod = 0;
+            pnv_tod_tbst->tb_ready_for_tod = 0;
         }
     }
 
@@ -361,6 +371,8 @@ target_ulong helper_load_tfmr(CPUPPCState *env)
 
 void helper_store_tfmr(CPUPPCState *env, target_ulong val)
 {
+    PowerPCCPU *cpu = env_archcpu(env);
+    struct pnv_tod_tbst *pnv_tod_tbst = cpu_get_tbst(cpu);
     uint64_t tfmr = env->spr[SPR_TFMR];
     uint64_t clear_on_write;
     unsigned int tbst = tfmr_get_tb_state(tfmr);
@@ -384,14 +396,7 @@ void helper_store_tfmr(CPUPPCState *env, target_ulong val)
      * after the second mfspr.
      */
     tfmr &= ~TFMR_TB_SYNC_OCCURED;
-    env->pnv_tod_tbst.tb_sync_pulse_timer = 1;
-
-    if (ppc_cpu_tir(env_archcpu(env)) != 0 &&
-        (val & (TFMR_LOAD_TOD_MOD | TFMR_MOVE_CHIP_TOD_TO_TB))) {
-        qemu_log_mask(LOG_UNIMP, "TFMR timebase state machine can only be "
-                                 "driven by thread 0\n");
-        goto out;
-    }
+    pnv_tod_tbst->tb_sync_pulse_timer = 1;
 
     if (((tfmr | val) & (TFMR_LOAD_TOD_MOD | TFMR_MOVE_CHIP_TOD_TO_TB)) ==
                         (TFMR_LOAD_TOD_MOD | TFMR_MOVE_CHIP_TOD_TO_TB)) {
@@ -399,7 +404,7 @@ void helper_store_tfmr(CPUPPCState *env, target_ulong val)
                                        "MOVE_CHIP_TOD_TO_TB both set\n");
         tfmr = tfmr_new_tb_state(tfmr, TBST_TB_ERROR);
         tfmr |= TFMR_FIRMWARE_CONTROL_ERROR;
-        env->pnv_tod_tbst.tb_ready_for_tod = 0;
+        pnv_tod_tbst->tb_ready_for_tod = 0;
         goto out;
     }
 
@@ -413,8 +418,8 @@ void helper_store_tfmr(CPUPPCState *env, target_ulong val)
         tfmr &= ~TFMR_LOAD_TOD_MOD;
         tfmr &= ~TFMR_MOVE_CHIP_TOD_TO_TB;
         tfmr &= ~TFMR_FIRMWARE_CONTROL_ERROR; /* XXX: should this be cleared? */
-        env->pnv_tod_tbst.tb_ready_for_tod = 0;
-        env->pnv_tod_tbst.tod_sent_to_tb = 0;
+        pnv_tod_tbst->tb_ready_for_tod = 0;
+        pnv_tod_tbst->tod_sent_to_tb = 0;
         goto out;
     }
 
@@ -427,19 +432,19 @@ void helper_store_tfmr(CPUPPCState *env, target_ulong val)
 
     if (tfmr & TFMR_LOAD_TOD_MOD) {
         /* Wait for an arbitrary 3 mfspr until the next state transition. */
-        env->pnv_tod_tbst.tb_state_timer = 3;
+        pnv_tod_tbst->tb_state_timer = 3;
     } else if (tfmr & TFMR_MOVE_CHIP_TOD_TO_TB) {
         if (tbst == TBST_NOT_SET) {
             tfmr = tfmr_new_tb_state(tfmr, TBST_SYNC_WAIT);
-            env->pnv_tod_tbst.tb_ready_for_tod = 1;
-            env->pnv_tod_tbst.tb_state_timer = 3; /* arbitrary */
+            pnv_tod_tbst->tb_ready_for_tod = 1;
+            pnv_tod_tbst->tb_state_timer = 3; /* arbitrary */
         } else {
             qemu_log_mask(LOG_GUEST_ERROR, "TFMR error: MOVE_CHIP_TOD_TO_TB "
                                            "not in TB not set state 0x%x\n",
                                            tbst);
             tfmr = tfmr_new_tb_state(tfmr, TBST_TB_ERROR);
             tfmr |= TFMR_FIRMWARE_CONTROL_ERROR;
-            env->pnv_tod_tbst.tb_ready_for_tod = 0;
+            pnv_tod_tbst->tb_ready_for_tod = 0;
         }
     }
 
-- 
2.43.0

Re: [RFC PATCH 02/10] ppc/pnv: Move timebase state into PnvCore

[Harsh Prateek Bora]

On 5/26/24 17:56, Nicholas Piggin wrote:
> The timebase state machine is per per-core state and can be driven
> by any thread in the core. It is currently implemented as a hack
> where the state is in a CPU structure and only thread 0's state is
> accessed by the chiptod, which limits programming the timebase
> side of the state machine to thread 0 of a core.
> 
> Move the state out into PnvCore and share it among all threads.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   include/hw/ppc/pnv_core.h    | 17 ++++++++++++
>   target/ppc/cpu.h             | 20 --------------
>   hw/ppc/pnv_chiptod.c         |  6 ++--
>   target/ppc/timebase_helper.c | 53 ++++++++++++++++++++----------------
>   4 files changed, 49 insertions(+), 47 deletions(-)
> 
> diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
> index 30c1e5b1a3..f434c71547 100644
> --- a/include/hw/ppc/pnv_core.h
> +++ b/include/hw/ppc/pnv_core.h
> @@ -25,6 +25,20 @@
>   #include "hw/ppc/pnv.h"
>   #include "qom/object.h"
>   
> +/* ChipTOD and TimeBase State Machine */
> +struct pnv_tod_tbst {
> +    int tb_ready_for_tod; /* core TB ready to receive TOD from chiptod */
> +    int tod_sent_to_tb;   /* chiptod sent TOD to the core TB */
> +
> +    /*
> +     * "Timers" for async TBST events are simulated by mfTFAC because TFAC
> +     * is polled for such events. These are just used to ensure firmware
> +     * performs the polling at least a few times.
> +     */
> +    int tb_state_timer;
> +    int tb_sync_pulse_timer;
> +};
> +
>   #define TYPE_PNV_CORE "powernv-cpu-core"
>   OBJECT_DECLARE_TYPE(PnvCore, PnvCoreClass,
>                       PNV_CORE)
> @@ -38,6 +52,9 @@ struct PnvCore {
>       uint32_t pir;
>       uint32_t hwid;
>       uint64_t hrmor;
> +
> +    struct pnv_tod_tbst pnv_tod_tbst;
> +

Now that it is part of struct PnvCore itself, we can drop pnv_ prefix
and just call the member variable as tod_tbst ?

>       PnvChip *chip;
>   
>       MemoryRegion xscom_regs;
> diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
> index 2015e603d4..1e86658da6 100644
> --- a/target/ppc/cpu.h
> +++ b/target/ppc/cpu.h
> @@ -1196,21 +1196,6 @@ DEXCR_ASPECT(SRAPD, 4)
>   DEXCR_ASPECT(NPHIE, 5)
>   DEXCR_ASPECT(PHIE, 6)
>   
> -/*****************************************************************************/
> -/* PowerNV ChipTOD and TimeBase State Machine */
> -struct pnv_tod_tbst {
> -    int tb_ready_for_tod; /* core TB ready to receive TOD from chiptod */
> -    int tod_sent_to_tb;   /* chiptod sent TOD to the core TB */
> -
> -    /*
> -     * "Timers" for async TBST events are simulated by mfTFAC because TFAC
> -     * is polled for such events. These are just used to ensure firmware
> -     * performs the polling at least a few times.
> -     */
> -    int tb_state_timer;
> -    int tb_sync_pulse_timer;
> -};
> -
>   /*****************************************************************************/
>   /* The whole PowerPC CPU context */
>   
> @@ -1292,11 +1277,6 @@ struct CPUArchState {
>   #define TLB_NEED_LOCAL_FLUSH   0x1
>   #define TLB_NEED_GLOBAL_FLUSH  0x2
>   
> -#if defined(TARGET_PPC64)
> -    /* PowerNV chiptod / timebase facility state. */
> -    /* Would be nice to put these into PnvCore */
> -    struct pnv_tod_tbst pnv_tod_tbst;
> -#endif
>   #endif
>   
>       /* Other registers */
> diff --git a/hw/ppc/pnv_chiptod.c b/hw/ppc/pnv_chiptod.c
> index 3831a72101..3eaddd66f0 100644
> --- a/hw/ppc/pnv_chiptod.c
> +++ b/hw/ppc/pnv_chiptod.c
> @@ -365,7 +365,7 @@ static void pnv_chiptod_xscom_write(void *opaque, hwaddr addr,
>                             " TOD_MOVE_TOD_TO_TB_REG with no slave target\n");
>           } else {
>               PowerPCCPU *cpu = chiptod->slave_pc_target->threads[0];
> -            CPUPPCState *env = &cpu->env;
> +            PnvCore *pc = pnv_cpu_state(cpu)->core;
>   
>               /*
>                * Moving TOD to TB will set the TB of all threads in a
> @@ -377,8 +377,8 @@ static void pnv_chiptod_xscom_write(void *opaque, hwaddr addr,
>                * thread 0.
>                */
>   
> -            if (env->pnv_tod_tbst.tb_ready_for_tod) {
> -                env->pnv_tod_tbst.tod_sent_to_tb = 1;
> +            if (pc->pnv_tod_tbst.tb_ready_for_tod) {
> +                pc->pnv_tod_tbst.tod_sent_to_tb = 1;
>               } else {
>                   qemu_log_mask(LOG_GUEST_ERROR, "pnv_chiptod: xscom write reg"
>                                 " TOD_MOVE_TOD_TO_TB_REG with TB not ready to"
> diff --git a/target/ppc/timebase_helper.c b/target/ppc/timebase_helper.c
> index 39d397416e..788c498d63 100644
> --- a/target/ppc/timebase_helper.c
> +++ b/target/ppc/timebase_helper.c
> @@ -19,6 +19,7 @@
>   #include "qemu/osdep.h"
>   #include "cpu.h"
>   #include "hw/ppc/ppc.h"
> +#include "hw/ppc/pnv_core.h"
>   #include "exec/helper-proto.h"
>   #include "exec/exec-all.h"
>   #include "qemu/log.h"
> @@ -298,8 +299,17 @@ static void write_tfmr(CPUPPCState *env, target_ulong val)
>       }
>   }
>   
> +static struct pnv_tod_tbst *cpu_get_tbst(PowerPCCPU *cpu)
> +{
> +    PnvCore *pc = pnv_cpu_state(cpu)->core;
> +
> +    return &pc->pnv_tod_tbst;
> +}
> +
>   static void tb_state_machine_step(CPUPPCState *env)
>   {
> +    PowerPCCPU *cpu = env_archcpu(env);
> +    struct pnv_tod_tbst *pnv_tod_tbst = cpu_get_tbst(cpu);

Since cpu is not used anywhere later, we could just do 
cpu_get_tbst(env_archcpu(env)) ?

>       uint64_t tfmr = env->spr[SPR_TFMR];
>       unsigned int tbst = tfmr_get_tb_state(tfmr);
>   
> @@ -307,15 +317,15 @@ static void tb_state_machine_step(CPUPPCState *env)
>           return;
>       }
>   
> -    if (env->pnv_tod_tbst.tb_sync_pulse_timer) {
> -        env->pnv_tod_tbst.tb_sync_pulse_timer--;
> +    if (pnv_tod_tbst->tb_sync_pulse_timer) {
> +        pnv_tod_tbst->tb_sync_pulse_timer--;
>       } else {
>           tfmr |= TFMR_TB_SYNC_OCCURED;
>           write_tfmr(env, tfmr);
>       }
>   
> -    if (env->pnv_tod_tbst.tb_state_timer) {
> -        env->pnv_tod_tbst.tb_state_timer--;
> +    if (pnv_tod_tbst->tb_state_timer) {
> +        pnv_tod_tbst->tb_state_timer--;
>           return;
>       }
>   
> @@ -332,20 +342,20 @@ static void tb_state_machine_step(CPUPPCState *env)
>       } else if (tfmr & TFMR_MOVE_CHIP_TOD_TO_TB) {
>           if (tbst == TBST_SYNC_WAIT) {
>               tfmr = tfmr_new_tb_state(tfmr, TBST_GET_TOD);
> -            env->pnv_tod_tbst.tb_state_timer = 3;
> +            pnv_tod_tbst->tb_state_timer = 3;
>           } else if (tbst == TBST_GET_TOD) {
> -            if (env->pnv_tod_tbst.tod_sent_to_tb) {
> +            if (pnv_tod_tbst->tod_sent_to_tb) {
>                   tfmr = tfmr_new_tb_state(tfmr, TBST_TB_RUNNING);
>                   tfmr &= ~TFMR_MOVE_CHIP_TOD_TO_TB;
> -                env->pnv_tod_tbst.tb_ready_for_tod = 0;
> -                env->pnv_tod_tbst.tod_sent_to_tb = 0;
> +                pnv_tod_tbst->tb_ready_for_tod = 0;
> +                pnv_tod_tbst->tod_sent_to_tb = 0;
>               }
>           } else {
>               qemu_log_mask(LOG_GUEST_ERROR, "TFMR error: MOVE_CHIP_TOD_TO_TB "
>                             "state machine in invalid state 0x%x\n", tbst);
>               tfmr = tfmr_new_tb_state(tfmr, TBST_TB_ERROR);
>               tfmr |= TFMR_FIRMWARE_CONTROL_ERROR;
> -            env->pnv_tod_tbst.tb_ready_for_tod = 0;
> +            pnv_tod_tbst->tb_ready_for_tod = 0;
>           }
>       }
>   
> @@ -361,6 +371,8 @@ target_ulong helper_load_tfmr(CPUPPCState *env)
>   
>   void helper_store_tfmr(CPUPPCState *env, target_ulong val)
>   {
> +    PowerPCCPU *cpu = env_archcpu(env);
> +    struct pnv_tod_tbst *pnv_tod_tbst = cpu_get_tbst(cpu);

... similarly here as well.

With suggested minor improvements,
Reviewed-by: Harsh Prateek Bora <harshpb@linux.ibm.com>

>       uint64_t tfmr = env->spr[SPR_TFMR];
>       uint64_t clear_on_write;
>       unsigned int tbst = tfmr_get_tb_state(tfmr);
> @@ -384,14 +396,7 @@ void helper_store_tfmr(CPUPPCState *env, target_ulong val)
>        * after the second mfspr.
>        */
>       tfmr &= ~TFMR_TB_SYNC_OCCURED;
> -    env->pnv_tod_tbst.tb_sync_pulse_timer = 1;
> -
> -    if (ppc_cpu_tir(env_archcpu(env)) != 0 &&
> -        (val & (TFMR_LOAD_TOD_MOD | TFMR_MOVE_CHIP_TOD_TO_TB))) {
> -        qemu_log_mask(LOG_UNIMP, "TFMR timebase state machine can only be "
> -                                 "driven by thread 0\n");
> -        goto out;
> -    }
> +    pnv_tod_tbst->tb_sync_pulse_timer = 1;
>   
>       if (((tfmr | val) & (TFMR_LOAD_TOD_MOD | TFMR_MOVE_CHIP_TOD_TO_TB)) ==
>                           (TFMR_LOAD_TOD_MOD | TFMR_MOVE_CHIP_TOD_TO_TB)) {
> @@ -399,7 +404,7 @@ void helper_store_tfmr(CPUPPCState *env, target_ulong val)
>                                          "MOVE_CHIP_TOD_TO_TB both set\n");
>           tfmr = tfmr_new_tb_state(tfmr, TBST_TB_ERROR);
>           tfmr |= TFMR_FIRMWARE_CONTROL_ERROR;
> -        env->pnv_tod_tbst.tb_ready_for_tod = 0;
> +        pnv_tod_tbst->tb_ready_for_tod = 0;
>           goto out;
>       }
>   
> @@ -413,8 +418,8 @@ void helper_store_tfmr(CPUPPCState *env, target_ulong val)
>           tfmr &= ~TFMR_LOAD_TOD_MOD;
>           tfmr &= ~TFMR_MOVE_CHIP_TOD_TO_TB;
>           tfmr &= ~TFMR_FIRMWARE_CONTROL_ERROR; /* XXX: should this be cleared? */
> -        env->pnv_tod_tbst.tb_ready_for_tod = 0;
> -        env->pnv_tod_tbst.tod_sent_to_tb = 0;
> +        pnv_tod_tbst->tb_ready_for_tod = 0;
> +        pnv_tod_tbst->tod_sent_to_tb = 0;
>           goto out;
>       }
>   
> @@ -427,19 +432,19 @@ void helper_store_tfmr(CPUPPCState *env, target_ulong val)
>   
>       if (tfmr & TFMR_LOAD_TOD_MOD) {
>           /* Wait for an arbitrary 3 mfspr until the next state transition. */
> -        env->pnv_tod_tbst.tb_state_timer = 3;
> +        pnv_tod_tbst->tb_state_timer = 3;
>       } else if (tfmr & TFMR_MOVE_CHIP_TOD_TO_TB) {
>           if (tbst == TBST_NOT_SET) {
>               tfmr = tfmr_new_tb_state(tfmr, TBST_SYNC_WAIT);
> -            env->pnv_tod_tbst.tb_ready_for_tod = 1;
> -            env->pnv_tod_tbst.tb_state_timer = 3; /* arbitrary */
> +            pnv_tod_tbst->tb_ready_for_tod = 1;
> +            pnv_tod_tbst->tb_state_timer = 3; /* arbitrary */
>           } else {
>               qemu_log_mask(LOG_GUEST_ERROR, "TFMR error: MOVE_CHIP_TOD_TO_TB "
>                                              "not in TB not set state 0x%x\n",
>                                              tbst);
>               tfmr = tfmr_new_tb_state(tfmr, TBST_TB_ERROR);
>               tfmr |= TFMR_FIRMWARE_CONTROL_ERROR;
> -            env->pnv_tod_tbst.tb_ready_for_tod = 0;
> +            pnv_tod_tbst->tb_ready_for_tod = 0;
>           }
>       }
>   

Re: [RFC PATCH 02/10] ppc/pnv: Move timebase state into PnvCore

[Cédric Le Goater]

On 5/28/24 08:28, Harsh Prateek Bora wrote:
> 
> 
> On 5/26/24 17:56, Nicholas Piggin wrote:
>> The timebase state machine is per per-core state and can be driven
>> by any thread in the core. It is currently implemented as a hack
>> where the state is in a CPU structure and only thread 0's state is
>> accessed by the chiptod, which limits programming the timebase
>> side of the state machine to thread 0 of a core.
>>
>> Move the state out into PnvCore and share it among all threads.
>>
>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>> ---
>>   include/hw/ppc/pnv_core.h    | 17 ++++++++++++
>>   target/ppc/cpu.h             | 20 --------------
>>   hw/ppc/pnv_chiptod.c         |  6 ++--
>>   target/ppc/timebase_helper.c | 53 ++++++++++++++++++++----------------
>>   4 files changed, 49 insertions(+), 47 deletions(-)
>>
>> diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
>> index 30c1e5b1a3..f434c71547 100644
>> --- a/include/hw/ppc/pnv_core.h
>> +++ b/include/hw/ppc/pnv_core.h
>> @@ -25,6 +25,20 @@
>>   #include "hw/ppc/pnv.h"
>>   #include "qom/object.h"
>> +/* ChipTOD and TimeBase State Machine */
>> +struct pnv_tod_tbst {
>> +    int tb_ready_for_tod; /* core TB ready to receive TOD from chiptod */
>> +    int tod_sent_to_tb;   /* chiptod sent TOD to the core TB */
>> +
>> +    /*
>> +     * "Timers" for async TBST events are simulated by mfTFAC because TFAC
>> +     * is polled for such events. These are just used to ensure firmware
>> +     * performs the polling at least a few times.
>> +     */
>> +    int tb_state_timer;
>> +    int tb_sync_pulse_timer;
>> +};
>> +
>>   #define TYPE_PNV_CORE "powernv-cpu-core"
>>   OBJECT_DECLARE_TYPE(PnvCore, PnvCoreClass,
>>                       PNV_CORE)
>> @@ -38,6 +52,9 @@ struct PnvCore {
>>       uint32_t pir;
>>       uint32_t hwid;
>>       uint64_t hrmor;
>> +
>> +    struct pnv_tod_tbst pnv_tod_tbst;
>> +
> 
> Now that it is part of struct PnvCore itself, we can drop pnv_ prefix
> and just call the member variable as tod_tbst ?

yes and rename pnv_tod_tbst using CamelCase please.


Thanks,

C.




> 
>>       PnvChip *chip;
>>       MemoryRegion xscom_regs;
>> diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
>> index 2015e603d4..1e86658da6 100644
>> --- a/target/ppc/cpu.h
>> +++ b/target/ppc/cpu.h
>> @@ -1196,21 +1196,6 @@ DEXCR_ASPECT(SRAPD, 4)
>>   DEXCR_ASPECT(NPHIE, 5)
>>   DEXCR_ASPECT(PHIE, 6)
>> -/*****************************************************************************/
>> -/* PowerNV ChipTOD and TimeBase State Machine */
>> -struct pnv_tod_tbst {
>> -    int tb_ready_for_tod; /* core TB ready to receive TOD from chiptod */
>> -    int tod_sent_to_tb;   /* chiptod sent TOD to the core TB */
>> -
>> -    /*
>> -     * "Timers" for async TBST events are simulated by mfTFAC because TFAC
>> -     * is polled for such events. These are just used to ensure firmware
>> -     * performs the polling at least a few times.
>> -     */
>> -    int tb_state_timer;
>> -    int tb_sync_pulse_timer;
>> -};
>> -
>>   /*****************************************************************************/
>>   /* The whole PowerPC CPU context */
>> @@ -1292,11 +1277,6 @@ struct CPUArchState {
>>   #define TLB_NEED_LOCAL_FLUSH   0x1
>>   #define TLB_NEED_GLOBAL_FLUSH  0x2
>> -#if defined(TARGET_PPC64)
>> -    /* PowerNV chiptod / timebase facility state. */
>> -    /* Would be nice to put these into PnvCore */
>> -    struct pnv_tod_tbst pnv_tod_tbst;
>> -#endif
>>   #endif
>>       /* Other registers */
>> diff --git a/hw/ppc/pnv_chiptod.c b/hw/ppc/pnv_chiptod.c
>> index 3831a72101..3eaddd66f0 100644
>> --- a/hw/ppc/pnv_chiptod.c
>> +++ b/hw/ppc/pnv_chiptod.c
>> @@ -365,7 +365,7 @@ static void pnv_chiptod_xscom_write(void *opaque, hwaddr addr,
>>                             " TOD_MOVE_TOD_TO_TB_REG with no slave target\n");
>>           } else {
>>               PowerPCCPU *cpu = chiptod->slave_pc_target->threads[0];
>> -            CPUPPCState *env = &cpu->env;
>> +            PnvCore *pc = pnv_cpu_state(cpu)->core;
>>               /*
>>                * Moving TOD to TB will set the TB of all threads in a
>> @@ -377,8 +377,8 @@ static void pnv_chiptod_xscom_write(void *opaque, hwaddr addr,
>>                * thread 0.
>>                */
>> -            if (env->pnv_tod_tbst.tb_ready_for_tod) {
>> -                env->pnv_tod_tbst.tod_sent_to_tb = 1;
>> +            if (pc->pnv_tod_tbst.tb_ready_for_tod) {
>> +                pc->pnv_tod_tbst.tod_sent_to_tb = 1;
>>               } else {
>>                   qemu_log_mask(LOG_GUEST_ERROR, "pnv_chiptod: xscom write reg"
>>                                 " TOD_MOVE_TOD_TO_TB_REG with TB not ready to"
>> diff --git a/target/ppc/timebase_helper.c b/target/ppc/timebase_helper.c
>> index 39d397416e..788c498d63 100644
>> --- a/target/ppc/timebase_helper.c
>> +++ b/target/ppc/timebase_helper.c
>> @@ -19,6 +19,7 @@
>>   #include "qemu/osdep.h"
>>   #include "cpu.h"
>>   #include "hw/ppc/ppc.h"
>> +#include "hw/ppc/pnv_core.h"
>>   #include "exec/helper-proto.h"
>>   #include "exec/exec-all.h"
>>   #include "qemu/log.h"
>> @@ -298,8 +299,17 @@ static void write_tfmr(CPUPPCState *env, target_ulong val)
>>       }
>>   }
>> +static struct pnv_tod_tbst *cpu_get_tbst(PowerPCCPU *cpu)
>> +{
>> +    PnvCore *pc = pnv_cpu_state(cpu)->core;
>> +
>> +    return &pc->pnv_tod_tbst;
>> +}
>> +
>>   static void tb_state_machine_step(CPUPPCState *env)
>>   {
>> +    PowerPCCPU *cpu = env_archcpu(env);
>> +    struct pnv_tod_tbst *pnv_tod_tbst = cpu_get_tbst(cpu);
> 
> Since cpu is not used anywhere later, we could just do cpu_get_tbst(env_archcpu(env)) ?
> 
>>       uint64_t tfmr = env->spr[SPR_TFMR];
>>       unsigned int tbst = tfmr_get_tb_state(tfmr);
>> @@ -307,15 +317,15 @@ static void tb_state_machine_step(CPUPPCState *env)
>>           return;
>>       }
>> -    if (env->pnv_tod_tbst.tb_sync_pulse_timer) {
>> -        env->pnv_tod_tbst.tb_sync_pulse_timer--;
>> +    if (pnv_tod_tbst->tb_sync_pulse_timer) {
>> +        pnv_tod_tbst->tb_sync_pulse_timer--;
>>       } else {
>>           tfmr |= TFMR_TB_SYNC_OCCURED;
>>           write_tfmr(env, tfmr);
>>       }
>> -    if (env->pnv_tod_tbst.tb_state_timer) {
>> -        env->pnv_tod_tbst.tb_state_timer--;
>> +    if (pnv_tod_tbst->tb_state_timer) {
>> +        pnv_tod_tbst->tb_state_timer--;
>>           return;
>>       }
>> @@ -332,20 +342,20 @@ static void tb_state_machine_step(CPUPPCState *env)
>>       } else if (tfmr & TFMR_MOVE_CHIP_TOD_TO_TB) {
>>           if (tbst == TBST_SYNC_WAIT) {
>>               tfmr = tfmr_new_tb_state(tfmr, TBST_GET_TOD);
>> -            env->pnv_tod_tbst.tb_state_timer = 3;
>> +            pnv_tod_tbst->tb_state_timer = 3;
>>           } else if (tbst == TBST_GET_TOD) {
>> -            if (env->pnv_tod_tbst.tod_sent_to_tb) {
>> +            if (pnv_tod_tbst->tod_sent_to_tb) {
>>                   tfmr = tfmr_new_tb_state(tfmr, TBST_TB_RUNNING);
>>                   tfmr &= ~TFMR_MOVE_CHIP_TOD_TO_TB;
>> -                env->pnv_tod_tbst.tb_ready_for_tod = 0;
>> -                env->pnv_tod_tbst.tod_sent_to_tb = 0;
>> +                pnv_tod_tbst->tb_ready_for_tod = 0;
>> +                pnv_tod_tbst->tod_sent_to_tb = 0;
>>               }
>>           } else {
>>               qemu_log_mask(LOG_GUEST_ERROR, "TFMR error: MOVE_CHIP_TOD_TO_TB "
>>                             "state machine in invalid state 0x%x\n", tbst);
>>               tfmr = tfmr_new_tb_state(tfmr, TBST_TB_ERROR);
>>               tfmr |= TFMR_FIRMWARE_CONTROL_ERROR;
>> -            env->pnv_tod_tbst.tb_ready_for_tod = 0;
>> +            pnv_tod_tbst->tb_ready_for_tod = 0;
>>           }
>>       }
>> @@ -361,6 +371,8 @@ target_ulong helper_load_tfmr(CPUPPCState *env)
>>   void helper_store_tfmr(CPUPPCState *env, target_ulong val)
>>   {
>> +    PowerPCCPU *cpu = env_archcpu(env);
>> +    struct pnv_tod_tbst *pnv_tod_tbst = cpu_get_tbst(cpu);
> 
> ... similarly here as well.
> 
> With suggested minor improvements,
> Reviewed-by: Harsh Prateek Bora <harshpb@linux.ibm.com>
> 
>>       uint64_t tfmr = env->spr[SPR_TFMR];
>>       uint64_t clear_on_write;
>>       unsigned int tbst = tfmr_get_tb_state(tfmr);
>> @@ -384,14 +396,7 @@ void helper_store_tfmr(CPUPPCState *env, target_ulong val)
>>        * after the second mfspr.
>>        */
>>       tfmr &= ~TFMR_TB_SYNC_OCCURED;
>> -    env->pnv_tod_tbst.tb_sync_pulse_timer = 1;
>> -
>> -    if (ppc_cpu_tir(env_archcpu(env)) != 0 &&
>> -        (val & (TFMR_LOAD_TOD_MOD | TFMR_MOVE_CHIP_TOD_TO_TB))) {
>> -        qemu_log_mask(LOG_UNIMP, "TFMR timebase state machine can only be "
>> -                                 "driven by thread 0\n");
>> -        goto out;
>> -    }
>> +    pnv_tod_tbst->tb_sync_pulse_timer = 1;
>>       if (((tfmr | val) & (TFMR_LOAD_TOD_MOD | TFMR_MOVE_CHIP_TOD_TO_TB)) ==
>>                           (TFMR_LOAD_TOD_MOD | TFMR_MOVE_CHIP_TOD_TO_TB)) {
>> @@ -399,7 +404,7 @@ void helper_store_tfmr(CPUPPCState *env, target_ulong val)
>>                                          "MOVE_CHIP_TOD_TO_TB both set\n");
>>           tfmr = tfmr_new_tb_state(tfmr, TBST_TB_ERROR);
>>           tfmr |= TFMR_FIRMWARE_CONTROL_ERROR;
>> -        env->pnv_tod_tbst.tb_ready_for_tod = 0;
>> +        pnv_tod_tbst->tb_ready_for_tod = 0;
>>           goto out;
>>       }
>> @@ -413,8 +418,8 @@ void helper_store_tfmr(CPUPPCState *env, target_ulong val)
>>           tfmr &= ~TFMR_LOAD_TOD_MOD;
>>           tfmr &= ~TFMR_MOVE_CHIP_TOD_TO_TB;
>>           tfmr &= ~TFMR_FIRMWARE_CONTROL_ERROR; /* XXX: should this be cleared? */
>> -        env->pnv_tod_tbst.tb_ready_for_tod = 0;
>> -        env->pnv_tod_tbst.tod_sent_to_tb = 0;
>> +        pnv_tod_tbst->tb_ready_for_tod = 0;
>> +        pnv_tod_tbst->tod_sent_to_tb = 0;
>>           goto out;
>>       }
>> @@ -427,19 +432,19 @@ void helper_store_tfmr(CPUPPCState *env, target_ulong val)
>>       if (tfmr & TFMR_LOAD_TOD_MOD) {
>>           /* Wait for an arbitrary 3 mfspr until the next state transition. */
>> -        env->pnv_tod_tbst.tb_state_timer = 3;
>> +        pnv_tod_tbst->tb_state_timer = 3;
>>       } else if (tfmr & TFMR_MOVE_CHIP_TOD_TO_TB) {
>>           if (tbst == TBST_NOT_SET) {
>>               tfmr = tfmr_new_tb_state(tfmr, TBST_SYNC_WAIT);
>> -            env->pnv_tod_tbst.tb_ready_for_tod = 1;
>> -            env->pnv_tod_tbst.tb_state_timer = 3; /* arbitrary */
>> +            pnv_tod_tbst->tb_ready_for_tod = 1;
>> +            pnv_tod_tbst->tb_state_timer = 3; /* arbitrary */
>>           } else {
>>               qemu_log_mask(LOG_GUEST_ERROR, "TFMR error: MOVE_CHIP_TOD_TO_TB "
>>                                              "not in TB not set state 0x%x\n",
>>                                              tbst);
>>               tfmr = tfmr_new_tb_state(tfmr, TBST_TB_ERROR);
>>               tfmr |= TFMR_FIRMWARE_CONTROL_ERROR;
>> -            env->pnv_tod_tbst.tb_ready_for_tod = 0;
>> +            pnv_tod_tbst->tb_ready_for_tod = 0;
>>           }
>>       }

Re: [RFC PATCH 02/10] ppc/pnv: Move timebase state into PnvCore

[Nicholas Piggin]

On Tue May 28, 2024 at 5:52 PM AEST, Cédric Le Goater wrote:
> On 5/28/24 08:28, Harsh Prateek Bora wrote:
> > 
> > 
> > On 5/26/24 17:56, Nicholas Piggin wrote:
> >> The timebase state machine is per per-core state and can be driven
> >> by any thread in the core. It is currently implemented as a hack
> >> where the state is in a CPU structure and only thread 0's state is
> >> accessed by the chiptod, which limits programming the timebase
> >> side of the state machine to thread 0 of a core.
> >>
> >> Move the state out into PnvCore and share it among all threads.
> >>
> >> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> >> ---
> >>   include/hw/ppc/pnv_core.h    | 17 ++++++++++++
> >>   target/ppc/cpu.h             | 20 --------------
> >>   hw/ppc/pnv_chiptod.c         |  6 ++--
> >>   target/ppc/timebase_helper.c | 53 ++++++++++++++++++++----------------
> >>   4 files changed, 49 insertions(+), 47 deletions(-)
> >>
> >> diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
> >> index 30c1e5b1a3..f434c71547 100644
> >> --- a/include/hw/ppc/pnv_core.h
> >> +++ b/include/hw/ppc/pnv_core.h
> >> @@ -25,6 +25,20 @@
> >>   #include "hw/ppc/pnv.h"
> >>   #include "qom/object.h"
> >> +/* ChipTOD and TimeBase State Machine */
> >> +struct pnv_tod_tbst {
> >> +    int tb_ready_for_tod; /* core TB ready to receive TOD from chiptod */
> >> +    int tod_sent_to_tb;   /* chiptod sent TOD to the core TB */
> >> +
> >> +    /*
> >> +     * "Timers" for async TBST events are simulated by mfTFAC because TFAC
> >> +     * is polled for such events. These are just used to ensure firmware
> >> +     * performs the polling at least a few times.
> >> +     */
> >> +    int tb_state_timer;
> >> +    int tb_sync_pulse_timer;
> >> +};
> >> +
> >>   #define TYPE_PNV_CORE "powernv-cpu-core"
> >>   OBJECT_DECLARE_TYPE(PnvCore, PnvCoreClass,
> >>                       PNV_CORE)
> >> @@ -38,6 +52,9 @@ struct PnvCore {
> >>       uint32_t pir;
> >>       uint32_t hwid;
> >>       uint64_t hrmor;
> >> +
> >> +    struct pnv_tod_tbst pnv_tod_tbst;
> >> +
> > 
> > Now that it is part of struct PnvCore itself, we can drop pnv_ prefix
> > and just call the member variable as tod_tbst ?
>
> yes and rename pnv_tod_tbst using CamelCase please.

Okay will do. That'll look nicer.

Thanks,
Nick

[RFC PATCH 03/10] target/ppc: Improve SPR indirect registers

[Nicholas Piggin]

SPRC/SPRD were recently added to all BookS CPUs supported, but
they are only tested on POWER9 and POWER10, so restrict them to
those CPUs.

SPR indirect scratch registers presently replicated per-CPU like
SMT SPRs, but the PnvCore is a better place for them since they
are restricted to P9/P10.

Also add SPR indirect read access to core thread state for POWER9
since skiboot accesses that when booting to check for big-core
mode.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 include/hw/ppc/pnv_core.h |  1 +
 target/ppc/cpu.h          |  3 --
 target/ppc/cpu_init.c     | 21 ++++++------
 target/ppc/misc_helper.c  | 67 ++++++++++++++++++++-------------------
 4 files changed, 46 insertions(+), 46 deletions(-)

diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
index f434c71547..21297262c1 100644
--- a/include/hw/ppc/pnv_core.h
+++ b/include/hw/ppc/pnv_core.h
@@ -53,6 +53,7 @@ struct PnvCore {
     uint32_t hwid;
     uint64_t hrmor;
 
+    target_ulong scratch[8]; /* SCRATCH registers */
     struct pnv_tod_tbst pnv_tod_tbst;
 
     PnvChip *chip;
diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 1e86658da6..dac13d4dac 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -1253,9 +1253,6 @@ struct CPUArchState {
     ppc_slb_t slb[MAX_SLB_ENTRIES]; /* PowerPC 64 SLB area */
     struct CPUBreakpoint *ciabr_breakpoint;
     struct CPUWatchpoint *dawr0_watchpoint;
-
-    /* POWER CPU regs/state */
-    target_ulong scratch[8]; /* SCRATCH registers (shared across core) */
 #endif
     target_ulong sr[32];   /* segment registers */
     uint32_t nb_BATs;      /* number of BATs */
diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
index 01e358a4a5..ae483e20c4 100644
--- a/target/ppc/cpu_init.c
+++ b/target/ppc/cpu_init.c
@@ -5759,16 +5759,6 @@ static void register_power_common_book4_sprs(CPUPPCState *env)
                  SPR_NOACCESS, SPR_NOACCESS,
                  &spr_read_generic, &spr_core_write_generic,
                  0x00000000);
-    spr_register_hv(env, SPR_POWER_SPRC, "SPRC",
-                 SPR_NOACCESS, SPR_NOACCESS,
-                 SPR_NOACCESS, SPR_NOACCESS,
-                 &spr_read_generic, &spr_write_sprc,
-                 0x00000000);
-    spr_register_hv(env, SPR_POWER_SPRD, "SPRD",
-                 SPR_NOACCESS, SPR_NOACCESS,
-                 SPR_NOACCESS, SPR_NOACCESS,
-                 &spr_read_sprd, &spr_write_sprd,
-                 0x00000000);
 #endif
 }
 
@@ -5781,6 +5771,17 @@ static void register_power9_book4_sprs(CPUPPCState *env)
                      SPR_NOACCESS, SPR_NOACCESS,
                      &spr_read_generic, &spr_write_generic,
                      KVM_REG_PPC_WORT, 0);
+    /* SPRC/SPRD exist in earlier CPUs but only tested on POWER9/10 */
+    spr_register_hv(env, SPR_POWER_SPRC, "SPRC",
+                 SPR_NOACCESS, SPR_NOACCESS,
+                 SPR_NOACCESS, SPR_NOACCESS,
+                 &spr_read_generic, &spr_write_sprc,
+                 0x00000000);
+    spr_register_hv(env, SPR_POWER_SPRD, "SPRD",
+                 SPR_NOACCESS, SPR_NOACCESS,
+                 SPR_NOACCESS, SPR_NOACCESS,
+                 &spr_read_sprd, &spr_write_sprd,
+                 0x00000000);
 #endif
 }
 
diff --git a/target/ppc/misc_helper.c b/target/ppc/misc_helper.c
index fa47be2298..46ba3a5584 100644
--- a/target/ppc/misc_helper.c
+++ b/target/ppc/misc_helper.c
@@ -26,6 +26,7 @@
 #include "qemu/main-loop.h"
 #include "mmu-book3s-v3.h"
 #include "hw/ppc/ppc.h"
+#include "hw/ppc/pnv_core.h"
 
 #include "helper_regs.h"
 
@@ -321,11 +322,25 @@ void helper_store_sprc(CPUPPCState *env, target_ulong val)
 
 target_ulong helper_load_sprd(CPUPPCState *env)
 {
+    PowerPCCPU *cpu = env_archcpu(env);
+    PnvCore *pc = pnv_cpu_state(cpu)->core;
     target_ulong sprc = env->spr[SPR_POWER_SPRC];
 
-    switch (sprc & 0x3c0) {
-    case 0: /* SCRATCH0-7 */
-        return env->scratch[(sprc >> 3) & 0x7];
+    switch (sprc & 0x3e0) {
+    case 0: /* SCRATCH0-3 */
+    case 1: /* SCRATCH4-7 */
+        return pc->scratch[(sprc >> 3) & 0x7];
+    case 0x1e0: /* core thread state */
+        if (env->excp_model == POWERPC_EXCP_POWER9) {
+            /*
+             * Only implement for POWER9 because skiboot uses it to check
+             * big-core mode. Other bits are unimplemented so we would
+             * prefer to get unimplemented message on POWER10 if it were
+             * used.
+             */
+            return 0;
+        }
+        /* fallthru */
     default:
         qemu_log_mask(LOG_UNIMP, "mfSPRD: Unimplemented SPRC:0x"
                                   TARGET_FMT_lx"\n", sprc);
@@ -334,41 +349,27 @@ target_ulong helper_load_sprd(CPUPPCState *env)
     return 0;
 }
 
-static void do_store_scratch(CPUPPCState *env, int nr, target_ulong val)
-{
-    CPUState *cs = env_cpu(env);
-    CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
-
-    /*
-     * Log stores to SCRATCH, because some firmware uses these for debugging
-     * and logging, but they would normally be read by the BMC, which is
-     * not implemented in QEMU yet. This gives a way to get at the information.
-     * Could also dump these upon checkstop.
-     */
-    qemu_log("SPRD write 0x" TARGET_FMT_lx " to SCRATCH%d\n", val, nr);
-
-    if (nr_threads == 1) {
-        env->scratch[nr] = val;
-        return;
-    }
-
-    THREAD_SIBLING_FOREACH(cs, ccs) {
-        CPUPPCState *cenv = &POWERPC_CPU(ccs)->env;
-        cenv->scratch[nr] = val;
-    }
-}
-
 void helper_store_sprd(CPUPPCState *env, target_ulong val)
 {
     target_ulong sprc = env->spr[SPR_POWER_SPRC];
-
-    switch (sprc & 0x3c0) {
-    case 0: /* SCRATCH0-7 */
-        do_store_scratch(env, (sprc >> 3) & 0x7, val);
+    PowerPCCPU *cpu = env_archcpu(env);
+    PnvCore *pc = pnv_cpu_state(cpu)->core;
+
+    switch (sprc & 0x3e0) {
+    case 0: /* SCRATCH0-3 */
+    case 1: /* SCRATCH4-7 */
+        /*
+         * Log stores to SCRATCH, because some firmware uses these for
+         * debugging and logging, but they would normally be read by the BMC,
+	 * which is not implemented in QEMU yet. This gives a way to get at the
+	 * information. Could also dump these upon checkstop.
+         */
+        int nr = (sprc >> 3) & 0x7;
+        qemu_log("SPRD write 0x" TARGET_FMT_lx " to SCRATCH%d\n", val, nr);
+        pc->scratch[nr] = val;
         break;
     default:
-        qemu_log_mask(LOG_UNIMP, "mfSPRD: Unimplemented SPRC:0x"
+        qemu_log_mask(LOG_UNIMP, "mtSPRD: Unimplemented SPRC:0x"
                                   TARGET_FMT_lx"\n", sprc);
         break;
     }
-- 
2.43.0

Re: [RFC PATCH 03/10] target/ppc: Improve SPR indirect registers

[Harsh Prateek Bora]

Hi Nick,

On 5/26/24 17:56, Nicholas Piggin wrote:
> SPRC/SPRD were recently added to all BookS CPUs supported, but
> they are only tested on POWER9 and POWER10, so restrict them to
> those CPUs.
> 

Hope you mean to restrict to P9/10 for both spapr and pnv or just pnv ?

> SPR indirect scratch registers presently replicated per-CPU like
> SMT SPRs, but the PnvCore is a better place for them since they
> are restricted to P9/P10.
> 
> Also add SPR indirect read access to core thread state for POWER9
> since skiboot accesses that when booting to check for big-core
> mode.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   include/hw/ppc/pnv_core.h |  1 +
>   target/ppc/cpu.h          |  3 --
>   target/ppc/cpu_init.c     | 21 ++++++------
>   target/ppc/misc_helper.c  | 67 ++++++++++++++++++++-------------------
>   4 files changed, 46 insertions(+), 46 deletions(-)
> 
> diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
> index f434c71547..21297262c1 100644
> --- a/include/hw/ppc/pnv_core.h
> +++ b/include/hw/ppc/pnv_core.h
> @@ -53,6 +53,7 @@ struct PnvCore {
>       uint32_t hwid;
>       uint64_t hrmor;
>   
> +    target_ulong scratch[8]; /* SCRATCH registers */
>       struct pnv_tod_tbst pnv_tod_tbst;
>   
>       PnvChip *chip;
> diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
> index 1e86658da6..dac13d4dac 100644
> --- a/target/ppc/cpu.h
> +++ b/target/ppc/cpu.h
> @@ -1253,9 +1253,6 @@ struct CPUArchState {
>       ppc_slb_t slb[MAX_SLB_ENTRIES]; /* PowerPC 64 SLB area */
>       struct CPUBreakpoint *ciabr_breakpoint;
>       struct CPUWatchpoint *dawr0_watchpoint;
> -
> -    /* POWER CPU regs/state */
> -    target_ulong scratch[8]; /* SCRATCH registers (shared across core) */
>   #endif
>       target_ulong sr[32];   /* segment registers */
>       uint32_t nb_BATs;      /* number of BATs */
> diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
> index 01e358a4a5..ae483e20c4 100644
> --- a/target/ppc/cpu_init.c
> +++ b/target/ppc/cpu_init.c
> @@ -5759,16 +5759,6 @@ static void register_power_common_book4_sprs(CPUPPCState *env)
>                    SPR_NOACCESS, SPR_NOACCESS,
>                    &spr_read_generic, &spr_core_write_generic,
>                    0x00000000);
> -    spr_register_hv(env, SPR_POWER_SPRC, "SPRC",
> -                 SPR_NOACCESS, SPR_NOACCESS,
> -                 SPR_NOACCESS, SPR_NOACCESS,
> -                 &spr_read_generic, &spr_write_sprc,
> -                 0x00000000);
> -    spr_register_hv(env, SPR_POWER_SPRD, "SPRD",
> -                 SPR_NOACCESS, SPR_NOACCESS,
> -                 SPR_NOACCESS, SPR_NOACCESS,
> -                 &spr_read_sprd, &spr_write_sprd,
> -                 0x00000000);
>   #endif
>   }
>   
> @@ -5781,6 +5771,17 @@ static void register_power9_book4_sprs(CPUPPCState *env)
>                        SPR_NOACCESS, SPR_NOACCESS,
>                        &spr_read_generic, &spr_write_generic,
>                        KVM_REG_PPC_WORT, 0);
> +    /* SPRC/SPRD exist in earlier CPUs but only tested on POWER9/10 */
> +    spr_register_hv(env, SPR_POWER_SPRC, "SPRC",
> +                 SPR_NOACCESS, SPR_NOACCESS,
> +                 SPR_NOACCESS, SPR_NOACCESS,
> +                 &spr_read_generic, &spr_write_sprc,
> +                 0x00000000);
> +    spr_register_hv(env, SPR_POWER_SPRD, "SPRD",
> +                 SPR_NOACCESS, SPR_NOACCESS,
> +                 SPR_NOACCESS, SPR_NOACCESS,
> +                 &spr_read_sprd, &spr_write_sprd,
> +                 0x00000000);
>   #endif
>   }
>   
> diff --git a/target/ppc/misc_helper.c b/target/ppc/misc_helper.c
> index fa47be2298..46ba3a5584 100644
> --- a/target/ppc/misc_helper.c
> +++ b/target/ppc/misc_helper.c
> @@ -26,6 +26,7 @@
>   #include "qemu/main-loop.h"
>   #include "mmu-book3s-v3.h"
>   #include "hw/ppc/ppc.h"
> +#include "hw/ppc/pnv_core.h"
>   
>   #include "helper_regs.h"
>   
> @@ -321,11 +322,25 @@ void helper_store_sprc(CPUPPCState *env, target_ulong val)
>   
>   target_ulong helper_load_sprd(CPUPPCState *env)
>   {
> +    PowerPCCPU *cpu = env_archcpu(env);
> +    PnvCore *pc = pnv_cpu_state(cpu)->core;

We may want to avoid creating local variable cpu here also like previous 
patches.

However, is this helper meant to be accessible for spapr as well ?

>       target_ulong sprc = env->spr[SPR_POWER_SPRC];
>   
> -    switch (sprc & 0x3c0) {
> -    case 0: /* SCRATCH0-7 */
> -        return env->scratch[(sprc >> 3) & 0x7];
> +    switch (sprc & 0x3e0) {
> +    case 0: /* SCRATCH0-3 */
> +    case 1: /* SCRATCH4-7 */
> +        return pc->scratch[(sprc >> 3) & 0x7];

If so, will pc be uninitialized in case of spapr ?

> +    case 0x1e0: /* core thread state */
> +        if (env->excp_model == POWERPC_EXCP_POWER9) {
> +            /*
> +             * Only implement for POWER9 because skiboot uses it to check
> +             * big-core mode. Other bits are unimplemented so we would
> +             * prefer to get unimplemented message on POWER10 if it were
> +             * used.
> +             */
> +            return 0;
> +        }
> +        /* fallthru */
>       default:
>           qemu_log_mask(LOG_UNIMP, "mfSPRD: Unimplemented SPRC:0x"
>                                     TARGET_FMT_lx"\n", sprc);
> @@ -334,41 +349,27 @@ target_ulong helper_load_sprd(CPUPPCState *env)
>       return 0;
>   }
>   
> -static void do_store_scratch(CPUPPCState *env, int nr, target_ulong val)
> -{
> -    CPUState *cs = env_cpu(env);
> -    CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
> -
> -    /*
> -     * Log stores to SCRATCH, because some firmware uses these for debugging
> -     * and logging, but they would normally be read by the BMC, which is
> -     * not implemented in QEMU yet. This gives a way to get at the information.
> -     * Could also dump these upon checkstop.
> -     */
> -    qemu_log("SPRD write 0x" TARGET_FMT_lx " to SCRATCH%d\n", val, nr);
> -
> -    if (nr_threads == 1) {
> -        env->scratch[nr] = val;
> -        return;
> -    }
> -
> -    THREAD_SIBLING_FOREACH(cs, ccs) {
> -        CPUPPCState *cenv = &POWERPC_CPU(ccs)->env;
> -        cenv->scratch[nr] = val;
> -    }
> -}
> -
>   void helper_store_sprd(CPUPPCState *env, target_ulong val)
>   {
>       target_ulong sprc = env->spr[SPR_POWER_SPRC];
> -
> -    switch (sprc & 0x3c0) {
> -    case 0: /* SCRATCH0-7 */
> -        do_store_scratch(env, (sprc >> 3) & 0x7, val);
> +    PowerPCCPU *cpu = env_archcpu(env);
> +    PnvCore *pc = pnv_cpu_state(cpu)->core;

Ditto?

> +
> +    switch (sprc & 0x3e0) {
> +    case 0: /* SCRATCH0-3 */
> +    case 1: /* SCRATCH4-7 */
> +        /*
> +         * Log stores to SCRATCH, because some firmware uses these for
> +         * debugging and logging, but they would normally be read by the BMC,
> +	 * which is not implemented in QEMU yet. This gives a way to get at the
> +	 * information. Could also dump these upon checkstop.
> +         */
> +        int nr = (sprc >> 3) & 0x7;
> +        qemu_log("SPRD write 0x" TARGET_FMT_lx " to SCRATCH%d\n", val, nr);
> +        pc->scratch[nr] = val;

regards,
Harsh
>           break;
>       default:
> -        qemu_log_mask(LOG_UNIMP, "mfSPRD: Unimplemented SPRC:0x"
> +        qemu_log_mask(LOG_UNIMP, "mtSPRD: Unimplemented SPRC:0x"
>                                     TARGET_FMT_lx"\n", sprc);
>           break;
>       }

Re: [RFC PATCH 03/10] target/ppc: Improve SPR indirect registers

[Nicholas Piggin]

On Tue May 28, 2024 at 4:50 PM AEST, Harsh Prateek Bora wrote:
>
> Hi Nick,
>
> On 5/26/24 17:56, Nicholas Piggin wrote:
> > SPRC/SPRD were recently added to all BookS CPUs supported, but
> > they are only tested on POWER9 and POWER10, so restrict them to
> > those CPUs.
> > 
>
> Hope you mean to restrict to P9/10 for both spapr and pnv or just pnv ?

For pnv, but they are hypervisor registers so they can not be
accessed with spapr.

[...]

> > @@ -321,11 +322,25 @@ void helper_store_sprc(CPUPPCState *env, target_ulong val)
> >   
> >   target_ulong helper_load_sprd(CPUPPCState *env)
> >   {
> > +    PowerPCCPU *cpu = env_archcpu(env);
> > +    PnvCore *pc = pnv_cpu_state(cpu)->core;
>
> We may want to avoid creating local variable cpu here also like previous 
> patches.

Since we have a maze of pointers and types, sometimes I like to
write the types down, but maybe that's just me :P

> However, is this helper meant to be accessible for spapr as well ?

Right, it's not. I *think* it should be okay to do this since it
should never be reached by spapr.

Thanks,
Nick

[RFC PATCH 04/10] ppc/pnv: specialise init for powernv8/9/10 machines

[Nicholas Piggin]

This will allow different settings and checks for different
machine types with later changes.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 hw/ppc/pnv.c | 35 ++++++++++++++++++++++++++++++-----
 1 file changed, 30 insertions(+), 5 deletions(-)

diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
index 6e3a5ccdec..a706de2e36 100644
--- a/hw/ppc/pnv.c
+++ b/hw/ppc/pnv.c
@@ -976,11 +976,6 @@ static void pnv_init(MachineState *machine)
     pnv->num_chips =
         machine->smp.max_cpus / (machine->smp.cores * machine->smp.threads);
 
-    if (machine->smp.threads > 8) {
-        error_report("Cannot support more than 8 threads/core "
-                     "on a powernv machine");
-        exit(1);
-    }
     if (!is_power_of_2(machine->smp.threads)) {
         error_report("Cannot support %d threads/core on a powernv"
                      "machine because it must be a power of 2",
@@ -1076,6 +1071,33 @@ static void pnv_init(MachineState *machine)
     }
 }
 
+static void pnv_power8_init(MachineState *machine)
+{
+    if (machine->smp.threads > 8) {
+        error_report("Cannot support more than 8 threads/core "
+                     "on a powernv POWER8 machine");
+        exit(1);
+    }
+
+    pnv_init(machine);
+}
+
+static void pnv_power9_init(MachineState *machine)
+{
+    if (machine->smp.threads > 8) {
+        error_report("Cannot support more than 8 threads/core "
+                     "on a powernv9/10 machine");
+        exit(1);
+    }
+
+    pnv_init(machine);
+}
+
+static void pnv_power10_init(MachineState *machine)
+{
+    pnv_power9_init(machine);
+}
+
 /*
  *    0:21  Reserved - Read as zeros
  *   22:24  Chip ID
@@ -2423,6 +2445,7 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
     };
 
     mc->desc = "IBM PowerNV (Non-Virtualized) POWER8";
+    mc->init = pnv_power8_init;
     mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power8_v2.0");
     compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
 
@@ -2449,6 +2472,7 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
     };
 
     mc->desc = "IBM PowerNV (Non-Virtualized) POWER9";
+    mc->init = pnv_power9_init;
     mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power9_v2.2");
     compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
 
@@ -2473,6 +2497,7 @@ static void pnv_machine_p10_common_class_init(ObjectClass *oc, void *data)
         { TYPE_PNV_PHB_ROOT_PORT, "version", "5" },
     };
 
+    mc->init = pnv_power10_init;
     mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power10_v2.0");
     compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
 
-- 
2.43.0

Re: [RFC PATCH 04/10] ppc/pnv: specialise init for powernv8/9/10 machines

[Harsh Prateek Bora]

Hi Nick,

On 5/26/24 17:56, Nicholas Piggin wrote:
> This will allow different settings and checks for different
> machine types with later changes.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   hw/ppc/pnv.c | 35 ++++++++++++++++++++++++++++++-----
>   1 file changed, 30 insertions(+), 5 deletions(-)
> 
> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
> index 6e3a5ccdec..a706de2e36 100644
> --- a/hw/ppc/pnv.c
> +++ b/hw/ppc/pnv.c
> @@ -976,11 +976,6 @@ static void pnv_init(MachineState *machine)
>       pnv->num_chips =
>           machine->smp.max_cpus / (machine->smp.cores * machine->smp.threads);
>   
> -    if (machine->smp.threads > 8) {
> -        error_report("Cannot support more than 8 threads/core "
> -                     "on a powernv machine");
> -        exit(1);
> -    }
>       if (!is_power_of_2(machine->smp.threads)) {
>           error_report("Cannot support %d threads/core on a powernv"
>                        "machine because it must be a power of 2",
> @@ -1076,6 +1071,33 @@ static void pnv_init(MachineState *machine)
>       }
>   }
>   
> +static void pnv_power8_init(MachineState *machine)
> +{
> +    if (machine->smp.threads > 8) {
> +        error_report("Cannot support more than 8 threads/core "
> +                     "on a powernv POWER8 machine");

We could use mc->desc for machine name above, so that ..

> +        exit(1);
> +    }

with this patch, we can reuse p8 init for both p9 and p10 (and not just 
reuse p9 for p10 with hard coded string?).

With that,
Reviewed-by: Harsh Prateek Bora <harshpb@linux.ibm.com>

> +
> +    pnv_init(machine);
> +}
> +
> +static void pnv_power9_init(MachineState *machine)
> +{
> +    if (machine->smp.threads > 8) {
> +        error_report("Cannot support more than 8 threads/core "
> +                     "on a powernv9/10 machine");
> +        exit(1);
> +    }
> +
> +    pnv_init(machine);
> +}
> +
> +static void pnv_power10_init(MachineState *machine)
> +{
> +    pnv_power9_init(machine);
> +}
> +
>   /*
>    *    0:21  Reserved - Read as zeros
>    *   22:24  Chip ID
> @@ -2423,6 +2445,7 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
>       };
>   
>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER8";
> +    mc->init = pnv_power8_init;
>       mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power8_v2.0");
>       compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
>   
> @@ -2449,6 +2472,7 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
>       };
>   
>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER9";
> +    mc->init = pnv_power9_init;
>       mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power9_v2.2");
>       compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
>   
> @@ -2473,6 +2497,7 @@ static void pnv_machine_p10_common_class_init(ObjectClass *oc, void *data)
>           { TYPE_PNV_PHB_ROOT_PORT, "version", "5" },
>       };
>   
> +    mc->init = pnv_power10_init;
>       mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power10_v2.0");
>       compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
>   

Re: [RFC PATCH 04/10] ppc/pnv: specialise init for powernv8/9/10 machines

[Cédric Le Goater]

On 5/28/24 09:10, Harsh Prateek Bora wrote:
> Hi Nick,
> 
> On 5/26/24 17:56, Nicholas Piggin wrote:
>> This will allow different settings and checks for different
>> machine types with later changes.
>>
>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>> ---
>>   hw/ppc/pnv.c | 35 ++++++++++++++++++++++++++++++-----
>>   1 file changed, 30 insertions(+), 5 deletions(-)
>>
>> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
>> index 6e3a5ccdec..a706de2e36 100644
>> --- a/hw/ppc/pnv.c
>> +++ b/hw/ppc/pnv.c
>> @@ -976,11 +976,6 @@ static void pnv_init(MachineState *machine)
>>       pnv->num_chips =
>>           machine->smp.max_cpus / (machine->smp.cores * machine->smp.threads);
>> -    if (machine->smp.threads > 8) {
>> -        error_report("Cannot support more than 8 threads/core "
>> -                     "on a powernv machine");
>> -        exit(1);
>> -    }
>>       if (!is_power_of_2(machine->smp.threads)) {
>>           error_report("Cannot support %d threads/core on a powernv"
>>                        "machine because it must be a power of 2",
>> @@ -1076,6 +1071,33 @@ static void pnv_init(MachineState *machine)
>>       }
>>   }
>> +static void pnv_power8_init(MachineState *machine)
>> +{
>> +    if (machine->smp.threads > 8) {
>> +        error_report("Cannot support more than 8 threads/core "
>> +                     "on a powernv POWER8 machine");
> 
> We could use mc->desc for machine name above, so that ..
> 
>> +        exit(1);
>> +    }
> 
> with this patch, we can reuse p8 init for both p9 and p10 (and not just reuse p9 for p10 with hard coded string?).

Good idea. You could add a 'max_smt' attribute to PnvMachineClass to limit
POWER8 to one.


Thanks,

C.


> 
> With that,
> Reviewed-by: Harsh Prateek Bora <harshpb@linux.ibm.com>
> 
>> +
>> +    pnv_init(machine);
>> +}
>> +
>> +static void pnv_power9_init(MachineState *machine)
>> +{
>> +    if (machine->smp.threads > 8) {
>> +        error_report("Cannot support more than 8 threads/core "
>> +                     "on a powernv9/10 machine");
>> +        exit(1);
>> +    }
>> +
>> +    pnv_init(machine);
>> +}
>> +
>> +static void pnv_power10_init(MachineState *machine)
>> +{
>> +    pnv_power9_init(machine);
>> +}
>> +
>>   /*
>>    *    0:21  Reserved - Read as zeros
>>    *   22:24  Chip ID
>> @@ -2423,6 +2445,7 @@ static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
>>       };
>>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER8";
>> +    mc->init = pnv_power8_init;
>>       mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power8_v2.0");
>>       compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
>> @@ -2449,6 +2472,7 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
>>       };
>>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER9";
>> +    mc->init = pnv_power9_init;
>>       mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power9_v2.2");
>>       compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));
>> @@ -2473,6 +2497,7 @@ static void pnv_machine_p10_common_class_init(ObjectClass *oc, void *data)
>>           { TYPE_PNV_PHB_ROOT_PORT, "version", "5" },
>>       };
>> +    mc->init = pnv_power10_init;
>>       mc->default_cpu_type = POWERPC_CPU_TYPE_NAME("power10_v2.0");
>>       compat_props_add(mc->compat_props, phb_compat, G_N_ELEMENTS(phb_compat));

Re: [RFC PATCH 04/10] ppc/pnv: specialise init for powernv8/9/10 machines

[Nicholas Piggin]

On Tue May 28, 2024 at 5:45 PM AEST, Cédric Le Goater wrote:
> On 5/28/24 09:10, Harsh Prateek Bora wrote:
> > Hi Nick,
> > 
> > On 5/26/24 17:56, Nicholas Piggin wrote:
> >> This will allow different settings and checks for different
> >> machine types with later changes.
> >>
> >> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> >> ---
> >>   hw/ppc/pnv.c | 35 ++++++++++++++++++++++++++++++-----
> >>   1 file changed, 30 insertions(+), 5 deletions(-)
> >>
> >> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
> >> index 6e3a5ccdec..a706de2e36 100644
> >> --- a/hw/ppc/pnv.c
> >> +++ b/hw/ppc/pnv.c
> >> @@ -976,11 +976,6 @@ static void pnv_init(MachineState *machine)
> >>       pnv->num_chips =
> >>           machine->smp.max_cpus / (machine->smp.cores * machine->smp.threads);
> >> -    if (machine->smp.threads > 8) {
> >> -        error_report("Cannot support more than 8 threads/core "
> >> -                     "on a powernv machine");
> >> -        exit(1);
> >> -    }
> >>       if (!is_power_of_2(machine->smp.threads)) {
> >>           error_report("Cannot support %d threads/core on a powernv"
> >>                        "machine because it must be a power of 2",
> >> @@ -1076,6 +1071,33 @@ static void pnv_init(MachineState *machine)
> >>       }
> >>   }
> >> +static void pnv_power8_init(MachineState *machine)
> >> +{
> >> +    if (machine->smp.threads > 8) {
> >> +        error_report("Cannot support more than 8 threads/core "
> >> +                     "on a powernv POWER8 machine");
> > 
> > We could use mc->desc for machine name above, so that ..
> > 
> >> +        exit(1);
> >> +    }
> > 
> > with this patch, we can reuse p8 init for both p9 and p10 (and not just reuse p9 for p10 with hard coded string?).
>
> Good idea. You could add a 'max_smt' attribute to PnvMachineClass to limit
> POWER8 to one.

Okay I'll see how that goes. Good suggestions.

Thanks,
Nick

[RFC PATCH 05/10] ppc/pnv: Extend chip_pir class method to TIR as well

[Nicholas Piggin]

The chip_pir chip class method allows the platform to set the PIR
processor identification register. Extend this to a more general
ID function which also allows the TIR to be set. This is in
preparation for "big core", which is a more complicated topology
of cores and threads.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 include/hw/ppc/pnv_chip.h |  3 +-
 hw/ppc/pnv.c              | 61 ++++++++++++++++++++++++---------------
 hw/ppc/pnv_core.c         | 10 ++++---
 3 files changed, 45 insertions(+), 29 deletions(-)

diff --git a/include/hw/ppc/pnv_chip.h b/include/hw/ppc/pnv_chip.h
index 8589f3291e..679723926a 100644
--- a/include/hw/ppc/pnv_chip.h
+++ b/include/hw/ppc/pnv_chip.h
@@ -147,7 +147,8 @@ struct PnvChipClass {
 
     DeviceRealize parent_realize;
 
-    uint32_t (*chip_pir)(PnvChip *chip, uint32_t core_id, uint32_t thread_id);
+    void (*processor_id)(PnvChip *chip, uint32_t core_id, uint32_t thread_id,
+                         uint32_t *pir, uint32_t *tir);
     void (*intc_create)(PnvChip *chip, PowerPCCPU *cpu, Error **errp);
     void (*intc_reset)(PnvChip *chip, PowerPCCPU *cpu);
     void (*intc_destroy)(PnvChip *chip, PowerPCCPU *cpu);
diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
index a706de2e36..7d062ec16c 100644
--- a/hw/ppc/pnv.c
+++ b/hw/ppc/pnv.c
@@ -144,7 +144,7 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
     PnvChipClass *pnv_cc = PNV_CHIP_GET_CLASS(chip);
     g_autofree uint32_t *servers_prop = g_new(uint32_t, smt_threads);
     int i;
-    uint32_t pir;
+    uint32_t pir, tir;
     uint32_t segs[] = {cpu_to_be32(28), cpu_to_be32(40),
                        0xffffffff, 0xffffffff};
     uint32_t tbfreq = PNV_TIMEBASE_FREQ;
@@ -155,7 +155,7 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
     char *nodename;
     int cpus_offset = get_cpus_node(fdt);
 
-    pir = pnv_cc->chip_pir(chip, pc->hwid, 0);
+    pnv_cc->processor_id(chip, pc->hwid, 0, &pir, &tir);
 
     nodename = g_strdup_printf("%s@%x", dc->fw_name, pir);
     offset = fdt_add_subnode(fdt, cpus_offset, nodename);
@@ -237,7 +237,8 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
 
     /* Build interrupt servers properties */
     for (i = 0; i < smt_threads; i++) {
-        servers_prop[i] = cpu_to_be32(pnv_cc->chip_pir(chip, pc->hwid, i));
+        pnv_cc->processor_id(chip, pc->hwid, i, &pir, &tir);
+        servers_prop[i] = cpu_to_be32(pir);
     }
     _FDT((fdt_setprop(fdt, offset, "ibm,ppc-interrupt-server#s",
                        servers_prop, sizeof(*servers_prop) * smt_threads)));
@@ -249,14 +250,17 @@ static void pnv_dt_icp(PnvChip *chip, void *fdt, uint32_t hwid,
                        uint32_t nr_threads)
 {
     PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip);
-    uint32_t pir = pcc->chip_pir(chip, hwid, 0);
-    uint64_t addr = PNV_ICP_BASE(chip) | (pir << 12);
+    uint32_t pir, tir;
+    uint64_t addr;
     char *name;
     const char compat[] = "IBM,power8-icp\0IBM,ppc-xicp";
     uint32_t irange[2], i, rsize;
     uint64_t *reg;
     int offset;
 
+    pcc->processor_id(chip, hwid, 0, &pir, &tir);
+    addr = PNV_ICP_BASE(chip) | (pir << 12);
+
     irange[0] = cpu_to_be32(pir);
     irange[1] = cpu_to_be32(nr_threads);
 
@@ -1104,10 +1108,12 @@ static void pnv_power10_init(MachineState *machine)
  *   25:28  Core number
  *   29:31  Thread ID
  */
-static uint32_t pnv_chip_pir_p8(PnvChip *chip, uint32_t core_id,
-                                uint32_t thread_id)
+static void pnv_processor_id_p8(PnvChip *chip,
+                                uint32_t core_id, uint32_t thread_id,
+                                uint32_t *pir, uint32_t *tir)
 {
-    return (chip->chip_id << 7) | (core_id << 3) | thread_id;
+    *pir = (chip->chip_id << 7) | (core_id << 3) | thread_id;
+    *tir = thread_id;
 }
 
 static void pnv_chip_power8_intc_create(PnvChip *chip, PowerPCCPU *cpu,
@@ -1159,15 +1165,17 @@ static void pnv_chip_power8_intc_print_info(PnvChip *chip, PowerPCCPU *cpu,
  *
  * We only care about the lower bits. uint32_t is fine for the moment.
  */
-static uint32_t pnv_chip_pir_p9(PnvChip *chip, uint32_t core_id,
-                                uint32_t thread_id)
+static void pnv_processor_id_p9(PnvChip *chip,
+                                uint32_t core_id, uint32_t thread_id,
+                                uint32_t *pir, uint32_t *tir)
 {
     if (chip->nr_threads == 8) {
-        return (chip->chip_id << 8) | ((thread_id & 1) << 2) | (core_id << 3) |
+        *pir = (chip->chip_id << 8) | ((thread_id & 1) << 2) | (core_id << 3) |
                (thread_id >> 1);
     } else {
-        return (chip->chip_id << 8) | (core_id << 2) | thread_id;
+        *pir = (chip->chip_id << 8) | (core_id << 2) | thread_id;
     }
+    *tir = thread_id;
 }
 
 /*
@@ -1181,15 +1189,17 @@ static uint32_t pnv_chip_pir_p9(PnvChip *chip, uint32_t core_id,
  *
  * We only care about the lower bits. uint32_t is fine for the moment.
  */
-static uint32_t pnv_chip_pir_p10(PnvChip *chip, uint32_t core_id,
-                                 uint32_t thread_id)
+static void pnv_processor_id_p10(PnvChip *chip,
+                                uint32_t core_id, uint32_t thread_id,
+                                uint32_t *pir, uint32_t *tir)
 {
     if (chip->nr_threads == 8) {
-        return (chip->chip_id << 8) | ((core_id / 4) << 4) |
-               ((core_id % 2) << 3) | thread_id;
+        *pir = (chip->chip_id << 8) | ((core_id / 4) << 4) |
+                ((core_id % 2) << 3) | thread_id;
     } else {
-        return (chip->chip_id << 8) | (core_id << 2) | thread_id;
+        *pir = (chip->chip_id << 8) | (core_id << 2) | thread_id;
     }
+    *tir = thread_id;
 }
 
 static void pnv_chip_power9_intc_create(PnvChip *chip, PowerPCCPU *cpu,
@@ -1368,8 +1378,11 @@ static void pnv_chip_icp_realize(Pnv8Chip *chip8, Error **errp)
         int core_hwid = CPU_CORE(pnv_core)->core_id;
 
         for (j = 0; j < CPU_CORE(pnv_core)->nr_threads; j++) {
-            uint32_t pir = pcc->chip_pir(chip, core_hwid, j);
-            PnvICPState *icp = PNV_ICP(xics_icp_get(chip8->xics, pir));
+            uint32_t pir, tir;
+            PnvICPState *icp;
+
+            pcc->processor_id(chip, core_hwid, j, &pir, &tir);
+            icp = PNV_ICP(xics_icp_get(chip8->xics, pir));
 
             memory_region_add_subregion(&chip8->icp_mmio, pir << 12,
                                         &icp->mmio);
@@ -1481,7 +1494,7 @@ static void pnv_chip_power8e_class_init(ObjectClass *klass, void *data)
     k->chip_cfam_id = 0x221ef04980000000ull;  /* P8 Murano DD2.1 */
     k->cores_mask = POWER8E_CORE_MASK;
     k->num_phbs = 3;
-    k->chip_pir = pnv_chip_pir_p8;
+    k->processor_id = pnv_processor_id_p8;
     k->intc_create = pnv_chip_power8_intc_create;
     k->intc_reset = pnv_chip_power8_intc_reset;
     k->intc_destroy = pnv_chip_power8_intc_destroy;
@@ -1505,7 +1518,7 @@ static void pnv_chip_power8_class_init(ObjectClass *klass, void *data)
     k->chip_cfam_id = 0x220ea04980000000ull; /* P8 Venice DD2.0 */
     k->cores_mask = POWER8_CORE_MASK;
     k->num_phbs = 3;
-    k->chip_pir = pnv_chip_pir_p8;
+    k->processor_id = pnv_processor_id_p8;
     k->intc_create = pnv_chip_power8_intc_create;
     k->intc_reset = pnv_chip_power8_intc_reset;
     k->intc_destroy = pnv_chip_power8_intc_destroy;
@@ -1529,7 +1542,7 @@ static void pnv_chip_power8nvl_class_init(ObjectClass *klass, void *data)
     k->chip_cfam_id = 0x120d304980000000ull;  /* P8 Naples DD1.0 */
     k->cores_mask = POWER8_CORE_MASK;
     k->num_phbs = 4;
-    k->chip_pir = pnv_chip_pir_p8;
+    k->processor_id = pnv_processor_id_p8;
     k->intc_create = pnv_chip_power8_intc_create;
     k->intc_reset = pnv_chip_power8_intc_reset;
     k->intc_destroy = pnv_chip_power8_intc_destroy;
@@ -1802,7 +1815,7 @@ static void pnv_chip_power9_class_init(ObjectClass *klass, void *data)
 
     k->chip_cfam_id = 0x220d104900008000ull; /* P9 Nimbus DD2.0 */
     k->cores_mask = POWER9_CORE_MASK;
-    k->chip_pir = pnv_chip_pir_p9;
+    k->processor_id = pnv_processor_id_p9;
     k->intc_create = pnv_chip_power9_intc_create;
     k->intc_reset = pnv_chip_power9_intc_reset;
     k->intc_destroy = pnv_chip_power9_intc_destroy;
@@ -2114,7 +2127,7 @@ static void pnv_chip_power10_class_init(ObjectClass *klass, void *data)
 
     k->chip_cfam_id = 0x120da04900008000ull; /* P10 DD1.0 (with NX) */
     k->cores_mask = POWER10_CORE_MASK;
-    k->chip_pir = pnv_chip_pir_p10;
+    k->processor_id = pnv_processor_id_p10;
     k->intc_create = pnv_chip_power10_intc_create;
     k->intc_reset = pnv_chip_power10_intc_reset;
     k->intc_destroy = pnv_chip_power10_intc_destroy;
diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
index 7b0ea7812b..9b5edd9e48 100644
--- a/hw/ppc/pnv_core.c
+++ b/hw/ppc/pnv_core.c
@@ -228,8 +228,9 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
     PnvCPUState *pnv_cpu = pnv_cpu_state(cpu);
     CPUPPCState *env = &cpu->env;
     int core_hwid;
-    ppc_spr_t *pir = &env->spr_cb[SPR_PIR];
-    ppc_spr_t *tir = &env->spr_cb[SPR_TIR];
+    ppc_spr_t *pir_spr = &env->spr_cb[SPR_PIR];
+    ppc_spr_t *tir_spr = &env->spr_cb[SPR_TIR];
+    uint32_t pir, tir;
     Error *local_err = NULL;
     PnvChipClass *pcc = PNV_CHIP_GET_CLASS(pc->chip);
 
@@ -247,8 +248,9 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
 
     core_hwid = object_property_get_uint(OBJECT(pc), "hwid", &error_abort);
 
-    tir->default_value = thread_index;
-    pir->default_value = pcc->chip_pir(pc->chip, core_hwid, thread_index);
+    pcc->processor_id(pc->chip, core_hwid, thread_index, &pir, &tir);
+    pir_spr->default_value = pir;
+    tir_spr->default_value = tir;
 
     /* Set time-base frequency to 512 MHz */
     cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
-- 
2.43.0

Re: [RFC PATCH 05/10] ppc/pnv: Extend chip_pir class method to TIR as well

[Harsh Prateek Bora]

On 5/26/24 17:56, Nicholas Piggin wrote:
> The chip_pir chip class method allows the platform to set the PIR
> processor identification register. Extend this to a more general
> ID function which also allows the TIR to be set. This is in
> preparation for "big core", which is a more complicated topology
> of cores and threads.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   include/hw/ppc/pnv_chip.h |  3 +-
>   hw/ppc/pnv.c              | 61 ++++++++++++++++++++++++---------------
>   hw/ppc/pnv_core.c         | 10 ++++---
>   3 files changed, 45 insertions(+), 29 deletions(-)
> 
> diff --git a/include/hw/ppc/pnv_chip.h b/include/hw/ppc/pnv_chip.h
> index 8589f3291e..679723926a 100644
> --- a/include/hw/ppc/pnv_chip.h
> +++ b/include/hw/ppc/pnv_chip.h
> @@ -147,7 +147,8 @@ struct PnvChipClass {
>   
>       DeviceRealize parent_realize;
>   
> -    uint32_t (*chip_pir)(PnvChip *chip, uint32_t core_id, uint32_t thread_id);
> +    void (*processor_id)(PnvChip *chip, uint32_t core_id, uint32_t thread_id,
> +                         uint32_t *pir, uint32_t *tir);

Should it be named get_chip_core_thread_regs() ?

>       void (*intc_create)(PnvChip *chip, PowerPCCPU *cpu, Error **errp);
>       void (*intc_reset)(PnvChip *chip, PowerPCCPU *cpu);
>       void (*intc_destroy)(PnvChip *chip, PowerPCCPU *cpu);
> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
> index a706de2e36..7d062ec16c 100644
> --- a/hw/ppc/pnv.c
> +++ b/hw/ppc/pnv.c
> @@ -144,7 +144,7 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
>       PnvChipClass *pnv_cc = PNV_CHIP_GET_CLASS(chip);
>       g_autofree uint32_t *servers_prop = g_new(uint32_t, smt_threads);
>       int i;
> -    uint32_t pir;
> +    uint32_t pir, tir;
>       uint32_t segs[] = {cpu_to_be32(28), cpu_to_be32(40),
>                          0xffffffff, 0xffffffff};
>       uint32_t tbfreq = PNV_TIMEBASE_FREQ;
> @@ -155,7 +155,7 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
>       char *nodename;
>       int cpus_offset = get_cpus_node(fdt);
>   
> -    pir = pnv_cc->chip_pir(chip, pc->hwid, 0);
> +    pnv_cc->processor_id(chip, pc->hwid, 0, &pir, &tir);

As a generic helper API and potentially expandable, it should allow 
passing NULL for registers whose values are not really sought to avoid 
having to create un-necessary local variables by the caller.

regards,
Harsh
>   
>       nodename = g_strdup_printf("%s@%x", dc->fw_name, pir);
>       offset = fdt_add_subnode(fdt, cpus_offset, nodename);
> @@ -237,7 +237,8 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
>   
>       /* Build interrupt servers properties */
>       for (i = 0; i < smt_threads; i++) {
> -        servers_prop[i] = cpu_to_be32(pnv_cc->chip_pir(chip, pc->hwid, i));
> +        pnv_cc->processor_id(chip, pc->hwid, i, &pir, &tir);
> +        servers_prop[i] = cpu_to_be32(pir);
>       }
>       _FDT((fdt_setprop(fdt, offset, "ibm,ppc-interrupt-server#s",
>                          servers_prop, sizeof(*servers_prop) * smt_threads)));
> @@ -249,14 +250,17 @@ static void pnv_dt_icp(PnvChip *chip, void *fdt, uint32_t hwid,
>                          uint32_t nr_threads)
>   {
>       PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip);
> -    uint32_t pir = pcc->chip_pir(chip, hwid, 0);
> -    uint64_t addr = PNV_ICP_BASE(chip) | (pir << 12);
> +    uint32_t pir, tir;
> +    uint64_t addr;
>       char *name;
>       const char compat[] = "IBM,power8-icp\0IBM,ppc-xicp";
>       uint32_t irange[2], i, rsize;
>       uint64_t *reg;
>       int offset;
>   
> +    pcc->processor_id(chip, hwid, 0, &pir, &tir);
> +    addr = PNV_ICP_BASE(chip) | (pir << 12);
> +
>       irange[0] = cpu_to_be32(pir);
>       irange[1] = cpu_to_be32(nr_threads);
>   
> @@ -1104,10 +1108,12 @@ static void pnv_power10_init(MachineState *machine)
>    *   25:28  Core number
>    *   29:31  Thread ID
>    */
> -static uint32_t pnv_chip_pir_p8(PnvChip *chip, uint32_t core_id,
> -                                uint32_t thread_id)
> +static void pnv_processor_id_p8(PnvChip *chip,
> +                                uint32_t core_id, uint32_t thread_id,
> +                                uint32_t *pir, uint32_t *tir)
>   {
> -    return (chip->chip_id << 7) | (core_id << 3) | thread_id;
> +    *pir = (chip->chip_id << 7) | (core_id << 3) | thread_id;
> +    *tir = thread_id;
>   }
>   
>   static void pnv_chip_power8_intc_create(PnvChip *chip, PowerPCCPU *cpu,
> @@ -1159,15 +1165,17 @@ static void pnv_chip_power8_intc_print_info(PnvChip *chip, PowerPCCPU *cpu,
>    *
>    * We only care about the lower bits. uint32_t is fine for the moment.
>    */
> -static uint32_t pnv_chip_pir_p9(PnvChip *chip, uint32_t core_id,
> -                                uint32_t thread_id)
> +static void pnv_processor_id_p9(PnvChip *chip,
> +                                uint32_t core_id, uint32_t thread_id,
> +                                uint32_t *pir, uint32_t *tir)
>   {
>       if (chip->nr_threads == 8) {
> -        return (chip->chip_id << 8) | ((thread_id & 1) << 2) | (core_id << 3) |
> +        *pir = (chip->chip_id << 8) | ((thread_id & 1) << 2) | (core_id << 3) |
>                  (thread_id >> 1);
>       } else {
> -        return (chip->chip_id << 8) | (core_id << 2) | thread_id;
> +        *pir = (chip->chip_id << 8) | (core_id << 2) | thread_id;
>       }
> +    *tir = thread_id;
>   }
>   
>   /*
> @@ -1181,15 +1189,17 @@ static uint32_t pnv_chip_pir_p9(PnvChip *chip, uint32_t core_id,
>    *
>    * We only care about the lower bits. uint32_t is fine for the moment.
>    */
> -static uint32_t pnv_chip_pir_p10(PnvChip *chip, uint32_t core_id,
> -                                 uint32_t thread_id)
> +static void pnv_processor_id_p10(PnvChip *chip,
> +                                uint32_t core_id, uint32_t thread_id,
> +                                uint32_t *pir, uint32_t *tir)
>   {
>       if (chip->nr_threads == 8) {
> -        return (chip->chip_id << 8) | ((core_id / 4) << 4) |
> -               ((core_id % 2) << 3) | thread_id;
> +        *pir = (chip->chip_id << 8) | ((core_id / 4) << 4) |
> +                ((core_id % 2) << 3) | thread_id;
>       } else {
> -        return (chip->chip_id << 8) | (core_id << 2) | thread_id;
> +        *pir = (chip->chip_id << 8) | (core_id << 2) | thread_id;
>       }
> +    *tir = thread_id;
>   }
>   
>   static void pnv_chip_power9_intc_create(PnvChip *chip, PowerPCCPU *cpu,
> @@ -1368,8 +1378,11 @@ static void pnv_chip_icp_realize(Pnv8Chip *chip8, Error **errp)
>           int core_hwid = CPU_CORE(pnv_core)->core_id;
>   
>           for (j = 0; j < CPU_CORE(pnv_core)->nr_threads; j++) {
> -            uint32_t pir = pcc->chip_pir(chip, core_hwid, j);
> -            PnvICPState *icp = PNV_ICP(xics_icp_get(chip8->xics, pir));
> +            uint32_t pir, tir;
> +            PnvICPState *icp;
> +
> +            pcc->processor_id(chip, core_hwid, j, &pir, &tir);
> +            icp = PNV_ICP(xics_icp_get(chip8->xics, pir));
>   
>               memory_region_add_subregion(&chip8->icp_mmio, pir << 12,
>                                           &icp->mmio);
> @@ -1481,7 +1494,7 @@ static void pnv_chip_power8e_class_init(ObjectClass *klass, void *data)
>       k->chip_cfam_id = 0x221ef04980000000ull;  /* P8 Murano DD2.1 */
>       k->cores_mask = POWER8E_CORE_MASK;
>       k->num_phbs = 3;
> -    k->chip_pir = pnv_chip_pir_p8;
> +    k->processor_id = pnv_processor_id_p8;
>       k->intc_create = pnv_chip_power8_intc_create;
>       k->intc_reset = pnv_chip_power8_intc_reset;
>       k->intc_destroy = pnv_chip_power8_intc_destroy;
> @@ -1505,7 +1518,7 @@ static void pnv_chip_power8_class_init(ObjectClass *klass, void *data)
>       k->chip_cfam_id = 0x220ea04980000000ull; /* P8 Venice DD2.0 */
>       k->cores_mask = POWER8_CORE_MASK;
>       k->num_phbs = 3;
> -    k->chip_pir = pnv_chip_pir_p8;
> +    k->processor_id = pnv_processor_id_p8;
>       k->intc_create = pnv_chip_power8_intc_create;
>       k->intc_reset = pnv_chip_power8_intc_reset;
>       k->intc_destroy = pnv_chip_power8_intc_destroy;
> @@ -1529,7 +1542,7 @@ static void pnv_chip_power8nvl_class_init(ObjectClass *klass, void *data)
>       k->chip_cfam_id = 0x120d304980000000ull;  /* P8 Naples DD1.0 */
>       k->cores_mask = POWER8_CORE_MASK;
>       k->num_phbs = 4;
> -    k->chip_pir = pnv_chip_pir_p8;
> +    k->processor_id = pnv_processor_id_p8;
>       k->intc_create = pnv_chip_power8_intc_create;
>       k->intc_reset = pnv_chip_power8_intc_reset;
>       k->intc_destroy = pnv_chip_power8_intc_destroy;
> @@ -1802,7 +1815,7 @@ static void pnv_chip_power9_class_init(ObjectClass *klass, void *data)
>   
>       k->chip_cfam_id = 0x220d104900008000ull; /* P9 Nimbus DD2.0 */
>       k->cores_mask = POWER9_CORE_MASK;
> -    k->chip_pir = pnv_chip_pir_p9;
> +    k->processor_id = pnv_processor_id_p9;
>       k->intc_create = pnv_chip_power9_intc_create;
>       k->intc_reset = pnv_chip_power9_intc_reset;
>       k->intc_destroy = pnv_chip_power9_intc_destroy;
> @@ -2114,7 +2127,7 @@ static void pnv_chip_power10_class_init(ObjectClass *klass, void *data)
>   
>       k->chip_cfam_id = 0x120da04900008000ull; /* P10 DD1.0 (with NX) */
>       k->cores_mask = POWER10_CORE_MASK;
> -    k->chip_pir = pnv_chip_pir_p10;
> +    k->processor_id = pnv_processor_id_p10;
>       k->intc_create = pnv_chip_power10_intc_create;
>       k->intc_reset = pnv_chip_power10_intc_reset;
>       k->intc_destroy = pnv_chip_power10_intc_destroy;
> diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
> index 7b0ea7812b..9b5edd9e48 100644
> --- a/hw/ppc/pnv_core.c
> +++ b/hw/ppc/pnv_core.c
> @@ -228,8 +228,9 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
>       PnvCPUState *pnv_cpu = pnv_cpu_state(cpu);
>       CPUPPCState *env = &cpu->env;
>       int core_hwid;
> -    ppc_spr_t *pir = &env->spr_cb[SPR_PIR];
> -    ppc_spr_t *tir = &env->spr_cb[SPR_TIR];
> +    ppc_spr_t *pir_spr = &env->spr_cb[SPR_PIR];
> +    ppc_spr_t *tir_spr = &env->spr_cb[SPR_TIR];
> +    uint32_t pir, tir;
>       Error *local_err = NULL;
>       PnvChipClass *pcc = PNV_CHIP_GET_CLASS(pc->chip);
>   
> @@ -247,8 +248,9 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
>   
>       core_hwid = object_property_get_uint(OBJECT(pc), "hwid", &error_abort);
>   
> -    tir->default_value = thread_index;
> -    pir->default_value = pcc->chip_pir(pc->chip, core_hwid, thread_index);
> +    pcc->processor_id(pc->chip, core_hwid, thread_index, &pir, &tir);
> +    pir_spr->default_value = pir;
> +    tir_spr->default_value = tir;
>   
>       /* Set time-base frequency to 512 MHz */
>       cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);

Re: [RFC PATCH 05/10] ppc/pnv: Extend chip_pir class method to TIR as well

[Nicholas Piggin]

On Tue May 28, 2024 at 6:32 PM AEST, Harsh Prateek Bora wrote:
>
>
> On 5/26/24 17:56, Nicholas Piggin wrote:
> > The chip_pir chip class method allows the platform to set the PIR
> > processor identification register. Extend this to a more general
> > ID function which also allows the TIR to be set. This is in
> > preparation for "big core", which is a more complicated topology
> > of cores and threads.
> > 
> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> > ---
> >   include/hw/ppc/pnv_chip.h |  3 +-
> >   hw/ppc/pnv.c              | 61 ++++++++++++++++++++++++---------------
> >   hw/ppc/pnv_core.c         | 10 ++++---
> >   3 files changed, 45 insertions(+), 29 deletions(-)
> > 
> > diff --git a/include/hw/ppc/pnv_chip.h b/include/hw/ppc/pnv_chip.h
> > index 8589f3291e..679723926a 100644
> > --- a/include/hw/ppc/pnv_chip.h
> > +++ b/include/hw/ppc/pnv_chip.h
> > @@ -147,7 +147,8 @@ struct PnvChipClass {
> >   
> >       DeviceRealize parent_realize;
> >   
> > -    uint32_t (*chip_pir)(PnvChip *chip, uint32_t core_id, uint32_t thread_id);
> > +    void (*processor_id)(PnvChip *chip, uint32_t core_id, uint32_t thread_id,
> > +                         uint32_t *pir, uint32_t *tir);
>
> Should it be named get_chip_core_thread_regs() ?

Yeah, the name isn't great. It is getting the regs, but the regs are the
"pervasive id" used as well... but maybe that's not too relevant here.
What about we drop chip_ since we have the chip and no other methods use
such prefix, then call it get_thread_pir_tir()?

> > @@ -155,7 +155,7 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
> >       char *nodename;
> >       int cpus_offset = get_cpus_node(fdt);
> >   
> > -    pir = pnv_cc->chip_pir(chip, pc->hwid, 0);
> > +    pnv_cc->processor_id(chip, pc->hwid, 0, &pir, &tir);
>
> As a generic helper API and potentially expandable, it should allow 
> passing NULL for registers whose values are not really sought to avoid 
> having to create un-necessary local variables by the caller.

I'll do that.

Thanks,
Nick

Re: [RFC PATCH 05/10] ppc/pnv: Extend chip_pir class method to TIR as well

[Cédric Le Goater]

On 5/29/24 02:24, Nicholas Piggin wrote:
> On Tue May 28, 2024 at 6:32 PM AEST, Harsh Prateek Bora wrote:
>>
>>
>> On 5/26/24 17:56, Nicholas Piggin wrote:
>>> The chip_pir chip class method allows the platform to set the PIR
>>> processor identification register. Extend this to a more general
>>> ID function which also allows the TIR to be set. This is in
>>> preparation for "big core", which is a more complicated topology
>>> of cores and threads.
>>>
>>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>> ---
>>>    include/hw/ppc/pnv_chip.h |  3 +-
>>>    hw/ppc/pnv.c              | 61 ++++++++++++++++++++++++---------------
>>>    hw/ppc/pnv_core.c         | 10 ++++---
>>>    3 files changed, 45 insertions(+), 29 deletions(-)
>>>
>>> diff --git a/include/hw/ppc/pnv_chip.h b/include/hw/ppc/pnv_chip.h
>>> index 8589f3291e..679723926a 100644
>>> --- a/include/hw/ppc/pnv_chip.h
>>> +++ b/include/hw/ppc/pnv_chip.h
>>> @@ -147,7 +147,8 @@ struct PnvChipClass {
>>>    
>>>        DeviceRealize parent_realize;
>>>    
>>> -    uint32_t (*chip_pir)(PnvChip *chip, uint32_t core_id, uint32_t thread_id);
>>> +    void (*processor_id)(PnvChip *chip, uint32_t core_id, uint32_t thread_id,
>>> +                         uint32_t *pir, uint32_t *tir);
>>
>> Should it be named get_chip_core_thread_regs() ?
> 
> Yeah, the name isn't great. It is getting the regs, but the regs are the
> "pervasive id" used as well... but maybe that's not too relevant here.
> What about we drop chip_ since we have the chip and no other methods use
> such prefix, then call it get_thread_pir_tir()?

processor relates to chip and so, processor_id() is not great indeed.
get_pir_tir() would be enough I think.

What would be good though, since pnv is growing, is to start adding
documentation to these common helpers.


Thanks,

C.



>>> @@ -155,7 +155,7 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
>>>        char *nodename;
>>>        int cpus_offset = get_cpus_node(fdt);
>>>    
>>> -    pir = pnv_cc->chip_pir(chip, pc->hwid, 0);
>>> +    pnv_cc->processor_id(chip, pc->hwid, 0, &pir, &tir);
>>
>> As a generic helper API and potentially expandable, it should allow
>> passing NULL for registers whose values are not really sought to avoid
>> having to create un-necessary local variables by the caller.
> 
> I'll do that.
> 
> Thanks,
> Nick

Re: [RFC PATCH 05/10] ppc/pnv: Extend chip_pir class method to TIR as well

[Nicholas Piggin]

On Wed May 29, 2024 at 4:30 PM AEST, Cédric Le Goater wrote:
> On 5/29/24 02:24, Nicholas Piggin wrote:
> > On Tue May 28, 2024 at 6:32 PM AEST, Harsh Prateek Bora wrote:
> >>
> >>
> >> On 5/26/24 17:56, Nicholas Piggin wrote:
> >>> The chip_pir chip class method allows the platform to set the PIR
> >>> processor identification register. Extend this to a more general
> >>> ID function which also allows the TIR to be set. This is in
> >>> preparation for "big core", which is a more complicated topology
> >>> of cores and threads.
> >>>
> >>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> >>> ---
> >>>    include/hw/ppc/pnv_chip.h |  3 +-
> >>>    hw/ppc/pnv.c              | 61 ++++++++++++++++++++++++---------------
> >>>    hw/ppc/pnv_core.c         | 10 ++++---
> >>>    3 files changed, 45 insertions(+), 29 deletions(-)
> >>>
> >>> diff --git a/include/hw/ppc/pnv_chip.h b/include/hw/ppc/pnv_chip.h
> >>> index 8589f3291e..679723926a 100644
> >>> --- a/include/hw/ppc/pnv_chip.h
> >>> +++ b/include/hw/ppc/pnv_chip.h
> >>> @@ -147,7 +147,8 @@ struct PnvChipClass {
> >>>    
> >>>        DeviceRealize parent_realize;
> >>>    
> >>> -    uint32_t (*chip_pir)(PnvChip *chip, uint32_t core_id, uint32_t thread_id);
> >>> +    void (*processor_id)(PnvChip *chip, uint32_t core_id, uint32_t thread_id,
> >>> +                         uint32_t *pir, uint32_t *tir);
> >>
> >> Should it be named get_chip_core_thread_regs() ?
> > 
> > Yeah, the name isn't great. It is getting the regs, but the regs are the
> > "pervasive id" used as well... but maybe that's not too relevant here.
> > What about we drop chip_ since we have the chip and no other methods use
> > such prefix, then call it get_thread_pir_tir()?
>
> processor relates to chip and so, processor_id() is not great indeed.
> get_pir_tir() would be enough I think.
>
> What would be good though, since pnv is growing, is to start adding
> documentation to these common helpers.

Okay we'll use that name.

You mean just a comment them in the header? Might as well do that for
new ones at least.

Thanks,
Nick

Re: [RFC PATCH 05/10] ppc/pnv: Extend chip_pir class method to TIR as well

[Cédric Le Goater]

On 5/30/24 08:38, Nicholas Piggin wrote:
> On Wed May 29, 2024 at 4:30 PM AEST, Cédric Le Goater wrote:
>> On 5/29/24 02:24, Nicholas Piggin wrote:
>>> On Tue May 28, 2024 at 6:32 PM AEST, Harsh Prateek Bora wrote:
>>>>
>>>>
>>>> On 5/26/24 17:56, Nicholas Piggin wrote:
>>>>> The chip_pir chip class method allows the platform to set the PIR
>>>>> processor identification register. Extend this to a more general
>>>>> ID function which also allows the TIR to be set. This is in
>>>>> preparation for "big core", which is a more complicated topology
>>>>> of cores and threads.
>>>>>
>>>>> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
>>>>> ---
>>>>>     include/hw/ppc/pnv_chip.h |  3 +-
>>>>>     hw/ppc/pnv.c              | 61 ++++++++++++++++++++++++---------------
>>>>>     hw/ppc/pnv_core.c         | 10 ++++---
>>>>>     3 files changed, 45 insertions(+), 29 deletions(-)
>>>>>
>>>>> diff --git a/include/hw/ppc/pnv_chip.h b/include/hw/ppc/pnv_chip.h
>>>>> index 8589f3291e..679723926a 100644
>>>>> --- a/include/hw/ppc/pnv_chip.h
>>>>> +++ b/include/hw/ppc/pnv_chip.h
>>>>> @@ -147,7 +147,8 @@ struct PnvChipClass {
>>>>>     
>>>>>         DeviceRealize parent_realize;
>>>>>     
>>>>> -    uint32_t (*chip_pir)(PnvChip *chip, uint32_t core_id, uint32_t thread_id);
>>>>> +    void (*processor_id)(PnvChip *chip, uint32_t core_id, uint32_t thread_id,
>>>>> +                         uint32_t *pir, uint32_t *tir);
>>>>
>>>> Should it be named get_chip_core_thread_regs() ?
>>>
>>> Yeah, the name isn't great. It is getting the regs, but the regs are the
>>> "pervasive id" used as well... but maybe that's not too relevant here.
>>> What about we drop chip_ since we have the chip and no other methods use
>>> such prefix, then call it get_thread_pir_tir()?
>>
>> processor relates to chip and so, processor_id() is not great indeed.
>> get_pir_tir() would be enough I think.
>>
>> What would be good though, since pnv is growing, is to start adding
>> documentation to these common helpers.
> 
> Okay we'll use that name.
> 
> You mean just a comment them in the header? Might as well do that for
> new ones at least.

Yes. All class handlers should have a minimal description.

Thanks,

C.

[RFC PATCH 06/10] ppc: Add a core_index to CPUPPCState for SMT vCPUs

[Nicholas Piggin]

The way SMT thread siblings are matched is clunky, using hard-coded
logic that checks the PIR SPR.

Change that to use a new core_index variable in the CPUPPCState,
where all siblings have the same core_index. CPU realize routines have
flexibility in setting core/sibling topology.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 target/ppc/cpu.h        | 5 ++++-
 hw/ppc/pnv_core.c       | 2 ++
 hw/ppc/spapr_cpu_core.c | 3 +++
 3 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index dac13d4dac..9a89083932 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -1247,6 +1247,9 @@ struct CPUArchState {
     /* when a memory exception occurs, the access type is stored here */
     int access_type;
 
+    /* For SMT processors */
+    int core_index;
+
 #if !defined(CONFIG_USER_ONLY)
     /* MMU context, only relevant for full system emulation */
 #if defined(TARGET_PPC64)
@@ -1404,7 +1407,7 @@ struct CPUArchState {
 };
 
 #define _CORE_ID(cs)                                            \
-    (POWERPC_CPU(cs)->env.spr_cb[SPR_PIR].default_value & ~(cs->nr_threads - 1))
+    (POWERPC_CPU(cs)->env.core_index)
 
 #define THREAD_SIBLING_FOREACH(cs, cs_sibling)                  \
     CPU_FOREACH(cs_sibling)                                     \
diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
index 9b5edd9e48..0f61aabb77 100644
--- a/hw/ppc/pnv_core.c
+++ b/hw/ppc/pnv_core.c
@@ -252,6 +252,8 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
     pir_spr->default_value = pir;
     tir_spr->default_value = tir;
 
+    env->core_index = core_hwid;
+
     /* Set time-base frequency to 512 MHz */
     cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
 }
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
index e7c9edd033..059d372c8a 100644
--- a/hw/ppc/spapr_cpu_core.c
+++ b/hw/ppc/spapr_cpu_core.c
@@ -300,16 +300,19 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
     g_autofree char *id = NULL;
     CPUState *cs;
     PowerPCCPU *cpu;
+    CPUPPCState *env;
 
     obj = object_new(scc->cpu_type);
 
     cs = CPU(obj);
     cpu = POWERPC_CPU(obj);
+    env = &cpu->env;
     /*
      * All CPUs start halted. CPU0 is unhalted from the machine level reset code
      * and the rest are explicitly started up by the guest using an RTAS call.
      */
     qdev_prop_set_bit(DEVICE(obj), "start-powered-off", true);
+    env->core_index = cc->core_id;
     cs->cpu_index = cc->core_id + i;
     if (!spapr_set_vcpu_id(cpu, cs->cpu_index, errp)) {
         return NULL;
-- 
2.43.0

Re: [RFC PATCH 06/10] ppc: Add a core_index to CPUPPCState for SMT vCPUs

[Harsh Prateek Bora]

On 5/26/24 17:56, Nicholas Piggin wrote:
> The way SMT thread siblings are matched is clunky, using hard-coded
> logic that checks the PIR SPR.
> 
> Change that to use a new core_index variable in the CPUPPCState,
> where all siblings have the same core_index. CPU realize routines have
> flexibility in setting core/sibling topology.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   target/ppc/cpu.h        | 5 ++++-
>   hw/ppc/pnv_core.c       | 2 ++
>   hw/ppc/spapr_cpu_core.c | 3 +++
>   3 files changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
> index dac13d4dac..9a89083932 100644
> --- a/target/ppc/cpu.h
> +++ b/target/ppc/cpu.h
> @@ -1247,6 +1247,9 @@ struct CPUArchState {
>       /* when a memory exception occurs, the access type is stored here */
>       int access_type;
>   
> +    /* For SMT processors */
> +    int core_index;
> +
>   #if !defined(CONFIG_USER_ONLY)
>       /* MMU context, only relevant for full system emulation */
>   #if defined(TARGET_PPC64)
> @@ -1404,7 +1407,7 @@ struct CPUArchState {
>   };
>   
>   #define _CORE_ID(cs)                                            \
> -    (POWERPC_CPU(cs)->env.spr_cb[SPR_PIR].default_value & ~(cs->nr_threads - 1))
> +    (POWERPC_CPU(cs)->env.core_index)

Dont we want to keep the logical & with ~(cs->nr_threads - 1) ?
How's it taken care ?

>   
>   #define THREAD_SIBLING_FOREACH(cs, cs_sibling)                  \
>       CPU_FOREACH(cs_sibling)                                     \
> diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
> index 9b5edd9e48..0f61aabb77 100644
> --- a/hw/ppc/pnv_core.c
> +++ b/hw/ppc/pnv_core.c
> @@ -252,6 +252,8 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
>       pir_spr->default_value = pir;
>       tir_spr->default_value = tir;
>   
> +    env->core_index = core_hwid;
> +
>       /* Set time-base frequency to 512 MHz */
>       cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
>   }
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> index e7c9edd033..059d372c8a 100644
> --- a/hw/ppc/spapr_cpu_core.c
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -300,16 +300,19 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
>       g_autofree char *id = NULL;
>       CPUState *cs;
>       PowerPCCPU *cpu;
> +    CPUPPCState *env;
>   
>       obj = object_new(scc->cpu_type);
>   
>       cs = CPU(obj);
>       cpu = POWERPC_CPU(obj);
> +    env = &cpu->env;
>       /*
>        * All CPUs start halted. CPU0 is unhalted from the machine level reset code
>        * and the rest are explicitly started up by the guest using an RTAS call.
>        */
>       qdev_prop_set_bit(DEVICE(obj), "start-powered-off", true);
> +    env->core_index = cc->core_id;

We could just do cpu->env.core_index and avoid creating local var env.

regards,
Harsh

>       cs->cpu_index = cc->core_id + i;
>       if (!spapr_set_vcpu_id(cpu, cs->cpu_index, errp)) {
>           return NULL;

Re: [RFC PATCH 06/10] ppc: Add a core_index to CPUPPCState for SMT vCPUs

[Harsh Prateek Bora]

corrected typo, it's bitwise.

On 5/28/24 14:18, Harsh Prateek Bora wrote:
>> -    (POWERPC_CPU(cs)->env.spr_cb[SPR_PIR].default_value & 
>> ~(cs->nr_threads - 1))
>> +    (POWERPC_CPU(cs)->env.core_index)
> 
> Dont we want to keep the bitwise & with ~(cs->nr_threads - 1) ?
> How's it taken care ?

Re: [RFC PATCH 06/10] ppc: Add a core_index to CPUPPCState for SMT vCPUs

[Nicholas Piggin]

On Tue May 28, 2024 at 6:52 PM AEST, Harsh Prateek Bora wrote:
> corrected typo, it's bitwise.
>
> On 5/28/24 14:18, Harsh Prateek Bora wrote:
> >> -    (POWERPC_CPU(cs)->env.spr_cb[SPR_PIR].default_value & 
> >> ~(cs->nr_threads - 1))
> >> +    (POWERPC_CPU(cs)->env.core_index)
> > 
> > Dont we want to keep the bitwise & with ~(cs->nr_threads - 1) ?
> > How's it taken care ?

For these accessors it actually just wants to have something that
compares if a CPU belongs to the same core or not, so exact value
doesn't really matter.

Maybe the helpers should do that comparison. It could possibly even
be a class method to be really clean, although that's more costly
to call (but writing to a SMT shared register is pretty costly anyway
so maybe doesn't matter).

I'll think a bit more.

Thanks,
Nick

[RFC PATCH 07/10] target/ppc: Add helpers to check for SMT sibling threads

[Nicholas Piggin]

Add helpers for TCG code to determine if there are SMT siblings
sharing per-core and per-lpar registers. This simplifies the
callers and makes SMT register topology simpler to modify with
later changes.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 target/ppc/cpu.h             |  7 +++++++
 target/ppc/cpu_init.c        |  2 +-
 target/ppc/excp_helper.c     | 16 +++++++---------
 target/ppc/misc_helper.c     | 27 ++++++---------------------
 target/ppc/timebase_helper.c | 20 +++++++-------------
 5 files changed, 28 insertions(+), 44 deletions(-)

diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 9a89083932..8fd6ade471 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -1406,6 +1406,13 @@ struct CPUArchState {
     uint64_t pmu_base_time;
 };
 
+#define PPC_CPU_HAS_CORE_SIBLINGS(cs)                           \
+    (cs->nr_threads > 1)
+
+#define PPC_CPU_HAS_LPAR_SIBLINGS(cs)                           \
+    ((POWERPC_CPU(cs)->env.flags & POWERPC_FLAG_SMT_1LPAR) &&   \
+     PPC_CPU_HAS_CORE_SIBLINGS(cs))
+
 #define _CORE_ID(cs)                                            \
     (POWERPC_CPU(cs)->env.core_index)
 
diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
index ae483e20c4..e71ee008ed 100644
--- a/target/ppc/cpu_init.c
+++ b/target/ppc/cpu_init.c
@@ -6975,7 +6975,7 @@ static void ppc_cpu_realize(DeviceState *dev, Error **errp)
 
     pcc->parent_realize(dev, errp);
 
-    if (env_cpu(env)->nr_threads > 1) {
+    if (PPC_CPU_HAS_CORE_SIBLINGS(cs)) {
         env->flags |= POWERPC_FLAG_SMT;
     }
 
diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c
index 0cd542675f..fd45da0f2b 100644
--- a/target/ppc/excp_helper.c
+++ b/target/ppc/excp_helper.c
@@ -3029,7 +3029,7 @@ void helper_book3s_msgsnd(CPUPPCState *env, target_ulong rb)
         brdcast = true;
     }
 
-    if (cs->nr_threads == 1 || !brdcast) {
+    if (!PPC_CPU_HAS_CORE_SIBLINGS(cs) || !brdcast) {
         ppc_set_irq(cpu, PPC_INTERRUPT_HDOORBELL, 1);
         return;
     }
@@ -3067,21 +3067,19 @@ void helper_book3s_msgsndp(CPUPPCState *env, target_ulong rb)
     CPUState *cs = env_cpu(env);
     PowerPCCPU *cpu = env_archcpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
     int ttir = rb & PPC_BITMASK(57, 63);
 
     helper_hfscr_facility_check(env, HFSCR_MSGP, "msgsndp", HFSCR_IC_MSGP);
 
-    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
-        nr_threads = 1; /* msgsndp behaves as 1-thread in LPAR-per-thread mode*/
-    }
-
-    if (!dbell_type_server(rb) || ttir >= nr_threads) {
+    if (!dbell_type_server(rb)) {
         return;
     }
 
-    if (nr_threads == 1) {
-        ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, 1);
+    /* msgsndp behaves as 1-thread in LPAR-per-thread mode*/
+    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
+        if (ttir == 0) {
+            ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, 1);
+        }
         return;
     }
 
diff --git a/target/ppc/misc_helper.c b/target/ppc/misc_helper.c
index 46ba3a5584..598c956cdd 100644
--- a/target/ppc/misc_helper.c
+++ b/target/ppc/misc_helper.c
@@ -49,9 +49,8 @@ void helper_spr_core_write_generic(CPUPPCState *env, uint32_t sprn,
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
-    if (nr_threads == 1) {
+    if (!PPC_CPU_HAS_CORE_SIBLINGS(cs)) {
         env->spr[sprn] = val;
         return;
     }
@@ -196,7 +195,7 @@ void helper_store_ptcr(CPUPPCState *env, target_ulong val)
             return;
         }
 
-        if (cs->nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+        if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
             env->spr[SPR_PTCR] = val;
             tlb_flush(cs);
         } else {
@@ -243,16 +242,12 @@ target_ulong helper_load_dpdes(CPUPPCState *env)
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
     target_ulong dpdes = 0;
 
     helper_hfscr_facility_check(env, HFSCR_MSGP, "load DPDES", HFSCR_IC_MSGP);
 
-    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
-        nr_threads = 1; /* DPDES behaves as 1-thread in LPAR-per-thread mode */
-    }
-
-    if (nr_threads == 1) {
+    /* DPDES behaves as 1-thread in LPAR-per-thread mode */
+    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
         if (env->pending_interrupts & PPC_INTERRUPT_DOORBELL) {
             dpdes = 1;
         }
@@ -279,21 +274,11 @@ void helper_store_dpdes(CPUPPCState *env, target_ulong val)
     PowerPCCPU *cpu = env_archcpu(env);
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
     helper_hfscr_facility_check(env, HFSCR_MSGP, "store DPDES", HFSCR_IC_MSGP);
 
-    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
-        nr_threads = 1; /* DPDES behaves as 1-thread in LPAR-per-thread mode */
-    }
-
-    if (val & ~(nr_threads - 1)) {
-        qemu_log_mask(LOG_GUEST_ERROR, "Invalid DPDES register value "
-                      TARGET_FMT_lx"\n", val);
-        val &= (nr_threads - 1); /* Ignore the invalid bits */
-    }
-
-    if (nr_threads == 1) {
+    /* DPDES behaves as 1-thread in LPAR-per-thread mode */
+    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
         ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, val & 0x1);
         return;
     }
diff --git a/target/ppc/timebase_helper.c b/target/ppc/timebase_helper.c
index 788c498d63..abe7b95696 100644
--- a/target/ppc/timebase_helper.c
+++ b/target/ppc/timebase_helper.c
@@ -63,9 +63,8 @@ void helper_store_purr(CPUPPCState *env, target_ulong val)
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
-    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
         cpu_ppc_store_purr(env, val);
         return;
     }
@@ -82,9 +81,8 @@ void helper_store_tbl(CPUPPCState *env, target_ulong val)
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
-    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
         cpu_ppc_store_tbl(env, val);
         return;
     }
@@ -99,9 +97,8 @@ void helper_store_tbu(CPUPPCState *env, target_ulong val)
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
-    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
         cpu_ppc_store_tbu(env, val);
         return;
     }
@@ -141,9 +138,8 @@ void helper_store_hdecr(CPUPPCState *env, target_ulong val)
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
-    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
         cpu_ppc_store_hdecr(env, val);
         return;
     }
@@ -158,9 +154,8 @@ void helper_store_vtb(CPUPPCState *env, target_ulong val)
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
-    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
         cpu_ppc_store_vtb(env, val);
         return;
     }
@@ -175,9 +170,8 @@ void helper_store_tbu40(CPUPPCState *env, target_ulong val)
 {
     CPUState *cs = env_cpu(env);
     CPUState *ccs;
-    uint32_t nr_threads = cs->nr_threads;
 
-    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
+    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
         cpu_ppc_store_tbu40(env, val);
         return;
     }
@@ -288,7 +282,7 @@ static void write_tfmr(CPUPPCState *env, target_ulong val)
 {
     CPUState *cs = env_cpu(env);
 
-    if (cs->nr_threads == 1) {
+    if (!PPC_CPU_HAS_CORE_SIBLINGS(cs)) {
         env->spr[SPR_TFMR] = val;
     } else {
         CPUState *ccs;
-- 
2.43.0

Re: [RFC PATCH 07/10] target/ppc: Add helpers to check for SMT sibling threads

[Harsh Prateek Bora]

On 5/26/24 17:56, Nicholas Piggin wrote:
> Add helpers for TCG code to determine if there are SMT siblings
> sharing per-core and per-lpar registers. This simplifies the
> callers and makes SMT register topology simpler to modify with
> later changes.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   target/ppc/cpu.h             |  7 +++++++
>   target/ppc/cpu_init.c        |  2 +-
>   target/ppc/excp_helper.c     | 16 +++++++---------
>   target/ppc/misc_helper.c     | 27 ++++++---------------------
>   target/ppc/timebase_helper.c | 20 +++++++-------------
>   5 files changed, 28 insertions(+), 44 deletions(-)
> 
> diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
> index 9a89083932..8fd6ade471 100644
> --- a/target/ppc/cpu.h
> +++ b/target/ppc/cpu.h
> @@ -1406,6 +1406,13 @@ struct CPUArchState {
>       uint64_t pmu_base_time;
>   };
>   
> +#define PPC_CPU_HAS_CORE_SIBLINGS(cs)                           \
> +    (cs->nr_threads > 1)
> +
> +#define PPC_CPU_HAS_LPAR_SIBLINGS(cs)                           \
> +    ((POWERPC_CPU(cs)->env.flags & POWERPC_FLAG_SMT_1LPAR) &&   \
> +     PPC_CPU_HAS_CORE_SIBLINGS(cs))
> +
>   #define _CORE_ID(cs)                                            \
>       (POWERPC_CPU(cs)->env.core_index)
>   
> diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
> index ae483e20c4..e71ee008ed 100644
> --- a/target/ppc/cpu_init.c
> +++ b/target/ppc/cpu_init.c
> @@ -6975,7 +6975,7 @@ static void ppc_cpu_realize(DeviceState *dev, Error **errp)
>   
>       pcc->parent_realize(dev, errp);
>   
> -    if (env_cpu(env)->nr_threads > 1) {
> +    if (PPC_CPU_HAS_CORE_SIBLINGS(cs)) {
>           env->flags |= POWERPC_FLAG_SMT;
>       }
>   
> diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c
> index 0cd542675f..fd45da0f2b 100644
> --- a/target/ppc/excp_helper.c
> +++ b/target/ppc/excp_helper.c
> @@ -3029,7 +3029,7 @@ void helper_book3s_msgsnd(CPUPPCState *env, target_ulong rb)
>           brdcast = true;
>       }
>   
> -    if (cs->nr_threads == 1 || !brdcast) {
> +    if (!PPC_CPU_HAS_CORE_SIBLINGS(cs) || !brdcast) {

Since there are multiple usage of above macro in negation below as well, 
we may probably want to introduce another macro PPC_CPU_HAS_SINGLE_CORE
which checks only for nr_threads == 1. Anyways,

Reviewed-by: Harsh Prateek Bora <harshpb@linux.ibm.com>


>           ppc_set_irq(cpu, PPC_INTERRUPT_HDOORBELL, 1);
>           return;
>       }
> @@ -3067,21 +3067,19 @@ void helper_book3s_msgsndp(CPUPPCState *env, target_ulong rb)
>       CPUState *cs = env_cpu(env);
>       PowerPCCPU *cpu = env_archcpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>       int ttir = rb & PPC_BITMASK(57, 63);
>   
>       helper_hfscr_facility_check(env, HFSCR_MSGP, "msgsndp", HFSCR_IC_MSGP);
>   
> -    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> -        nr_threads = 1; /* msgsndp behaves as 1-thread in LPAR-per-thread mode*/
> -    }
> -
> -    if (!dbell_type_server(rb) || ttir >= nr_threads) {
> +    if (!dbell_type_server(rb)) {
>           return;
>       }
>   
> -    if (nr_threads == 1) {
> -        ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, 1);
> +    /* msgsndp behaves as 1-thread in LPAR-per-thread mode*/
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
> +        if (ttir == 0) {
> +            ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, 1);
> +        }
>           return;
>       }
>   
> diff --git a/target/ppc/misc_helper.c b/target/ppc/misc_helper.c
> index 46ba3a5584..598c956cdd 100644
> --- a/target/ppc/misc_helper.c
> +++ b/target/ppc/misc_helper.c
> @@ -49,9 +49,8 @@ void helper_spr_core_write_generic(CPUPPCState *env, uint32_t sprn,
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
> -    if (nr_threads == 1) {
> +    if (!PPC_CPU_HAS_CORE_SIBLINGS(cs)) {
>           env->spr[sprn] = val;
>           return;
>       }
> @@ -196,7 +195,7 @@ void helper_store_ptcr(CPUPPCState *env, target_ulong val)
>               return;
>           }
>   
> -        if (cs->nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> +        if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>               env->spr[SPR_PTCR] = val;
>               tlb_flush(cs);
>           } else {
> @@ -243,16 +242,12 @@ target_ulong helper_load_dpdes(CPUPPCState *env)
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>       target_ulong dpdes = 0;
>   
>       helper_hfscr_facility_check(env, HFSCR_MSGP, "load DPDES", HFSCR_IC_MSGP);
>   
> -    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> -        nr_threads = 1; /* DPDES behaves as 1-thread in LPAR-per-thread mode */
> -    }
> -
> -    if (nr_threads == 1) {
> +    /* DPDES behaves as 1-thread in LPAR-per-thread mode */
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           if (env->pending_interrupts & PPC_INTERRUPT_DOORBELL) {
>               dpdes = 1;
>           }
> @@ -279,21 +274,11 @@ void helper_store_dpdes(CPUPPCState *env, target_ulong val)
>       PowerPCCPU *cpu = env_archcpu(env);
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
>       helper_hfscr_facility_check(env, HFSCR_MSGP, "store DPDES", HFSCR_IC_MSGP);
>   
> -    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> -        nr_threads = 1; /* DPDES behaves as 1-thread in LPAR-per-thread mode */
> -    }
> -
> -    if (val & ~(nr_threads - 1)) {
> -        qemu_log_mask(LOG_GUEST_ERROR, "Invalid DPDES register value "
> -                      TARGET_FMT_lx"\n", val);
> -        val &= (nr_threads - 1); /* Ignore the invalid bits */
> -    }
> -
> -    if (nr_threads == 1) {
> +    /* DPDES behaves as 1-thread in LPAR-per-thread mode */
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, val & 0x1);
>           return;
>       }
> diff --git a/target/ppc/timebase_helper.c b/target/ppc/timebase_helper.c
> index 788c498d63..abe7b95696 100644
> --- a/target/ppc/timebase_helper.c
> +++ b/target/ppc/timebase_helper.c
> @@ -63,9 +63,8 @@ void helper_store_purr(CPUPPCState *env, target_ulong val)
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
> -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           cpu_ppc_store_purr(env, val);
>           return;
>       }
> @@ -82,9 +81,8 @@ void helper_store_tbl(CPUPPCState *env, target_ulong val)
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
> -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           cpu_ppc_store_tbl(env, val);
>           return;
>       }
> @@ -99,9 +97,8 @@ void helper_store_tbu(CPUPPCState *env, target_ulong val)
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
> -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           cpu_ppc_store_tbu(env, val);
>           return;
>       }
> @@ -141,9 +138,8 @@ void helper_store_hdecr(CPUPPCState *env, target_ulong val)
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
> -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           cpu_ppc_store_hdecr(env, val);
>           return;
>       }
> @@ -158,9 +154,8 @@ void helper_store_vtb(CPUPPCState *env, target_ulong val)
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
> -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           cpu_ppc_store_vtb(env, val);
>           return;
>       }
> @@ -175,9 +170,8 @@ void helper_store_tbu40(CPUPPCState *env, target_ulong val)
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
> -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           cpu_ppc_store_tbu40(env, val);
>           return;
>       }
> @@ -288,7 +282,7 @@ static void write_tfmr(CPUPPCState *env, target_ulong val)
>   {
>       CPUState *cs = env_cpu(env);
>   
> -    if (cs->nr_threads == 1) {
> +    if (!PPC_CPU_HAS_CORE_SIBLINGS(cs)) {
>           env->spr[SPR_TFMR] = val;
>       } else {
>           CPUState *ccs;

Re: [RFC PATCH 07/10] target/ppc: Add helpers to check for SMT sibling threads

[Nicholas Piggin]

On Tue May 28, 2024 at 7:16 PM AEST, Harsh Prateek Bora wrote:
>
>
> On 5/26/24 17:56, Nicholas Piggin wrote:
> > Add helpers for TCG code to determine if there are SMT siblings
> > sharing per-core and per-lpar registers. This simplifies the
> > callers and makes SMT register topology simpler to modify with
> > later changes.
> > 
> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> > ---
> >   target/ppc/cpu.h             |  7 +++++++
> >   target/ppc/cpu_init.c        |  2 +-
> >   target/ppc/excp_helper.c     | 16 +++++++---------
> >   target/ppc/misc_helper.c     | 27 ++++++---------------------
> >   target/ppc/timebase_helper.c | 20 +++++++-------------
> >   5 files changed, 28 insertions(+), 44 deletions(-)
> > 
> > diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
> > index 9a89083932..8fd6ade471 100644
> > --- a/target/ppc/cpu.h
> > +++ b/target/ppc/cpu.h
> > @@ -1406,6 +1406,13 @@ struct CPUArchState {
> >       uint64_t pmu_base_time;
> >   };
> >   
> > +#define PPC_CPU_HAS_CORE_SIBLINGS(cs)                           \
> > +    (cs->nr_threads > 1)
> > +
> > +#define PPC_CPU_HAS_LPAR_SIBLINGS(cs)                           \
> > +    ((POWERPC_CPU(cs)->env.flags & POWERPC_FLAG_SMT_1LPAR) &&   \
> > +     PPC_CPU_HAS_CORE_SIBLINGS(cs))
> > +
> >   #define _CORE_ID(cs)                                            \
> >       (POWERPC_CPU(cs)->env.core_index)
> >   
> > diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
> > index ae483e20c4..e71ee008ed 100644
> > --- a/target/ppc/cpu_init.c
> > +++ b/target/ppc/cpu_init.c
> > @@ -6975,7 +6975,7 @@ static void ppc_cpu_realize(DeviceState *dev, Error **errp)
> >   
> >       pcc->parent_realize(dev, errp);
> >   
> > -    if (env_cpu(env)->nr_threads > 1) {
> > +    if (PPC_CPU_HAS_CORE_SIBLINGS(cs)) {
> >           env->flags |= POWERPC_FLAG_SMT;
> >       }
> >   
> > diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c
> > index 0cd542675f..fd45da0f2b 100644
> > --- a/target/ppc/excp_helper.c
> > +++ b/target/ppc/excp_helper.c
> > @@ -3029,7 +3029,7 @@ void helper_book3s_msgsnd(CPUPPCState *env, target_ulong rb)
> >           brdcast = true;
> >       }
> >   
> > -    if (cs->nr_threads == 1 || !brdcast) {
> > +    if (!PPC_CPU_HAS_CORE_SIBLINGS(cs) || !brdcast) {
>
> Since there are multiple usage of above macro in negation below as well, 
> we may probably want to introduce another macro PPC_CPU_HAS_SINGLE_CORE

Ah, you mean SINGLE_THREAD. Yes it would read a bit better.

Thanks,
Nick

> which checks only for nr_threads == 1. Anyways,
>
> Reviewed-by: Harsh Prateek Bora <harshpb@linux.ibm.com>
>
>
> >           ppc_set_irq(cpu, PPC_INTERRUPT_HDOORBELL, 1);
> >           return;
> >       }
> > @@ -3067,21 +3067,19 @@ void helper_book3s_msgsndp(CPUPPCState *env, target_ulong rb)
> >       CPUState *cs = env_cpu(env);
> >       PowerPCCPU *cpu = env_archcpu(env);
> >       CPUState *ccs;
> > -    uint32_t nr_threads = cs->nr_threads;
> >       int ttir = rb & PPC_BITMASK(57, 63);
> >   
> >       helper_hfscr_facility_check(env, HFSCR_MSGP, "msgsndp", HFSCR_IC_MSGP);
> >   
> > -    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> > -        nr_threads = 1; /* msgsndp behaves as 1-thread in LPAR-per-thread mode*/
> > -    }
> > -
> > -    if (!dbell_type_server(rb) || ttir >= nr_threads) {
> > +    if (!dbell_type_server(rb)) {
> >           return;
> >       }
> >   
> > -    if (nr_threads == 1) {
> > -        ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, 1);
> > +    /* msgsndp behaves as 1-thread in LPAR-per-thread mode*/
> > +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
> > +        if (ttir == 0) {
> > +            ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, 1);
> > +        }
> >           return;
> >       }
> >   
> > diff --git a/target/ppc/misc_helper.c b/target/ppc/misc_helper.c
> > index 46ba3a5584..598c956cdd 100644
> > --- a/target/ppc/misc_helper.c
> > +++ b/target/ppc/misc_helper.c
> > @@ -49,9 +49,8 @@ void helper_spr_core_write_generic(CPUPPCState *env, uint32_t sprn,
> >   {
> >       CPUState *cs = env_cpu(env);
> >       CPUState *ccs;
> > -    uint32_t nr_threads = cs->nr_threads;
> >   
> > -    if (nr_threads == 1) {
> > +    if (!PPC_CPU_HAS_CORE_SIBLINGS(cs)) {
> >           env->spr[sprn] = val;
> >           return;
> >       }
> > @@ -196,7 +195,7 @@ void helper_store_ptcr(CPUPPCState *env, target_ulong val)
> >               return;
> >           }
> >   
> > -        if (cs->nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> > +        if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
> >               env->spr[SPR_PTCR] = val;
> >               tlb_flush(cs);
> >           } else {
> > @@ -243,16 +242,12 @@ target_ulong helper_load_dpdes(CPUPPCState *env)
> >   {
> >       CPUState *cs = env_cpu(env);
> >       CPUState *ccs;
> > -    uint32_t nr_threads = cs->nr_threads;
> >       target_ulong dpdes = 0;
> >   
> >       helper_hfscr_facility_check(env, HFSCR_MSGP, "load DPDES", HFSCR_IC_MSGP);
> >   
> > -    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> > -        nr_threads = 1; /* DPDES behaves as 1-thread in LPAR-per-thread mode */
> > -    }
> > -
> > -    if (nr_threads == 1) {
> > +    /* DPDES behaves as 1-thread in LPAR-per-thread mode */
> > +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
> >           if (env->pending_interrupts & PPC_INTERRUPT_DOORBELL) {
> >               dpdes = 1;
> >           }
> > @@ -279,21 +274,11 @@ void helper_store_dpdes(CPUPPCState *env, target_ulong val)
> >       PowerPCCPU *cpu = env_archcpu(env);
> >       CPUState *cs = env_cpu(env);
> >       CPUState *ccs;
> > -    uint32_t nr_threads = cs->nr_threads;
> >   
> >       helper_hfscr_facility_check(env, HFSCR_MSGP, "store DPDES", HFSCR_IC_MSGP);
> >   
> > -    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> > -        nr_threads = 1; /* DPDES behaves as 1-thread in LPAR-per-thread mode */
> > -    }
> > -
> > -    if (val & ~(nr_threads - 1)) {
> > -        qemu_log_mask(LOG_GUEST_ERROR, "Invalid DPDES register value "
> > -                      TARGET_FMT_lx"\n", val);
> > -        val &= (nr_threads - 1); /* Ignore the invalid bits */
> > -    }
> > -
> > -    if (nr_threads == 1) {
> > +    /* DPDES behaves as 1-thread in LPAR-per-thread mode */
> > +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
> >           ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, val & 0x1);
> >           return;
> >       }
> > diff --git a/target/ppc/timebase_helper.c b/target/ppc/timebase_helper.c
> > index 788c498d63..abe7b95696 100644
> > --- a/target/ppc/timebase_helper.c
> > +++ b/target/ppc/timebase_helper.c
> > @@ -63,9 +63,8 @@ void helper_store_purr(CPUPPCState *env, target_ulong val)
> >   {
> >       CPUState *cs = env_cpu(env);
> >       CPUState *ccs;
> > -    uint32_t nr_threads = cs->nr_threads;
> >   
> > -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> > +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
> >           cpu_ppc_store_purr(env, val);
> >           return;
> >       }
> > @@ -82,9 +81,8 @@ void helper_store_tbl(CPUPPCState *env, target_ulong val)
> >   {
> >       CPUState *cs = env_cpu(env);
> >       CPUState *ccs;
> > -    uint32_t nr_threads = cs->nr_threads;
> >   
> > -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> > +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
> >           cpu_ppc_store_tbl(env, val);
> >           return;
> >       }
> > @@ -99,9 +97,8 @@ void helper_store_tbu(CPUPPCState *env, target_ulong val)
> >   {
> >       CPUState *cs = env_cpu(env);
> >       CPUState *ccs;
> > -    uint32_t nr_threads = cs->nr_threads;
> >   
> > -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> > +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
> >           cpu_ppc_store_tbu(env, val);
> >           return;
> >       }
> > @@ -141,9 +138,8 @@ void helper_store_hdecr(CPUPPCState *env, target_ulong val)
> >   {
> >       CPUState *cs = env_cpu(env);
> >       CPUState *ccs;
> > -    uint32_t nr_threads = cs->nr_threads;
> >   
> > -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> > +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
> >           cpu_ppc_store_hdecr(env, val);
> >           return;
> >       }
> > @@ -158,9 +154,8 @@ void helper_store_vtb(CPUPPCState *env, target_ulong val)
> >   {
> >       CPUState *cs = env_cpu(env);
> >       CPUState *ccs;
> > -    uint32_t nr_threads = cs->nr_threads;
> >   
> > -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> > +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
> >           cpu_ppc_store_vtb(env, val);
> >           return;
> >       }
> > @@ -175,9 +170,8 @@ void helper_store_tbu40(CPUPPCState *env, target_ulong val)
> >   {
> >       CPUState *cs = env_cpu(env);
> >       CPUState *ccs;
> > -    uint32_t nr_threads = cs->nr_threads;
> >   
> > -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> > +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
> >           cpu_ppc_store_tbu40(env, val);
> >           return;
> >       }
> > @@ -288,7 +282,7 @@ static void write_tfmr(CPUPPCState *env, target_ulong val)
> >   {
> >       CPUState *cs = env_cpu(env);
> >   
> > -    if (cs->nr_threads == 1) {
> > +    if (!PPC_CPU_HAS_CORE_SIBLINGS(cs)) {
> >           env->spr[SPR_TFMR] = val;
> >       } else {
> >           CPUState *ccs;

Re: [RFC PATCH 07/10] target/ppc: Add helpers to check for SMT sibling threads

[Cédric Le Goater]

On 5/26/24 14:26, Nicholas Piggin wrote:
> Add helpers for TCG code to determine if there are SMT siblings
> sharing per-core and per-lpar registers. This simplifies the
> callers and makes SMT register topology simpler to modify with
> later changes.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   target/ppc/cpu.h             |  7 +++++++
>   target/ppc/cpu_init.c        |  2 +-
>   target/ppc/excp_helper.c     | 16 +++++++---------
>   target/ppc/misc_helper.c     | 27 ++++++---------------------
>   target/ppc/timebase_helper.c | 20 +++++++-------------
>   5 files changed, 28 insertions(+), 44 deletions(-)
> 
> diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
> index 9a89083932..8fd6ade471 100644
> --- a/target/ppc/cpu.h
> +++ b/target/ppc/cpu.h
> @@ -1406,6 +1406,13 @@ struct CPUArchState {
>       uint64_t pmu_base_time;
>   };
>   
> +#define PPC_CPU_HAS_CORE_SIBLINGS(cs)                           \
> +    (cs->nr_threads > 1)

        (cs)->nr_threads

> +
> +#define PPC_CPU_HAS_LPAR_SIBLINGS(cs)                           \
> +    ((POWERPC_CPU(cs)->env.flags & POWERPC_FLAG_SMT_1LPAR) &&   \
> +     PPC_CPU_HAS_CORE_SIBLINGS(cs))
> +

I tend to prefer static inline when things get complex.

The rest looks good.

Thanks,

C.



>   #define _CORE_ID(cs)                                            \
>       (POWERPC_CPU(cs)->env.core_index)
>   
> diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
> index ae483e20c4..e71ee008ed 100644
> --- a/target/ppc/cpu_init.c
> +++ b/target/ppc/cpu_init.c
> @@ -6975,7 +6975,7 @@ static void ppc_cpu_realize(DeviceState *dev, Error **errp)
>   
>       pcc->parent_realize(dev, errp);
>   
> -    if (env_cpu(env)->nr_threads > 1) {
> +    if (PPC_CPU_HAS_CORE_SIBLINGS(cs)) {
>           env->flags |= POWERPC_FLAG_SMT;
>       }
>   
> diff --git a/target/ppc/excp_helper.c b/target/ppc/excp_helper.c
> index 0cd542675f..fd45da0f2b 100644
> --- a/target/ppc/excp_helper.c
> +++ b/target/ppc/excp_helper.c
> @@ -3029,7 +3029,7 @@ void helper_book3s_msgsnd(CPUPPCState *env, target_ulong rb)
>           brdcast = true;
>       }
>   
> -    if (cs->nr_threads == 1 || !brdcast) {
> +    if (!PPC_CPU_HAS_CORE_SIBLINGS(cs) || !brdcast) {
>           ppc_set_irq(cpu, PPC_INTERRUPT_HDOORBELL, 1);
>           return;
>       }
> @@ -3067,21 +3067,19 @@ void helper_book3s_msgsndp(CPUPPCState *env, target_ulong rb)
>       CPUState *cs = env_cpu(env);
>       PowerPCCPU *cpu = env_archcpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>       int ttir = rb & PPC_BITMASK(57, 63);
>   
>       helper_hfscr_facility_check(env, HFSCR_MSGP, "msgsndp", HFSCR_IC_MSGP);
>   
> -    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> -        nr_threads = 1; /* msgsndp behaves as 1-thread in LPAR-per-thread mode*/
> -    }
> -
> -    if (!dbell_type_server(rb) || ttir >= nr_threads) {
> +    if (!dbell_type_server(rb)) {
>           return;
>       }
>   
> -    if (nr_threads == 1) {
> -        ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, 1);
> +    /* msgsndp behaves as 1-thread in LPAR-per-thread mode*/
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
> +        if (ttir == 0) {
> +            ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, 1);
> +        }
>           return;
>       }
>   
> diff --git a/target/ppc/misc_helper.c b/target/ppc/misc_helper.c
> index 46ba3a5584..598c956cdd 100644
> --- a/target/ppc/misc_helper.c
> +++ b/target/ppc/misc_helper.c
> @@ -49,9 +49,8 @@ void helper_spr_core_write_generic(CPUPPCState *env, uint32_t sprn,
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
> -    if (nr_threads == 1) {
> +    if (!PPC_CPU_HAS_CORE_SIBLINGS(cs)) {
>           env->spr[sprn] = val;
>           return;
>       }
> @@ -196,7 +195,7 @@ void helper_store_ptcr(CPUPPCState *env, target_ulong val)
>               return;
>           }
>   
> -        if (cs->nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> +        if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>               env->spr[SPR_PTCR] = val;
>               tlb_flush(cs);
>           } else {
> @@ -243,16 +242,12 @@ target_ulong helper_load_dpdes(CPUPPCState *env)
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>       target_ulong dpdes = 0;
>   
>       helper_hfscr_facility_check(env, HFSCR_MSGP, "load DPDES", HFSCR_IC_MSGP);
>   
> -    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> -        nr_threads = 1; /* DPDES behaves as 1-thread in LPAR-per-thread mode */
> -    }
> -
> -    if (nr_threads == 1) {
> +    /* DPDES behaves as 1-thread in LPAR-per-thread mode */
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           if (env->pending_interrupts & PPC_INTERRUPT_DOORBELL) {
>               dpdes = 1;
>           }
> @@ -279,21 +274,11 @@ void helper_store_dpdes(CPUPPCState *env, target_ulong val)
>       PowerPCCPU *cpu = env_archcpu(env);
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
>       helper_hfscr_facility_check(env, HFSCR_MSGP, "store DPDES", HFSCR_IC_MSGP);
>   
> -    if (!(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> -        nr_threads = 1; /* DPDES behaves as 1-thread in LPAR-per-thread mode */
> -    }
> -
> -    if (val & ~(nr_threads - 1)) {
> -        qemu_log_mask(LOG_GUEST_ERROR, "Invalid DPDES register value "
> -                      TARGET_FMT_lx"\n", val);
> -        val &= (nr_threads - 1); /* Ignore the invalid bits */
> -    }
> -
> -    if (nr_threads == 1) {
> +    /* DPDES behaves as 1-thread in LPAR-per-thread mode */
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           ppc_set_irq(cpu, PPC_INTERRUPT_DOORBELL, val & 0x1);
>           return;
>       }
> diff --git a/target/ppc/timebase_helper.c b/target/ppc/timebase_helper.c
> index 788c498d63..abe7b95696 100644
> --- a/target/ppc/timebase_helper.c
> +++ b/target/ppc/timebase_helper.c
> @@ -63,9 +63,8 @@ void helper_store_purr(CPUPPCState *env, target_ulong val)
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
> -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           cpu_ppc_store_purr(env, val);
>           return;
>       }
> @@ -82,9 +81,8 @@ void helper_store_tbl(CPUPPCState *env, target_ulong val)
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
> -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           cpu_ppc_store_tbl(env, val);
>           return;
>       }
> @@ -99,9 +97,8 @@ void helper_store_tbu(CPUPPCState *env, target_ulong val)
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
> -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           cpu_ppc_store_tbu(env, val);
>           return;
>       }
> @@ -141,9 +138,8 @@ void helper_store_hdecr(CPUPPCState *env, target_ulong val)
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
> -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           cpu_ppc_store_hdecr(env, val);
>           return;
>       }
> @@ -158,9 +154,8 @@ void helper_store_vtb(CPUPPCState *env, target_ulong val)
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
> -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           cpu_ppc_store_vtb(env, val);
>           return;
>       }
> @@ -175,9 +170,8 @@ void helper_store_tbu40(CPUPPCState *env, target_ulong val)
>   {
>       CPUState *cs = env_cpu(env);
>       CPUState *ccs;
> -    uint32_t nr_threads = cs->nr_threads;
>   
> -    if (nr_threads == 1 || !(env->flags & POWERPC_FLAG_SMT_1LPAR)) {
> +    if (!PPC_CPU_HAS_LPAR_SIBLINGS(cs)) {
>           cpu_ppc_store_tbu40(env, val);
>           return;
>       }
> @@ -288,7 +282,7 @@ static void write_tfmr(CPUPPCState *env, target_ulong val)
>   {
>       CPUState *cs = env_cpu(env);
>   
> -    if (cs->nr_threads == 1) {
> +    if (!PPC_CPU_HAS_CORE_SIBLINGS(cs)) {
>           env->spr[SPR_TFMR] = val;
>       } else {
>           CPUState *ccs;

Re: [RFC PATCH 07/10] target/ppc: Add helpers to check for SMT sibling threads

[Nicholas Piggin]

On Wed May 29, 2024 at 4:34 PM AEST, Cédric Le Goater wrote:
> On 5/26/24 14:26, Nicholas Piggin wrote:
> > Add helpers for TCG code to determine if there are SMT siblings
> > sharing per-core and per-lpar registers. This simplifies the
> > callers and makes SMT register topology simpler to modify with
> > later changes.
> > 
> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> > ---
> >   target/ppc/cpu.h             |  7 +++++++
> >   target/ppc/cpu_init.c        |  2 +-
> >   target/ppc/excp_helper.c     | 16 +++++++---------
> >   target/ppc/misc_helper.c     | 27 ++++++---------------------
> >   target/ppc/timebase_helper.c | 20 +++++++-------------
> >   5 files changed, 28 insertions(+), 44 deletions(-)
> > 
> > diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
> > index 9a89083932..8fd6ade471 100644
> > --- a/target/ppc/cpu.h
> > +++ b/target/ppc/cpu.h
> > @@ -1406,6 +1406,13 @@ struct CPUArchState {
> >       uint64_t pmu_base_time;
> >   };
> >   
> > +#define PPC_CPU_HAS_CORE_SIBLINGS(cs)                           \
> > +    (cs->nr_threads > 1)
>
>         (cs)->nr_threads
>
> > +
> > +#define PPC_CPU_HAS_LPAR_SIBLINGS(cs)                           \
> > +    ((POWERPC_CPU(cs)->env.flags & POWERPC_FLAG_SMT_1LPAR) &&   \
> > +     PPC_CPU_HAS_CORE_SIBLINGS(cs))
> > +
>
> I tend to prefer static inline when things get complex.

Yeah agree it has become too much.

Thanks,
Nick

[RFC PATCH 08/10] ppc/pnv: Invert the design for big-core machine modelling

[Nicholas Piggin]

POWER9 and POWER10 machines come in two variants, "big-core" and
"small-core".

Big core machines are SMT8 from the software point of view, but in the
low level platform topology ("xscom registers and pervasive
addressing"), these look more like a pair of small cores ganged
together.

Presently, the way this is modelled is to create an SMT8 PnvCore and
add special cases to xscom and pervasive for big-core mode. This is
becoming too complicated to manage as more of the machine is modelled.
The better approach looks like the inverse, which is creating 2xPnvCore
ganging them together to look like an SMT8 core in TCG. The TCG SMT code
is quite simple to do that, and then the xscom and pervasive modelling
does not need to differentiate big and small core modes for the most
part.

device-tree building does need a special case to only build one
CPU node for each big-core because that's what the firmware expects.
And so does a special case workaround in the ChipTOD model.

A big-core machine option is added for powernv9 and 10 machines.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 include/hw/ppc/pnv.h         |   3 +
 include/hw/ppc/pnv_core.h    |   8 ++
 target/ppc/cpu.h             |   4 +-
 hw/ppc/pnv.c                 | 183 ++++++++++++++++++++++++++++-------
 hw/ppc/pnv_core.c            |  20 +++-
 hw/ppc/spapr_cpu_core.c      |   6 +-
 target/ppc/misc_helper.c     |   6 +-
 target/ppc/timebase_helper.c |   9 ++
 8 files changed, 197 insertions(+), 42 deletions(-)

diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
index 476b136146..93ecb062b4 100644
--- a/include/hw/ppc/pnv.h
+++ b/include/hw/ppc/pnv.h
@@ -100,6 +100,9 @@ struct PnvMachineState {
     PnvPnor      *pnor;
 
     hwaddr       fw_load_addr;
+
+    bool         big_core;
+    bool         big_core_tbst_quirk;
 };
 
 PnvChip *pnv_get_chip(PnvMachineState *pnv, uint32_t chip_id);
diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
index 21297262c1..39f8f33e6c 100644
--- a/include/hw/ppc/pnv_core.h
+++ b/include/hw/ppc/pnv_core.h
@@ -27,6 +27,13 @@
 
 /* ChipTOD and TimeBase State Machine */
 struct pnv_tod_tbst {
+    /*
+     * POWER10 DD2.0 - big core TFMR drives the state machine on the even
+     * small core. Skiboot has a workaround that targets the even small core
+     * for CHIPTOD_TO_TB ops.
+     */
+    bool big_core_quirk;
+
     int tb_ready_for_tod; /* core TB ready to receive TOD from chiptod */
     int tod_sent_to_tb;   /* chiptod sent TOD to the core TB */
 
@@ -49,6 +56,7 @@ struct PnvCore {
 
     /*< public >*/
     PowerPCCPU **threads;
+    bool big_core;
     uint32_t pir;
     uint32_t hwid;
     uint64_t hrmor;
diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
index 8fd6ade471..de15e38af8 100644
--- a/target/ppc/cpu.h
+++ b/target/ppc/cpu.h
@@ -1248,6 +1248,7 @@ struct CPUArchState {
     int access_type;
 
     /* For SMT processors */
+    int has_smt_siblings;
     int core_index;
 
 #if !defined(CONFIG_USER_ONLY)
@@ -1276,7 +1277,6 @@ struct CPUArchState {
     uint32_t tlb_need_flush; /* Delayed flush needed */
 #define TLB_NEED_LOCAL_FLUSH   0x1
 #define TLB_NEED_GLOBAL_FLUSH  0x2
-
 #endif
 
     /* Other registers */
@@ -1407,7 +1407,7 @@ struct CPUArchState {
 };
 
 #define PPC_CPU_HAS_CORE_SIBLINGS(cs)                           \
-    (cs->nr_threads > 1)
+    (POWERPC_CPU(cs)->env.has_smt_siblings)
 
 #define PPC_CPU_HAS_LPAR_SIBLINGS(cs)                           \
     ((POWERPC_CPU(cs)->env.flags & POWERPC_FLAG_SMT_1LPAR) &&   \
diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
index 7d062ec16c..5364c55bbb 100644
--- a/hw/ppc/pnv.c
+++ b/hw/ppc/pnv.c
@@ -142,7 +142,7 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
     CPUPPCState *env = &cpu->env;
     PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cs);
     PnvChipClass *pnv_cc = PNV_CHIP_GET_CLASS(chip);
-    g_autofree uint32_t *servers_prop = g_new(uint32_t, smt_threads);
+    uint32_t *servers_prop;
     int i;
     uint32_t pir, tir;
     uint32_t segs[] = {cpu_to_be32(28), cpu_to_be32(40),
@@ -157,6 +157,14 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
 
     pnv_cc->processor_id(chip, pc->hwid, 0, &pir, &tir);
 
+    /* Only one DT node per (big) core */
+    if (tir != 0) {
+        g_assert(pc->big_core);
+        g_assert(tir == 1);
+        g_assert(pc->hwid & 1);
+        return -1;
+    }
+
     nodename = g_strdup_printf("%s@%x", dc->fw_name, pir);
     offset = fdt_add_subnode(fdt, cpus_offset, nodename);
     _FDT(offset);
@@ -236,12 +244,28 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
     }
 
     /* Build interrupt servers properties */
-    for (i = 0; i < smt_threads; i++) {
-        pnv_cc->processor_id(chip, pc->hwid, i, &pir, &tir);
-        servers_prop[i] = cpu_to_be32(pir);
+    if (pc->big_core) {
+        servers_prop = g_new(uint32_t, smt_threads * 2);
+        for (i = 0; i < smt_threads; i++) {
+            pnv_cc->processor_id(chip, pc->hwid, i, &pir, &tir);
+            servers_prop[i * 2] = cpu_to_be32(pir);
+
+            pnv_cc->processor_id(chip, pc->hwid + 1, i, &pir, &tir);
+            servers_prop[i * 2 + 1] = cpu_to_be32(pir);
+        }
+        _FDT((fdt_setprop(fdt, offset, "ibm,ppc-interrupt-server#s",
+                          servers_prop, sizeof(*servers_prop) * smt_threads
+                                        * 2)));
+    } else {
+        servers_prop = g_new(uint32_t, smt_threads);
+        for (i = 0; i < smt_threads; i++) {
+            pnv_cc->processor_id(chip, pc->hwid, i, &pir, &tir);
+            servers_prop[i] = cpu_to_be32(pir);
+        }
+        _FDT((fdt_setprop(fdt, offset, "ibm,ppc-interrupt-server#s",
+                          servers_prop, sizeof(*servers_prop) * smt_threads)));
     }
-    _FDT((fdt_setprop(fdt, offset, "ibm,ppc-interrupt-server#s",
-                       servers_prop, sizeof(*servers_prop) * smt_threads)));
+    g_free(servers_prop);
 
     return offset;
 }
@@ -330,6 +354,8 @@ static void pnv_chip_power8_dt_populate(PnvChip *chip, void *fdt)
         int offset;
 
         offset = pnv_dt_core(chip, pnv_core, fdt);
+        if (offset == -1)
+            continue;
 
         _FDT((fdt_setprop(fdt, offset, "ibm,pa-features",
                            pa_features_207, sizeof(pa_features_207))));
@@ -387,6 +413,8 @@ static void pnv_chip_power9_dt_populate(PnvChip *chip, void *fdt)
         int offset;
 
         offset = pnv_dt_core(chip, pnv_core, fdt);
+        if (offset == -1)
+            continue;
 
         _FDT((fdt_setprop(fdt, offset, "ibm,pa-features",
                            pa_features_300, sizeof(pa_features_300))));
@@ -448,6 +476,8 @@ static void pnv_chip_power10_dt_populate(PnvChip *chip, void *fdt)
         int offset;
 
         offset = pnv_dt_core(chip, pnv_core, fdt);
+        if (offset == -1)
+            continue;
 
         _FDT((fdt_setprop(fdt, offset, "ibm,pa-features",
                            pa_features_31, sizeof(pa_features_31))));
@@ -981,11 +1011,12 @@ static void pnv_init(MachineState *machine)
         machine->smp.max_cpus / (machine->smp.cores * machine->smp.threads);
 
     if (!is_power_of_2(machine->smp.threads)) {
-        error_report("Cannot support %d threads/core on a powernv"
+        error_report("Cannot support %d threads/core on a powernv "
                      "machine because it must be a power of 2",
                      machine->smp.threads);
         exit(1);
     }
+
     /*
      * TODO: should we decide on how many chips we can create based
      * on #cores and Venice vs. Murano vs. Naples chip type etc...,
@@ -1088,10 +1119,37 @@ static void pnv_power8_init(MachineState *machine)
 
 static void pnv_power9_init(MachineState *machine)
 {
-    if (machine->smp.threads > 8) {
-        error_report("Cannot support more than 8 threads/core "
-                     "on a powernv9/10 machine");
-        exit(1);
+    PnvMachineState *pnv = PNV_MACHINE(machine);
+
+    if (pnv->big_core) {
+        if (machine->smp.threads > 8) {
+            error_report("Cannot support more than 8 threads/core "
+                         "on a powernv9/10  machine");
+            exit(1);
+        }
+        if (machine->smp.threads % 2 == 1) {
+            error_report("Cannot support %d threads with big-core option "
+                         "because it must be an even number",
+                         machine->smp.threads);
+            exit(1);
+        }
+
+        /*
+         * powernv models PnvCore as a SMT4 core. Big-core requires 2xPnvCore
+         * per core, so adjust topology here. pnv_dt_core() processor
+         * device-tree and TCG SMT code make the 2 cores appear as one big core
+         * from software point of view. pnv pervasive models and xscoms tend to
+         * see the big core as 2 small core halves.
+         */
+        machine->smp.cores *= 2;
+        machine->smp.threads /= 2;
+    } else {
+        if (machine->smp.threads > 4) {
+            error_report("Cannot support more than 4 threads/core "
+                         "on a powernv9/10 small-core machine "
+                         "(use big-core=on for 8 threads per core)");
+            exit(1);
+        }
     }
 
     pnv_init(machine);
@@ -1099,6 +1157,8 @@ static void pnv_power9_init(MachineState *machine)
 
 static void pnv_power10_init(MachineState *machine)
 {
+    PnvMachineState *pnv = PNV_MACHINE(machine);
+    pnv->big_core_tbst_quirk = true;
     pnv_power9_init(machine);
 }
 
@@ -1169,9 +1229,15 @@ static void pnv_processor_id_p9(PnvChip *chip,
                                 uint32_t core_id, uint32_t thread_id,
                                 uint32_t *pir, uint32_t *tir)
 {
-    if (chip->nr_threads == 8) {
-        *pir = (chip->chip_id << 8) | ((thread_id & 1) << 2) | (core_id << 3) |
-               (thread_id >> 1);
+    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
+
+    if (pnv->big_core) {
+        /* Big-core interleaves thread ID between small-cores */
+        thread_id <<= 1;
+        thread_id |= core_id & 1;
+        core_id >>= 1;
+
+        *pir = (chip->chip_id << 8) | (core_id << 3) | thread_id;
     } else {
         *pir = (chip->chip_id << 8) | (core_id << 2) | thread_id;
     }
@@ -1193,9 +1259,15 @@ static void pnv_processor_id_p10(PnvChip *chip,
                                 uint32_t core_id, uint32_t thread_id,
                                 uint32_t *pir, uint32_t *tir)
 {
-    if (chip->nr_threads == 8) {
-        *pir = (chip->chip_id << 8) | ((core_id / 4) << 4) |
-                ((core_id % 2) << 3) | thread_id;
+    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
+
+    if (pnv->big_core) {
+        /* Big-core interleaves thread ID between small-cores */
+        thread_id <<= 1;
+        thread_id |= core_id & 1;
+        core_id >>= 1;
+
+        *pir = (chip->chip_id << 8) | (core_id << 3) | thread_id;
     } else {
         *pir = (chip->chip_id << 8) | (core_id << 2) | thread_id;
     }
@@ -2146,7 +2218,8 @@ static void pnv_chip_power10_class_init(ObjectClass *klass, void *data)
                                     &k->parent_realize);
 }
 
-static void pnv_chip_core_sanitize(PnvChip *chip, Error **errp)
+static void pnv_chip_core_sanitize(PnvMachineState *pnv, PnvChip *chip,
+                                   Error **errp)
 {
     PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip);
     int cores_max;
@@ -2167,6 +2240,17 @@ static void pnv_chip_core_sanitize(PnvChip *chip, Error **errp)
     }
     chip->cores_mask &= pcc->cores_mask;
 
+    /* Ensure small-cores a paired up in big-core mode */
+    if (pnv->big_core) {
+        uint64_t even_cores = chip->cores_mask & 0x5555555555555555ULL;
+        uint64_t odd_cores = chip->cores_mask & 0xaaaaaaaaaaaaaaaaULL;
+
+        if (even_cores ^ (odd_cores >> 1)) {
+            error_setg(errp, "warning: unpaired cores in big-core mode !");
+            return;
+        }
+    }
+
     /* now that we have a sane layout, let check the number of cores */
     cores_max = ctpop64(chip->cores_mask);
     if (chip->nr_cores > cores_max) {
@@ -2190,7 +2274,7 @@ static void pnv_chip_core_realize(PnvChip *chip, Error **errp)
     }
 
     /* Cores */
-    pnv_chip_core_sanitize(chip, &error);
+    pnv_chip_core_sanitize(pnv, chip, &error);
     if (error) {
         error_propagate(errp, error);
         return;
@@ -2445,6 +2529,34 @@ static int pnv10_xive_match_nvt(XiveFabric *xfb, uint8_t format,
     return total_count;
 }
 
+static bool pnv_machine_get_big_core(Object *obj, Error **errp)
+{
+    PnvMachineState *pnv = PNV_MACHINE(obj);
+    return pnv->big_core;
+}
+
+static void pnv_machine_set_big_core(Object *obj, bool value, Error **errp)
+{
+    PnvMachineState *pnv = PNV_MACHINE(obj);
+    pnv->big_core = value;
+}
+
+static bool pnv_machine_get_hb(Object *obj, Error **errp)
+{
+    PnvMachineState *pnv = PNV_MACHINE(obj);
+
+    return !!pnv->fw_load_addr;
+}
+
+static void pnv_machine_set_hb(Object *obj, bool value, Error **errp)
+{
+    PnvMachineState *pnv = PNV_MACHINE(obj);
+
+    if (value) {
+        pnv->fw_load_addr = 0x8000000;
+    }
+}
+
 static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
 {
     MachineClass *mc = MACHINE_CLASS(oc);
@@ -2496,6 +2608,12 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
     pmc->dt_power_mgt = pnv_dt_power_mgt;
 
     machine_class_allow_dynamic_sysbus_dev(mc, TYPE_PNV_PHB);
+
+    object_class_property_add_bool(oc, "big-core",
+                                   pnv_machine_get_big_core,
+                                   pnv_machine_set_big_core);
+    object_class_property_set_description(oc, "big-core",
+                              "Use big-core (aka fused-core) mode");
 }
 
 static void pnv_machine_p10_common_class_init(ObjectClass *oc, void *data)
@@ -2531,6 +2649,17 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
 
     pnv_machine_p10_common_class_init(oc, data);
     mc->desc = "IBM PowerNV (Non-Virtualized) POWER10";
+
+    /*
+     * This is the parent of POWER10 Rainier class, so properies go here
+     * rather than common init (which would add them to both parent and
+     * child which is invalid).
+     */
+    object_class_property_add_bool(oc, "big-core",
+                                   pnv_machine_get_big_core,
+                                   pnv_machine_set_big_core);
+    object_class_property_set_description(oc, "big-core",
+                              "Use big-core (aka fused-core) mode");
 }
 
 static void pnv_machine_p10_rainier_class_init(ObjectClass *oc, void *data)
@@ -2543,22 +2672,6 @@ static void pnv_machine_p10_rainier_class_init(ObjectClass *oc, void *data)
     pmc->i2c_init = pnv_rainier_i2c_init;
 }
 
-static bool pnv_machine_get_hb(Object *obj, Error **errp)
-{
-    PnvMachineState *pnv = PNV_MACHINE(obj);
-
-    return !!pnv->fw_load_addr;
-}
-
-static void pnv_machine_set_hb(Object *obj, bool value, Error **errp)
-{
-    PnvMachineState *pnv = PNV_MACHINE(obj);
-
-    if (value) {
-        pnv->fw_load_addr = 0x8000000;
-    }
-}
-
 static void pnv_cpu_do_nmi_on_cpu(CPUState *cs, run_on_cpu_data arg)
 {
     CPUPPCState *env = cpu_env(cs);
diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
index 0f61aabb77..10417d92ae 100644
--- a/hw/ppc/pnv_core.c
+++ b/hw/ppc/pnv_core.c
@@ -58,6 +58,9 @@ static void pnv_core_cpu_reset(PnvCore *pc, PowerPCCPU *cpu)
     env->nip = 0x10;
     env->msr |= MSR_HVB; /* Hypervisor mode */
     env->spr[SPR_HRMOR] = pc->hrmor;
+    if (pc->big_core) {
+        env->spr[SPR_PVR] &= ~PPC_BIT(51); /* Clear "small core" bit */
+    }
     hreg_compute_hflags(env);
     ppc_maybe_interrupt(env);
 
@@ -252,7 +255,12 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
     pir_spr->default_value = pir;
     tir_spr->default_value = tir;
 
-    env->core_index = core_hwid;
+    if (pc->big_core) {
+        /* 2 "small cores" get the same core index for SMT operations */
+        env->core_index = core_hwid >> 1;
+    } else {
+        env->core_index = core_hwid;
+    }
 
     /* Set time-base frequency to 512 MHz */
     cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
@@ -271,6 +279,7 @@ static void pnv_core_reset(void *dev)
 
 static void pnv_core_realize(DeviceState *dev, Error **errp)
 {
+    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
     PnvCore *pc = PNV_CORE(OBJECT(dev));
     PnvCoreClass *pcc = PNV_CORE_GET_CLASS(pc);
     CPUCore *cc = CPU_CORE(OBJECT(dev));
@@ -282,14 +291,19 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
 
     assert(pc->chip);
 
+    pc->big_core = pnv->big_core;
+    pc->pnv_tod_tbst.big_core_quirk = pnv->big_core_tbst_quirk;;
+
     pc->threads = g_new(PowerPCCPU *, cc->nr_threads);
     for (i = 0; i < cc->nr_threads; i++) {
         PowerPCCPU *cpu;
 
         obj = object_new(typename);
         cpu = POWERPC_CPU(obj);
-
-        pc->threads[i] = POWERPC_CPU(obj);
+        pc->threads[i] = cpu;
+        if (cc->nr_threads > 1) {
+            cpu->env.has_smt_siblings = true;
+        }
 
         snprintf(name, sizeof(name), "thread[%d]", i);
         object_property_add_child(OBJECT(pc), name, obj);
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
index 059d372c8a..05195527a5 100644
--- a/hw/ppc/spapr_cpu_core.c
+++ b/hw/ppc/spapr_cpu_core.c
@@ -312,12 +312,16 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
      * and the rest are explicitly started up by the guest using an RTAS call.
      */
     qdev_prop_set_bit(DEVICE(obj), "start-powered-off", true);
-    env->core_index = cc->core_id;
     cs->cpu_index = cc->core_id + i;
     if (!spapr_set_vcpu_id(cpu, cs->cpu_index, errp)) {
         return NULL;
     }
 
+    env->core_index = cc->core_id;
+    if (cs->nr_threads > 1) {
+        env->has_smt_siblings = true;
+    }
+
     cpu->node_id = sc->node_id;
 
     id = g_strdup_printf("thread[%d]", i);
diff --git a/target/ppc/misc_helper.c b/target/ppc/misc_helper.c
index 598c956cdd..1d56d8f85f 100644
--- a/target/ppc/misc_helper.c
+++ b/target/ppc/misc_helper.c
@@ -323,7 +323,11 @@ target_ulong helper_load_sprd(CPUPPCState *env)
              * prefer to get unimplemented message on POWER10 if it were
              * used.
              */
-            return 0;
+            if (pc->big_core) {
+                return PPC_BIT(63);
+            } else {
+                return 0;
+            }
         }
         /* fallthru */
     default:
diff --git a/target/ppc/timebase_helper.c b/target/ppc/timebase_helper.c
index abe7b95696..6163d4ecb3 100644
--- a/target/ppc/timebase_helper.c
+++ b/target/ppc/timebase_helper.c
@@ -20,6 +20,7 @@
 #include "cpu.h"
 #include "hw/ppc/ppc.h"
 #include "hw/ppc/pnv_core.h"
+#include "hw/ppc/pnv_chip.h"
 #include "exec/helper-proto.h"
 #include "exec/exec-all.h"
 #include "qemu/log.h"
@@ -297,6 +298,14 @@ static struct pnv_tod_tbst *cpu_get_tbst(PowerPCCPU *cpu)
 {
     PnvCore *pc = pnv_cpu_state(cpu)->core;
 
+    if (pc->big_core && pc->pnv_tod_tbst.big_core_quirk) {
+        /* Must operate on the even small core */
+        int core_id = CPU_CORE(pc)->core_id;
+        if (core_id & 1) {
+            pc = pc->chip->cores[core_id & ~1];
+        }
+    }
+
     return &pc->pnv_tod_tbst;
 }
 
-- 
2.43.0

Re: [RFC PATCH 08/10] ppc/pnv: Invert the design for big-core machine modelling

[Cédric Le Goater]

On 5/26/24 14:26, Nicholas Piggin wrote:
> POWER9 and POWER10 machines come in two variants, "big-core" and
> "small-core".
> 
> Big core machines are SMT8 from the software point of view, but in the
> low level platform topology ("xscom registers and pervasive
> addressing"), these look more like a pair of small cores ganged
> together.
> 
> Presently, the way this is modelled is to create an SMT8 PnvCore and
> add special cases to xscom and pervasive for big-core mode. This is
> becoming too complicated to manage as more of the machine is modelled.
> The better approach looks like the inverse, which is creating 2xPnvCore
> ganging them together to look like an SMT8 core in TCG. The TCG SMT code
> is quite simple to do that, and then the xscom and pervasive modelling
> does not need to differentiate big and small core modes for the most
> part.
> 
> device-tree building does need a special case to only build one
> CPU node for each big-core because that's what the firmware expects.
> And so does a special case workaround in the ChipTOD model.
> 
> A big-core machine option is added for powernv9 and 10 machines.

That's another patch.

It is difficult to follow all the changes. I think this patch
needs further splitting.

> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   include/hw/ppc/pnv.h         |   3 +
>   include/hw/ppc/pnv_core.h    |   8 ++
>   target/ppc/cpu.h             |   4 +-
>   hw/ppc/pnv.c                 | 183 ++++++++++++++++++++++++++++-------
>   hw/ppc/pnv_core.c            |  20 +++-
>   hw/ppc/spapr_cpu_core.c      |   6 +-
>   target/ppc/misc_helper.c     |   6 +-
>   target/ppc/timebase_helper.c |   9 ++
>   8 files changed, 197 insertions(+), 42 deletions(-)
> 
> diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
> index 476b136146..93ecb062b4 100644
> --- a/include/hw/ppc/pnv.h
> +++ b/include/hw/ppc/pnv.h
> @@ -100,6 +100,9 @@ struct PnvMachineState {
>       PnvPnor      *pnor;
>   
>       hwaddr       fw_load_addr;
> +
> +    bool         big_core;
> +    bool         big_core_tbst_quirk;

I think the quirk should be introduced in its own patch.

>   };
>   
>   PnvChip *pnv_get_chip(PnvMachineState *pnv, uint32_t chip_id);
> diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
> index 21297262c1..39f8f33e6c 100644
> --- a/include/hw/ppc/pnv_core.h
> +++ b/include/hw/ppc/pnv_core.h
> @@ -27,6 +27,13 @@
>   
>   /* ChipTOD and TimeBase State Machine */
>   struct pnv_tod_tbst {
> +    /*
> +     * POWER10 DD2.0 - big core TFMR drives the state machine on the even
> +     * small core. Skiboot has a workaround that targets the even small core
> +     * for CHIPTOD_TO_TB ops.
> +     */
> +    bool big_core_quirk;
> +
>       int tb_ready_for_tod; /* core TB ready to receive TOD from chiptod */
>       int tod_sent_to_tb;   /* chiptod sent TOD to the core TB */
>   
> @@ -49,6 +56,7 @@ struct PnvCore {
>   
>       /*< public >*/
>       PowerPCCPU **threads;
> +    bool big_core;
>       uint32_t pir;
>       uint32_t hwid;
>       uint64_t hrmor;
> diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
> index 8fd6ade471..de15e38af8 100644
> --- a/target/ppc/cpu.h
> +++ b/target/ppc/cpu.h
> @@ -1248,6 +1248,7 @@ struct CPUArchState {
>       int access_type;
>   
>       /* For SMT processors */
> +    int has_smt_siblings;
>       int core_index;
>   
>   #if !defined(CONFIG_USER_ONLY)
> @@ -1276,7 +1277,6 @@ struct CPUArchState {
>       uint32_t tlb_need_flush; /* Delayed flush needed */
>   #define TLB_NEED_LOCAL_FLUSH   0x1
>   #define TLB_NEED_GLOBAL_FLUSH  0x2
> -
>   #endif
>   
>       /* Other registers */
> @@ -1407,7 +1407,7 @@ struct CPUArchState {
>   };
>   
>   #define PPC_CPU_HAS_CORE_SIBLINGS(cs)                           \
> -    (cs->nr_threads > 1)
> +    (POWERPC_CPU(cs)->env.has_smt_siblings)
>   
>   #define PPC_CPU_HAS_LPAR_SIBLINGS(cs)                           \
>       ((POWERPC_CPU(cs)->env.flags & POWERPC_FLAG_SMT_1LPAR) &&   \
> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
> index 7d062ec16c..5364c55bbb 100644
> --- a/hw/ppc/pnv.c
> +++ b/hw/ppc/pnv.c
> @@ -142,7 +142,7 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
>       CPUPPCState *env = &cpu->env;
>       PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cs);
>       PnvChipClass *pnv_cc = PNV_CHIP_GET_CLASS(chip);
> -    g_autofree uint32_t *servers_prop = g_new(uint32_t, smt_threads);
> +    uint32_t *servers_prop;
>       int i;
>       uint32_t pir, tir;
>       uint32_t segs[] = {cpu_to_be32(28), cpu_to_be32(40),
> @@ -157,6 +157,14 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
>   
>       pnv_cc->processor_id(chip, pc->hwid, 0, &pir, &tir);
>   
> +    /* Only one DT node per (big) core */
> +    if (tir != 0) {
> +        g_assert(pc->big_core);
> +        g_assert(tir == 1);
> +        g_assert(pc->hwid & 1);
> +        return -1;

return is -1 but it's not an error. right ?

> +    }
> +
>       nodename = g_strdup_printf("%s@%x", dc->fw_name, pir);
>       offset = fdt_add_subnode(fdt, cpus_offset, nodename);
>       _FDT(offset);
> @@ -236,12 +244,28 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
>       }
>   
>       /* Build interrupt servers properties */
> -    for (i = 0; i < smt_threads; i++) {
> -        pnv_cc->processor_id(chip, pc->hwid, i, &pir, &tir);
> -        servers_prop[i] = cpu_to_be32(pir);
> +    if (pc->big_core) {
> +        servers_prop = g_new(uint32_t, smt_threads * 2);
> +        for (i = 0; i < smt_threads; i++) {
> +            pnv_cc->processor_id(chip, pc->hwid, i, &pir, &tir);
> +            servers_prop[i * 2] = cpu_to_be32(pir);
> +
> +            pnv_cc->processor_id(chip, pc->hwid + 1, i, &pir, &tir);
> +            servers_prop[i * 2 + 1] = cpu_to_be32(pir);
> +        }
> +        _FDT((fdt_setprop(fdt, offset, "ibm,ppc-interrupt-server#s",
> +                          servers_prop, sizeof(*servers_prop) * smt_threads
> +                                        * 2)));
> +    } else {
> +        servers_prop = g_new(uint32_t, smt_threads);
> +        for (i = 0; i < smt_threads; i++) {
> +            pnv_cc->processor_id(chip, pc->hwid, i, &pir, &tir);
> +            servers_prop[i] = cpu_to_be32(pir);
> +        }
> +        _FDT((fdt_setprop(fdt, offset, "ibm,ppc-interrupt-server#s",
> +                          servers_prop, sizeof(*servers_prop) * smt_threads)));
>       }
> -    _FDT((fdt_setprop(fdt, offset, "ibm,ppc-interrupt-server#s",
> -                       servers_prop, sizeof(*servers_prop) * smt_threads)));
> +    g_free(servers_prop);
>   
>       return offset;
>   }
> @@ -330,6 +354,8 @@ static void pnv_chip_power8_dt_populate(PnvChip *chip, void *fdt)
>           int offset;
>   
>           offset = pnv_dt_core(chip, pnv_core, fdt);
> +        if (offset == -1)
> +            continue;
>   
>           _FDT((fdt_setprop(fdt, offset, "ibm,pa-features",
>                              pa_features_207, sizeof(pa_features_207))));
> @@ -387,6 +413,8 @@ static void pnv_chip_power9_dt_populate(PnvChip *chip, void *fdt)
>           int offset;
>   
>           offset = pnv_dt_core(chip, pnv_core, fdt);
> +        if (offset == -1)
> +            continue;
>   
>           _FDT((fdt_setprop(fdt, offset, "ibm,pa-features",
>                              pa_features_300, sizeof(pa_features_300))));
> @@ -448,6 +476,8 @@ static void pnv_chip_power10_dt_populate(PnvChip *chip, void *fdt)
>           int offset;
>   
>           offset = pnv_dt_core(chip, pnv_core, fdt);
> +        if (offset == -1)
> +            continue;
>   
>           _FDT((fdt_setprop(fdt, offset, "ibm,pa-features",
>                              pa_features_31, sizeof(pa_features_31))));
> @@ -981,11 +1011,12 @@ static void pnv_init(MachineState *machine)
>           machine->smp.max_cpus / (machine->smp.cores * machine->smp.threads);
>   
>       if (!is_power_of_2(machine->smp.threads)) {
> -        error_report("Cannot support %d threads/core on a powernv"
> +        error_report("Cannot support %d threads/core on a powernv "
>                        "machine because it must be a power of 2",
>                        machine->smp.threads);
>           exit(1);
>       }
> +
>       /*
>        * TODO: should we decide on how many chips we can create based
>        * on #cores and Venice vs. Murano vs. Naples chip type etc...,
> @@ -1088,10 +1119,37 @@ static void pnv_power8_init(MachineState *machine)
>   
>   static void pnv_power9_init(MachineState *machine)
>   {
> -    if (machine->smp.threads > 8) {
> -        error_report("Cannot support more than 8 threads/core "
> -                     "on a powernv9/10 machine");
> -        exit(1);
> +    PnvMachineState *pnv = PNV_MACHINE(machine);
> +
> +    if (pnv->big_core) {

It would be interesting to have a max_smt machine class attribute too.

> +        if (machine->smp.threads > 8) {
> +            error_report("Cannot support more than 8 threads/core "
> +                         "on a powernv9/10  machine");
> +            exit(1);
> +        }
> +        if (machine->smp.threads % 2 == 1) {

is_power_of_2()

> +            error_report("Cannot support %d threads with big-core option "
> +                         "because it must be an even number",
> +                         machine->smp.threads);
> +            exit(1);
> +        }
> +
> +        /*
> +         * powernv models PnvCore as a SMT4 core. Big-core requires 2xPnvCore
> +         * per core, so adjust topology here. pnv_dt_core() processor
> +         * device-tree and TCG SMT code make the 2 cores appear as one big core
> +         * from software point of view. pnv pervasive models and xscoms tend to
> +         * see the big core as 2 small core halves.
> +         */
> +        machine->smp.cores *= 2;
> +        machine->smp.threads /= 2;
> +    } else {
> +        if (machine->smp.threads > 4) {
> +            error_report("Cannot support more than 4 threads/core "
> +                         "on a powernv9/10 small-core machine "
> +                         "(use big-core=on for 8 threads per core)");
> +            exit(1);
> +        }
>       }
>   
>       pnv_init(machine);
> @@ -1099,6 +1157,8 @@ static void pnv_power9_init(MachineState *machine)
>   
>   static void pnv_power10_init(MachineState *machine)
>   {
> +    PnvMachineState *pnv = PNV_MACHINE(machine);
> +    pnv->big_core_tbst_quirk = true;
>       pnv_power9_init(machine);
>   }
>   
> @@ -1169,9 +1229,15 @@ static void pnv_processor_id_p9(PnvChip *chip,
>                                   uint32_t core_id, uint32_t thread_id,
>                                   uint32_t *pir, uint32_t *tir)
>   {
> -    if (chip->nr_threads == 8) {
> -        *pir = (chip->chip_id << 8) | ((thread_id & 1) << 2) | (core_id << 3) |
> -               (thread_id >> 1);
> +    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());

arg. We should avoid these qdev_get_machine() calls. Could big_core be a
chip property instead ?

> +
> +    if (pnv->big_core) {
> +        /* Big-core interleaves thread ID between small-cores */
> +        thread_id <<= 1;
> +        thread_id |= core_id & 1;
> +        core_id >>= 1;
> +
> +        *pir = (chip->chip_id << 8) | (core_id << 3) | thread_id;
>       } else {
>           *pir = (chip->chip_id << 8) | (core_id << 2) | thread_id;
>       }
> @@ -1193,9 +1259,15 @@ static void pnv_processor_id_p10(PnvChip *chip,
>                                   uint32_t core_id, uint32_t thread_id,
>                                   uint32_t *pir, uint32_t *tir)
>   {
> -    if (chip->nr_threads == 8) {
> -        *pir = (chip->chip_id << 8) | ((core_id / 4) << 4) |
> -                ((core_id % 2) << 3) | thread_id;
> +    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
> +
> +    if (pnv->big_core) {
> +        /* Big-core interleaves thread ID between small-cores */
> +        thread_id <<= 1;
> +        thread_id |= core_id & 1;
> +        core_id >>= 1;
> +
> +        *pir = (chip->chip_id << 8) | (core_id << 3) | thread_id;
>       } else {
>           *pir = (chip->chip_id << 8) | (core_id << 2) | thread_id;
>       }
> @@ -2146,7 +2218,8 @@ static void pnv_chip_power10_class_init(ObjectClass *klass, void *data)
>                                       &k->parent_realize);
>   }
>   
> -static void pnv_chip_core_sanitize(PnvChip *chip, Error **errp)
> +static void pnv_chip_core_sanitize(PnvMachineState *pnv, PnvChip *chip,
> +                                   Error **errp)
>   {
>       PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip);
>       int cores_max;
> @@ -2167,6 +2240,17 @@ static void pnv_chip_core_sanitize(PnvChip *chip, Error **errp)
>       }
>       chip->cores_mask &= pcc->cores_mask;
>   
> +    /* Ensure small-cores a paired up in big-core mode */
> +    if (pnv->big_core) {
> +        uint64_t even_cores = chip->cores_mask & 0x5555555555555555ULL;
> +        uint64_t odd_cores = chip->cores_mask & 0xaaaaaaaaaaaaaaaaULL;
> +
> +        if (even_cores ^ (odd_cores >> 1)) {
> +            error_setg(errp, "warning: unpaired cores in big-core mode !");
> +            return;
> +        }
> +    }
> +
>       /* now that we have a sane layout, let check the number of cores */
>       cores_max = ctpop64(chip->cores_mask);
>       if (chip->nr_cores > cores_max) {
> @@ -2190,7 +2274,7 @@ static void pnv_chip_core_realize(PnvChip *chip, Error **errp)
>       }
>   
>       /* Cores */
> -    pnv_chip_core_sanitize(chip, &error);
> +    pnv_chip_core_sanitize(pnv, chip, &error);
>       if (error) {
>           error_propagate(errp, error);
>           return;
> @@ -2445,6 +2529,34 @@ static int pnv10_xive_match_nvt(XiveFabric *xfb, uint8_t format,
>       return total_count;
>   }
>   
> +static bool pnv_machine_get_big_core(Object *obj, Error **errp)
> +{
> +    PnvMachineState *pnv = PNV_MACHINE(obj);
> +    return pnv->big_core;
> +}
> +
> +static void pnv_machine_set_big_core(Object *obj, bool value, Error **errp)
> +{
> +    PnvMachineState *pnv = PNV_MACHINE(obj);
> +    pnv->big_core = value;
> +}
> +
> +static bool pnv_machine_get_hb(Object *obj, Error **errp)
> +{
> +    PnvMachineState *pnv = PNV_MACHINE(obj);
> +
> +    return !!pnv->fw_load_addr;
> +}
> +
> +static void pnv_machine_set_hb(Object *obj, bool value, Error **errp)
> +{
> +    PnvMachineState *pnv = PNV_MACHINE(obj);
> +
> +    if (value) {
> +        pnv->fw_load_addr = 0x8000000;
> +    }
> +}

we might want to get rid of the hostboot mode oneday. This was really
experimental stuff.

> +
>   static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
>   {
>       MachineClass *mc = MACHINE_CLASS(oc);
> @@ -2496,6 +2608,12 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
>       pmc->dt_power_mgt = pnv_dt_power_mgt;
>   
>       machine_class_allow_dynamic_sysbus_dev(mc, TYPE_PNV_PHB);
> +
> +    object_class_property_add_bool(oc, "big-core",
> +                                   pnv_machine_get_big_core,
> +                                   pnv_machine_set_big_core);
> +    object_class_property_set_description(oc, "big-core",
> +                              "Use big-core (aka fused-core) mode");
>   }
>   
>   static void pnv_machine_p10_common_class_init(ObjectClass *oc, void *data)
> @@ -2531,6 +2649,17 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
>   
>       pnv_machine_p10_common_class_init(oc, data);
>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER10";
> +
> +    /*
> +     * This is the parent of POWER10 Rainier class, so properies go here
> +     * rather than common init (which would add them to both parent and
> +     * child which is invalid).
> +     */
> +    object_class_property_add_bool(oc, "big-core",
> +                                   pnv_machine_get_big_core,
> +                                   pnv_machine_set_big_core);
> +    object_class_property_set_description(oc, "big-core",
> +                              "Use big-core (aka fused-core) mode");
>   }
>   
>   static void pnv_machine_p10_rainier_class_init(ObjectClass *oc, void *data)
> @@ -2543,22 +2672,6 @@ static void pnv_machine_p10_rainier_class_init(ObjectClass *oc, void *data)
>       pmc->i2c_init = pnv_rainier_i2c_init;
>   }
>   
> -static bool pnv_machine_get_hb(Object *obj, Error **errp)
> -{
> -    PnvMachineState *pnv = PNV_MACHINE(obj);
> -
> -    return !!pnv->fw_load_addr;
> -}
> -
> -static void pnv_machine_set_hb(Object *obj, bool value, Error **errp)
> -{
> -    PnvMachineState *pnv = PNV_MACHINE(obj);
> -
> -    if (value) {
> -        pnv->fw_load_addr = 0x8000000;
> -    }
> -}
> -
>   static void pnv_cpu_do_nmi_on_cpu(CPUState *cs, run_on_cpu_data arg)
>   {
>       CPUPPCState *env = cpu_env(cs);
> diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
> index 0f61aabb77..10417d92ae 100644
> --- a/hw/ppc/pnv_core.c
> +++ b/hw/ppc/pnv_core.c
> @@ -58,6 +58,9 @@ static void pnv_core_cpu_reset(PnvCore *pc, PowerPCCPU *cpu)
>       env->nip = 0x10;
>       env->msr |= MSR_HVB; /* Hypervisor mode */
>       env->spr[SPR_HRMOR] = pc->hrmor;
> +    if (pc->big_core) {
> +        env->spr[SPR_PVR] &= ~PPC_BIT(51); /* Clear "small core" bit */
> +    }
>       hreg_compute_hflags(env);
>       ppc_maybe_interrupt(env);
>   
> @@ -252,7 +255,12 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
>       pir_spr->default_value = pir;
>       tir_spr->default_value = tir;
>   
> -    env->core_index = core_hwid;
> +    if (pc->big_core) {
> +        /* 2 "small cores" get the same core index for SMT operations */
> +        env->core_index = core_hwid >> 1;
> +    } else {
> +        env->core_index = core_hwid;
> +    }
>   
>       /* Set time-base frequency to 512 MHz */
>       cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
> @@ -271,6 +279,7 @@ static void pnv_core_reset(void *dev)
>   
>   static void pnv_core_realize(DeviceState *dev, Error **errp)
>   {
> +    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());

no qdev_get_machine() please. I think you could use a property instead ?

>       PnvCore *pc = PNV_CORE(OBJECT(dev));
>       PnvCoreClass *pcc = PNV_CORE_GET_CLASS(pc);
>       CPUCore *cc = CPU_CORE(OBJECT(dev));
> @@ -282,14 +291,19 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
>   
>       assert(pc->chip);
>   
> +    pc->big_core = pnv->big_core;
> +    pc->pnv_tod_tbst.big_core_quirk = pnv->big_core_tbst_quirk;;
> +
>       pc->threads = g_new(PowerPCCPU *, cc->nr_threads);
>       for (i = 0; i < cc->nr_threads; i++) {
>           PowerPCCPU *cpu;
>   
>           obj = object_new(typename);
>           cpu = POWERPC_CPU(obj);
> -
> -        pc->threads[i] = POWERPC_CPU(obj);
> +        pc->threads[i] = cpu;
> +        if (cc->nr_threads > 1) {
> +            cpu->env.has_smt_siblings = true;
> +        }
>   
>           snprintf(name, sizeof(name), "thread[%d]", i);
>           object_property_add_child(OBJECT(pc), name, obj);
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> index 059d372c8a..05195527a5 100644
> --- a/hw/ppc/spapr_cpu_core.c
> +++ b/hw/ppc/spapr_cpu_core.c

This change should come in another patch preferably

> @@ -312,12 +312,16 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
>        * and the rest are explicitly started up by the guest using an RTAS call.
>        */
>       qdev_prop_set_bit(DEVICE(obj), "start-powered-off", true);
> -    env->core_index = cc->core_id;
>       cs->cpu_index = cc->core_id + i;
>       if (!spapr_set_vcpu_id(cpu, cs->cpu_index, errp)) {
>           return NULL;
>       }
>   
> +    env->core_index = cc->core_id;
> +    if (cs->nr_threads > 1) {
> +        env->has_smt_siblings = true;
> +    }
> +
>       cpu->node_id = sc->node_id;
>   
>       id = g_strdup_printf("thread[%d]", i);
> diff --git a/target/ppc/misc_helper.c b/target/ppc/misc_helper.c
> index 598c956cdd..1d56d8f85f 100644
> --- a/target/ppc/misc_helper.c
> +++ b/target/ppc/misc_helper.c
> @@ -323,7 +323,11 @@ target_ulong helper_load_sprd(CPUPPCState *env)
>                * prefer to get unimplemented message on POWER10 if it were
>                * used.
>                */
> -            return 0;
> +            if (pc->big_core) {
> +                return PPC_BIT(63);
> +            } else {
> +                return 0;
> +            }
>           }
>           /* fallthru */
>       default:
> diff --git a/target/ppc/timebase_helper.c b/target/ppc/timebase_helper.c
> index abe7b95696..6163d4ecb3 100644
> --- a/target/ppc/timebase_helper.c
> +++ b/target/ppc/timebase_helper.c
> @@ -20,6 +20,7 @@
>   #include "cpu.h"
>   #include "hw/ppc/ppc.h"
>   #include "hw/ppc/pnv_core.h"
> +#include "hw/ppc/pnv_chip.h"
>   #include "exec/helper-proto.h"
>   #include "exec/exec-all.h"
>   #include "qemu/log.h"
> @@ -297,6 +298,14 @@ static struct pnv_tod_tbst *cpu_get_tbst(PowerPCCPU *cpu)
>   {
>       PnvCore *pc = pnv_cpu_state(cpu)->core;
>   
> +    if (pc->big_core && pc->pnv_tod_tbst.big_core_quirk) {
> +        /* Must operate on the even small core */
> +        int core_id = CPU_CORE(pc)->core_id;
> +        if (core_id & 1) {
> +            pc = pc->chip->cores[core_id & ~1];
> +        }
> +    }
> +
>       return &pc->pnv_tod_tbst;
>   }
>   


Thanks,

C.

Re: [RFC PATCH 08/10] ppc/pnv: Invert the design for big-core machine modelling

[Nicholas Piggin]

On Wed May 29, 2024 at 4:57 PM AEST, Cédric Le Goater wrote:
> On 5/26/24 14:26, Nicholas Piggin wrote:
> > POWER9 and POWER10 machines come in two variants, "big-core" and
> > "small-core".
> > 
> > Big core machines are SMT8 from the software point of view, but in the
> > low level platform topology ("xscom registers and pervasive
> > addressing"), these look more like a pair of small cores ganged
> > together.
> > 
> > Presently, the way this is modelled is to create an SMT8 PnvCore and
> > add special cases to xscom and pervasive for big-core mode. This is
> > becoming too complicated to manage as more of the machine is modelled.
> > The better approach looks like the inverse, which is creating 2xPnvCore
> > ganging them together to look like an SMT8 core in TCG. The TCG SMT code
> > is quite simple to do that, and then the xscom and pervasive modelling
> > does not need to differentiate big and small core modes for the most
> > part.
> > 
> > device-tree building does need a special case to only build one
> > CPU node for each big-core because that's what the firmware expects.
> > And so does a special case workaround in the ChipTOD model.
> > 
> > A big-core machine option is added for powernv9 and 10 machines.
>
> That's another patch.

Okay.

> It is difficult to follow all the changes. I think this patch
> needs further splitting.

Sure.

> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> > ---
> >   include/hw/ppc/pnv.h         |   3 +
> >   include/hw/ppc/pnv_core.h    |   8 ++
> >   target/ppc/cpu.h             |   4 +-
> >   hw/ppc/pnv.c                 | 183 ++++++++++++++++++++++++++++-------
> >   hw/ppc/pnv_core.c            |  20 +++-
> >   hw/ppc/spapr_cpu_core.c      |   6 +-
> >   target/ppc/misc_helper.c     |   6 +-
> >   target/ppc/timebase_helper.c |   9 ++
> >   8 files changed, 197 insertions(+), 42 deletions(-)
> > 
> > diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
> > index 476b136146..93ecb062b4 100644
> > --- a/include/hw/ppc/pnv.h
> > +++ b/include/hw/ppc/pnv.h
> > @@ -100,6 +100,9 @@ struct PnvMachineState {
> >       PnvPnor      *pnor;
> >   
> >       hwaddr       fw_load_addr;
> > +
> > +    bool         big_core;
> > +    bool         big_core_tbst_quirk;
>
> I think the quirk should be introduced in its own patch.

Good idea.

> > @@ -157,6 +157,14 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
> >   
> >       pnv_cc->processor_id(chip, pc->hwid, 0, &pir, &tir);
> >   
> > +    /* Only one DT node per (big) core */
> > +    if (tir != 0) {
> > +        g_assert(pc->big_core);
> > +        g_assert(tir == 1);
> > +        g_assert(pc->hwid & 1);
> > +        return -1;
>
> return is -1 but it's not an error. right ?

Not an error just a "no CPU node to insert".

It's a bit ugly. Could return bool for yes/no and take a *offset
maybe?

> > @@ -1088,10 +1119,37 @@ static void pnv_power8_init(MachineState *machine)
> >   
> >   static void pnv_power9_init(MachineState *machine)
> >   {
> > -    if (machine->smp.threads > 8) {
> > -        error_report("Cannot support more than 8 threads/core "
> > -                     "on a powernv9/10 machine");
> > -        exit(1);
> > +    PnvMachineState *pnv = PNV_MACHINE(machine);
> > +
> > +    if (pnv->big_core) {
>
> It would be interesting to have a max_smt machine class attribute too.

Yeah, as we talked about in the other thread. Probably helps
reduce code.

> > +        if (machine->smp.threads > 8) {
> > +            error_report("Cannot support more than 8 threads/core "
> > +                         "on a powernv9/10  machine");
> > +            exit(1);
> > +        }
> > +        if (machine->smp.threads % 2 == 1) {
>
> is_power_of_2()

It does have that check later in pnv_init(), but I wanted
to be careful that we're dividing by 2 below I think it makes
it more obvious (and big-core can't have 1 thread per big core).

> > @@ -1099,6 +1157,8 @@ static void pnv_power9_init(MachineState *machine)
> >   
> >   static void pnv_power10_init(MachineState *machine)
> >   {
> > +    PnvMachineState *pnv = PNV_MACHINE(machine);
> > +    pnv->big_core_tbst_quirk = true;
> >       pnv_power9_init(machine);
> >   }
> >   
> > @@ -1169,9 +1229,15 @@ static void pnv_processor_id_p9(PnvChip *chip,
> >                                   uint32_t core_id, uint32_t thread_id,
> >                                   uint32_t *pir, uint32_t *tir)
> >   {
> > -    if (chip->nr_threads == 8) {
> > -        *pir = (chip->chip_id << 8) | ((thread_id & 1) << 2) | (core_id << 3) |
> > -               (thread_id >> 1);
> > +    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
>
> arg. We should avoid these qdev_get_machine() calls. Could big_core be a
> chip property instead ?

We could, but per machine probably makes more sense. It's
funny there seems to be no good way to get machine from CPU.
Maybe we can just add a machine pointer in PnvChip?

I'l probably leave that for another series and try to convert
most things.

> > +static bool pnv_machine_get_hb(Object *obj, Error **errp)
> > +{
> > +    PnvMachineState *pnv = PNV_MACHINE(obj);
> > +
> > +    return !!pnv->fw_load_addr;
> > +}
> > +
> > +static void pnv_machine_set_hb(Object *obj, bool value, Error **errp)
> > +{
> > +    PnvMachineState *pnv = PNV_MACHINE(obj);
> > +
> > +    if (value) {
> > +        pnv->fw_load_addr = 0x8000000;
> > +    }
> > +}
>
> we might want to get rid of the hostboot mode oneday. This was really
> experimental stuff.

Okay sure, I don't use it. Although we may want to run the
open source hostboot part of the firmware on QEMU one day,
we can always add back some options for it.

We do have a PowerVM mode too which tweaks a few things, but
that's no use for upstream.

> > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > index 059d372c8a..05195527a5 100644
> > --- a/hw/ppc/spapr_cpu_core.c
> > +++ b/hw/ppc/spapr_cpu_core.c
>
> This change should come in another patch preferably

Yeah this might have got into the wrong patch.

Thanks,
Nick

Re: [RFC PATCH 08/10] ppc/pnv: Invert the design for big-core machine modelling

[Cédric Le Goater]

>>> @@ -157,6 +157,14 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
>>>    
>>>        pnv_cc->processor_id(chip, pc->hwid, 0, &pir, &tir);
>>>    
>>> +    /* Only one DT node per (big) core */
>>> +    if (tir != 0) {
>>> +        g_assert(pc->big_core);
>>> +        g_assert(tir == 1);
>>> +        g_assert(pc->hwid & 1);
>>> +        return -1;
>>
>> return is -1 but it's not an error. right ?
> 
> Not an error just a "no CPU node to insert".
> 
> It's a bit ugly. Could return bool for yes/no and take a *offset
> maybe?

or we could pass the pa_features array  ?


>>> +        if (machine->smp.threads > 8) {
>>> +            error_report("Cannot support more than 8 threads/core "
>>> +                         "on a powernv9/10  machine");
>>> +            exit(1);
>>> +        }
>>> +        if (machine->smp.threads % 2 == 1) {
>>
>> is_power_of_2()
> 
> It does have that check later in pnv_init(), but I wanted
> to be careful that we're dividing by 2 below I think it makes
> it more obvious (and big-core can't have 1 thread per big core).

ok


> 
>>> @@ -1099,6 +1157,8 @@ static void pnv_power9_init(MachineState *machine)
>>>    
>>>    static void pnv_power10_init(MachineState *machine)
>>>    {
>>> +    PnvMachineState *pnv = PNV_MACHINE(machine);
>>> +    pnv->big_core_tbst_quirk = true;
>>>        pnv_power9_init(machine);
>>>    }
>>>    
>>> @@ -1169,9 +1229,15 @@ static void pnv_processor_id_p9(PnvChip *chip,
>>>                                    uint32_t core_id, uint32_t thread_id,
>>>                                    uint32_t *pir, uint32_t *tir)
>>>    {
>>> -    if (chip->nr_threads == 8) {
>>> -        *pir = (chip->chip_id << 8) | ((thread_id & 1) << 2) | (core_id << 3) |
>>> -               (thread_id >> 1);
>>> +    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
>>
>> arg. We should avoid these qdev_get_machine() calls. Could big_core be a
>> chip property instead ?
> 
> We could, but per machine probably makes more sense. It's
> funny there seems to be no good way to get machine from CPU.
> Maybe we can just add a machine pointer in PnvChip?


It would be easier/cleaner to propagate the machine settings to
the chip unit and subunits. If I remember correctly, real HW has a
scan init sequence doing something similar.

> I'l probably leave that for another series and try to convert
> most things.
> 
>>> +static bool pnv_machine_get_hb(Object *obj, Error **errp)
>>> +{
>>> +    PnvMachineState *pnv = PNV_MACHINE(obj);
>>> +
>>> +    return !!pnv->fw_load_addr;
>>> +}
>>> +
>>> +static void pnv_machine_set_hb(Object *obj, bool value, Error **errp)
>>> +{
>>> +    PnvMachineState *pnv = PNV_MACHINE(obj);
>>> +
>>> +    if (value) {
>>> +        pnv->fw_load_addr = 0x8000000;
>>> +    }
>>> +}
>>
>> we might want to get rid of the hostboot mode oneday. This was really
>> experimental stuff.
> 
> Okay sure, I don't use it. Although we may want to run the
> open source hostboot part of the firmware on QEMU one day,
> we can always add back some options for it.

It's not invasive either. Let's keep it. It use to work with a
trimdown Linux image.


Thanks,

C.


> 
> We do have a PowerVM mode too which tweaks a few things, but
> that's no use for upstream.
> 
>>> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
>>> index 059d372c8a..05195527a5 100644
>>> --- a/hw/ppc/spapr_cpu_core.c
>>> +++ b/hw/ppc/spapr_cpu_core.c
>>
>> This change should come in another patch preferably
> 
> Yeah this might have got into the wrong patch.
> 
> Thanks,
> Nick

Re: [RFC PATCH 08/10] ppc/pnv: Invert the design for big-core machine modelling

[Nicholas Piggin]

On Thu May 30, 2024 at 5:46 PM AEST, Cédric Le Goater wrote:
>
> >>> @@ -157,6 +157,14 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
> >>>    
> >>>        pnv_cc->processor_id(chip, pc->hwid, 0, &pir, &tir);
> >>>    
> >>> +    /* Only one DT node per (big) core */
> >>> +    if (tir != 0) {
> >>> +        g_assert(pc->big_core);
> >>> +        g_assert(tir == 1);
> >>> +        g_assert(pc->hwid & 1);
> >>> +        return -1;
> >>
> >> return is -1 but it's not an error. right ?
> > 
> > Not an error just a "no CPU node to insert".
> > 
> > It's a bit ugly. Could return bool for yes/no and take a *offset
> > maybe?
>
> or we could pass the pa_features array  ?

That might work better. I'll try it.

> >>> +        if (machine->smp.threads > 8) {
> >>> +            error_report("Cannot support more than 8 threads/core "
> >>> +                         "on a powernv9/10  machine");
> >>> +            exit(1);
> >>> +        }
> >>> +        if (machine->smp.threads % 2 == 1) {
> >>
> >> is_power_of_2()
> > 
> > It does have that check later in pnv_init(), but I wanted
> > to be careful that we're dividing by 2 below I think it makes
> > it more obvious (and big-core can't have 1 thread per big core).
>
> ok
>
>
> > 
> >>> @@ -1099,6 +1157,8 @@ static void pnv_power9_init(MachineState *machine)
> >>>    
> >>>    static void pnv_power10_init(MachineState *machine)
> >>>    {
> >>> +    PnvMachineState *pnv = PNV_MACHINE(machine);
> >>> +    pnv->big_core_tbst_quirk = true;
> >>>        pnv_power9_init(machine);
> >>>    }
> >>>    
> >>> @@ -1169,9 +1229,15 @@ static void pnv_processor_id_p9(PnvChip *chip,
> >>>                                    uint32_t core_id, uint32_t thread_id,
> >>>                                    uint32_t *pir, uint32_t *tir)
> >>>    {
> >>> -    if (chip->nr_threads == 8) {
> >>> -        *pir = (chip->chip_id << 8) | ((thread_id & 1) << 2) | (core_id << 3) |
> >>> -               (thread_id >> 1);
> >>> +    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
> >>
> >> arg. We should avoid these qdev_get_machine() calls. Could big_core be a
> >> chip property instead ?
> > 
> > We could, but per machine probably makes more sense. It's
> > funny there seems to be no good way to get machine from CPU.
> > Maybe we can just add a machine pointer in PnvChip?
>
>
> It would be easier/cleaner to propagate the machine settings to
> the chip unit and subunits. If I remember correctly, real HW has a
> scan init sequence doing something similar.

Sure. There wll be logic inside the core and chip that controls the
switch so it is not incorrect to model that way.

>
> > I'l probably leave that for another series and try to convert
> > most things.
> > 
> >>> +static bool pnv_machine_get_hb(Object *obj, Error **errp)
> >>> +{
> >>> +    PnvMachineState *pnv = PNV_MACHINE(obj);
> >>> +
> >>> +    return !!pnv->fw_load_addr;
> >>> +}
> >>> +
> >>> +static void pnv_machine_set_hb(Object *obj, bool value, Error **errp)
> >>> +{
> >>> +    PnvMachineState *pnv = PNV_MACHINE(obj);
> >>> +
> >>> +    if (value) {
> >>> +        pnv->fw_load_addr = 0x8000000;
> >>> +    }
> >>> +}
> >>
> >> we might want to get rid of the hostboot mode oneday. This was really
> >> experimental stuff.
> > 
> > Okay sure, I don't use it. Although we may want to run the
> > open source hostboot part of the firmware on QEMU one day,
> > we can always add back some options for it.
>
> It's not invasive either. Let's keep it. It use to work with a
> trimdown Linux image.

We'll keep it for now.

Thanks,
Nick

Re: [RFC PATCH 08/10] ppc/pnv: Invert the design for big-core machine modelling

[Harsh Prateek Bora]

Hi Nick,

On 5/26/24 17:56, Nicholas Piggin wrote:
> POWER9 and POWER10 machines come in two variants, "big-core" and
> "small-core".
> 
> Big core machines are SMT8 from the software point of view, but in the
> low level platform topology ("xscom registers and pervasive
> addressing"), these look more like a pair of small cores ganged
> together.
> 
> Presently, the way this is modelled is to create an SMT8 PnvCore and
> add special cases to xscom and pervasive for big-core mode. This is
> becoming too complicated to manage as more of the machine is modelled.
> The better approach looks like the inverse, which is creating 2xPnvCore
> ganging them together to look like an SMT8 core in TCG. The TCG SMT code
> is quite simple to do that, and then the xscom and pervasive modelling
> does not need to differentiate big and small core modes for the most
> part.
> 
> device-tree building does need a special case to only build one
> CPU node for each big-core because that's what the firmware expects.
> And so does a special case workaround in the ChipTOD model.
> 
> A big-core machine option is added for powernv9 and 10 machines.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   include/hw/ppc/pnv.h         |   3 +
>   include/hw/ppc/pnv_core.h    |   8 ++
>   target/ppc/cpu.h             |   4 +-
>   hw/ppc/pnv.c                 | 183 ++++++++++++++++++++++++++++-------
>   hw/ppc/pnv_core.c            |  20 +++-
>   hw/ppc/spapr_cpu_core.c      |   6 +-
>   target/ppc/misc_helper.c     |   6 +-
>   target/ppc/timebase_helper.c |   9 ++
>   8 files changed, 197 insertions(+), 42 deletions(-)
> 
> diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
> index 476b136146..93ecb062b4 100644
> --- a/include/hw/ppc/pnv.h
> +++ b/include/hw/ppc/pnv.h
> @@ -100,6 +100,9 @@ struct PnvMachineState {
>       PnvPnor      *pnor;
>   
>       hwaddr       fw_load_addr;
> +
> +    bool         big_core;
> +    bool         big_core_tbst_quirk;
>   };
>   
>   PnvChip *pnv_get_chip(PnvMachineState *pnv, uint32_t chip_id);
> diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
> index 21297262c1..39f8f33e6c 100644
> --- a/include/hw/ppc/pnv_core.h
> +++ b/include/hw/ppc/pnv_core.h
> @@ -27,6 +27,13 @@
>   
>   /* ChipTOD and TimeBase State Machine */
>   struct pnv_tod_tbst {
> +    /*
> +     * POWER10 DD2.0 - big core TFMR drives the state machine on the even
> +     * small core. Skiboot has a workaround that targets the even small core
> +     * for CHIPTOD_TO_TB ops.
> +     */
> +    bool big_core_quirk;
> +
>       int tb_ready_for_tod; /* core TB ready to receive TOD from chiptod */
>       int tod_sent_to_tb;   /* chiptod sent TOD to the core TB */
>   
> @@ -49,6 +56,7 @@ struct PnvCore {
>   
>       /*< public >*/
>       PowerPCCPU **threads;
> +    bool big_core;
>       uint32_t pir;
>       uint32_t hwid;
>       uint64_t hrmor;
> diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
> index 8fd6ade471..de15e38af8 100644
> --- a/target/ppc/cpu.h
> +++ b/target/ppc/cpu.h
> @@ -1248,6 +1248,7 @@ struct CPUArchState {
>       int access_type;
>   
>       /* For SMT processors */
> +    int has_smt_siblings;

        bool ?

>       int core_index;
>   
>   #if !defined(CONFIG_USER_ONLY)
> @@ -1276,7 +1277,6 @@ struct CPUArchState {
>       uint32_t tlb_need_flush; /* Delayed flush needed */
>   #define TLB_NEED_LOCAL_FLUSH   0x1
>   #define TLB_NEED_GLOBAL_FLUSH  0x2
> -
>   #endif
>   
>       /* Other registers */
> @@ -1407,7 +1407,7 @@ struct CPUArchState {
>   };
>   
>   #define PPC_CPU_HAS_CORE_SIBLINGS(cs)                           \
> -    (cs->nr_threads > 1)
> +    (POWERPC_CPU(cs)->env.has_smt_siblings)
>   
>   #define PPC_CPU_HAS_LPAR_SIBLINGS(cs)                           \
>       ((POWERPC_CPU(cs)->env.flags & POWERPC_FLAG_SMT_1LPAR) &&   \
> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
> index 7d062ec16c..5364c55bbb 100644
> --- a/hw/ppc/pnv.c
> +++ b/hw/ppc/pnv.c
> @@ -142,7 +142,7 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
>       CPUPPCState *env = &cpu->env;
>       PowerPCCPUClass *pcc = POWERPC_CPU_GET_CLASS(cs);
>       PnvChipClass *pnv_cc = PNV_CHIP_GET_CLASS(chip);
> -    g_autofree uint32_t *servers_prop = g_new(uint32_t, smt_threads);
> +    uint32_t *servers_prop;
>       int i;
>       uint32_t pir, tir;
>       uint32_t segs[] = {cpu_to_be32(28), cpu_to_be32(40),
> @@ -157,6 +157,14 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
>   
>       pnv_cc->processor_id(chip, pc->hwid, 0, &pir, &tir);
>   
> +    /* Only one DT node per (big) core */
> +    if (tir != 0) {
> +        g_assert(pc->big_core);
> +        g_assert(tir == 1);
> +        g_assert(pc->hwid & 1) > +        return -1;
> +    }
> +
>       nodename = g_strdup_printf("%s@%x", dc->fw_name, pir);
>       offset = fdt_add_subnode(fdt, cpus_offset, nodename);
>       _FDT(offset);
> @@ -236,12 +244,28 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, void *fdt)
>       }
>   
>       /* Build interrupt servers properties */
> -    for (i = 0; i < smt_threads; i++) {
> -        pnv_cc->processor_id(chip, pc->hwid, i, &pir, &tir);
> -        servers_prop[i] = cpu_to_be32(pir);
> +    if (pc->big_core) {
> +        servers_prop = g_new(uint32_t, smt_threads * 2);
> +        for (i = 0; i < smt_threads; i++) {
> +            pnv_cc->processor_id(chip, pc->hwid, i, &pir, &tir);
> +            servers_prop[i * 2] = cpu_to_be32(pir);
> +
> +            pnv_cc->processor_id(chip, pc->hwid + 1, i, &pir, &tir);
> +            servers_prop[i * 2 + 1] = cpu_to_be32(pir);
> +        }
> +        _FDT((fdt_setprop(fdt, offset, "ibm,ppc-interrupt-server#s",
> +                          servers_prop, sizeof(*servers_prop) * smt_threads
> +                                        * 2)));
> +    } else {
> +        servers_prop = g_new(uint32_t, smt_threads);
> +        for (i = 0; i < smt_threads; i++) {
> +            pnv_cc->processor_id(chip, pc->hwid, i, &pir, &tir);
> +            servers_prop[i] = cpu_to_be32(pir);
> +        }
> +        _FDT((fdt_setprop(fdt, offset, "ibm,ppc-interrupt-server#s",
> +                          servers_prop, sizeof(*servers_prop) * smt_threads)));
>       }
> -    _FDT((fdt_setprop(fdt, offset, "ibm,ppc-interrupt-server#s",
> -                       servers_prop, sizeof(*servers_prop) * smt_threads)));
> +    g_free(servers_prop);
>   
>       return offset;
>   }
> @@ -330,6 +354,8 @@ static void pnv_chip_power8_dt_populate(PnvChip *chip, void *fdt)
>           int offset;
>   
>           offset = pnv_dt_core(chip, pnv_core, fdt);
> +        if (offset == -1)
> +            continue;
>   
>           _FDT((fdt_setprop(fdt, offset, "ibm,pa-features",
>                              pa_features_207, sizeof(pa_features_207))));
> @@ -387,6 +413,8 @@ static void pnv_chip_power9_dt_populate(PnvChip *chip, void *fdt)
>           int offset;
>   
>           offset = pnv_dt_core(chip, pnv_core, fdt);
> +        if (offset == -1)
> +            continue;
>   
>           _FDT((fdt_setprop(fdt, offset, "ibm,pa-features",
>                              pa_features_300, sizeof(pa_features_300))));
> @@ -448,6 +476,8 @@ static void pnv_chip_power10_dt_populate(PnvChip *chip, void *fdt)
>           int offset;
>   
>           offset = pnv_dt_core(chip, pnv_core, fdt);
> +        if (offset == -1)
> +            continue;
>   
>           _FDT((fdt_setprop(fdt, offset, "ibm,pa-features",
>                              pa_features_31, sizeof(pa_features_31))));
> @@ -981,11 +1011,12 @@ static void pnv_init(MachineState *machine)
>           machine->smp.max_cpus / (machine->smp.cores * machine->smp.threads);
>   
>       if (!is_power_of_2(machine->smp.threads)) {
> -        error_report("Cannot support %d threads/core on a powernv"
> +        error_report("Cannot support %d threads/core on a powernv "
>                        "machine because it must be a power of 2",
>                        machine->smp.threads);
>           exit(1);
>       }
> +
>       /*
>        * TODO: should we decide on how many chips we can create based
>        * on #cores and Venice vs. Murano vs. Naples chip type etc...,
> @@ -1088,10 +1119,37 @@ static void pnv_power8_init(MachineState *machine)
>   
>   static void pnv_power9_init(MachineState *machine)
>   {
> -    if (machine->smp.threads > 8) {
> -        error_report("Cannot support more than 8 threads/core "
> -                     "on a powernv9/10 machine");
> -        exit(1);
> +    PnvMachineState *pnv = PNV_MACHINE(machine);
> +
> +    if (pnv->big_core) {
> +        if (machine->smp.threads > 8) {
> +            error_report("Cannot support more than 8 threads/core "
> +                         "on a powernv9/10  machine");
> +            exit(1);
> +        }
> +        if (machine->smp.threads % 2 == 1) {
> +            error_report("Cannot support %d threads with big-core option "
> +                         "because it must be an even number",
> +                         machine->smp.threads);
> +            exit(1);
> +        }
> +
> +        /*
> +         * powernv models PnvCore as a SMT4 core. Big-core requires 2xPnvCore
> +         * per core, so adjust topology here. pnv_dt_core() processor
> +         * device-tree and TCG SMT code make the 2 cores appear as one big core
> +         * from software point of view. pnv pervasive models and xscoms tend to
> +         * see the big core as 2 small core halves.
> +         */
> +        machine->smp.cores *= 2;
> +        machine->smp.threads /= 2;
> +    } else {
> +        if (machine->smp.threads > 4) {
> +            error_report("Cannot support more than 4 threads/core "
> +                         "on a powernv9/10 small-core machine "
> +                         "(use big-core=on for 8 threads per core)");
> +            exit(1);
> +        }
>       }
>   
>       pnv_init(machine);
> @@ -1099,6 +1157,8 @@ static void pnv_power9_init(MachineState *machine)
>   
>   static void pnv_power10_init(MachineState *machine)
>   {
> +    PnvMachineState *pnv = PNV_MACHINE(machine);
> +    pnv->big_core_tbst_quirk = true;

If this is always and only true for p10, do we really need a variable 
for this since it is only used to init the 
pc->pnv_tod_tbst.big_core_quirk which could otherwise just check if it's 
a P10 and initialize accordingly?


>       pnv_power9_init(machine);
>   }
>   
> @@ -1169,9 +1229,15 @@ static void pnv_processor_id_p9(PnvChip *chip,
>                                   uint32_t core_id, uint32_t thread_id,
>                                   uint32_t *pir, uint32_t *tir)
>   {
> -    if (chip->nr_threads == 8) {
> -        *pir = (chip->chip_id << 8) | ((thread_id & 1) << 2) | (core_id << 3) |
> -               (thread_id >> 1);
> +    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
> +
> +    if (pnv->big_core) {
> +        /* Big-core interleaves thread ID between small-cores */
> +        thread_id <<= 1;
> +        thread_id |= core_id & 1;
> +        core_id >>= 1;
> +
> +        *pir = (chip->chip_id << 8) | (core_id << 3) | thread_id;
>       } else {
>           *pir = (chip->chip_id << 8) | (core_id << 2) | thread_id;
>       }
> @@ -1193,9 +1259,15 @@ static void pnv_processor_id_p10(PnvChip *chip,
>                                   uint32_t core_id, uint32_t thread_id,
>                                   uint32_t *pir, uint32_t *tir)
>   {
> -    if (chip->nr_threads == 8) {
> -        *pir = (chip->chip_id << 8) | ((core_id / 4) << 4) |
> -                ((core_id % 2) << 3) | thread_id;
> +    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
> +
> +    if (pnv->big_core) {
> +        /* Big-core interleaves thread ID between small-cores */
> +        thread_id <<= 1;
> +        thread_id |= core_id & 1;
> +        core_id >>= 1;
> +
> +        *pir = (chip->chip_id << 8) | (core_id << 3) | thread_id;
>       } else {
>           *pir = (chip->chip_id << 8) | (core_id << 2) | thread_id;
>       }

With above changes, both pnv_processor_id_<p9|p10> are becoming 
duplicate. Should one be shared across both ?

> @@ -2146,7 +2218,8 @@ static void pnv_chip_power10_class_init(ObjectClass *klass, void *data)
>                                       &k->parent_realize);
>   }
>   
> -static void pnv_chip_core_sanitize(PnvChip *chip, Error **errp)
> +static void pnv_chip_core_sanitize(PnvMachineState *pnv, PnvChip *chip,
> +                                   Error **errp)
>   {
>       PnvChipClass *pcc = PNV_CHIP_GET_CLASS(chip);
>       int cores_max;
> @@ -2167,6 +2240,17 @@ static void pnv_chip_core_sanitize(PnvChip *chip, Error **errp)
>       }
>       chip->cores_mask &= pcc->cores_mask;
>   
> +    /* Ensure small-cores a paired up in big-core mode */
> +    if (pnv->big_core) {
> +        uint64_t even_cores = chip->cores_mask & 0x5555555555555555ULL;
> +        uint64_t odd_cores = chip->cores_mask & 0xaaaaaaaaaaaaaaaaULL;

Should these masks be macros ?

> +
> +        if (even_cores ^ (odd_cores >> 1)) {
> +            error_setg(errp, "warning: unpaired cores in big-core mode !");
> +            return;

Since commit e3fe3988d7, we are supposed to return false in case of
failure if using Error** APIs, hence helpers below taking Error ** need
to return bool instead of void as well.

> +        }
> +    }
> +
>       /* now that we have a sane layout, let check the number of cores */
>       cores_max = ctpop64(chip->cores_mask);
>       if (chip->nr_cores > cores_max) {
> @@ -2190,7 +2274,7 @@ static void pnv_chip_core_realize(PnvChip *chip, Error **errp)
>       }
>   
>       /* Cores */
> -    pnv_chip_core_sanitize(chip, &error);
> +    pnv_chip_core_sanitize(pnv, chip, &error);
>       if (error) {
>           error_propagate(errp, error);
>           return;
> @@ -2445,6 +2529,34 @@ static int pnv10_xive_match_nvt(XiveFabric *xfb, uint8_t format,
>       return total_count;
>   }
>   
> +static bool pnv_machine_get_big_core(Object *obj, Error **errp)
> +{
> +    PnvMachineState *pnv = PNV_MACHINE(obj);
> +    return pnv->big_core;
> +}
> +
> +static void pnv_machine_set_big_core(Object *obj, bool value, Error **errp)
> +{
> +    PnvMachineState *pnv = PNV_MACHINE(obj);
> +    pnv->big_core = value;
> +}
> +
> +static bool pnv_machine_get_hb(Object *obj, Error **errp)
> +{
> +    PnvMachineState *pnv = PNV_MACHINE(obj);
> +
> +    return !!pnv->fw_load_addr;
> +}
> +
> +static void pnv_machine_set_hb(Object *obj, bool value, Error **errp)
> +{
> +    PnvMachineState *pnv = PNV_MACHINE(obj);
> +
> +    if (value) {
> +        pnv->fw_load_addr = 0x8000000;
> +    }
> +}
> +
>   static void pnv_machine_power8_class_init(ObjectClass *oc, void *data)
>   {
>       MachineClass *mc = MACHINE_CLASS(oc);
> @@ -2496,6 +2608,12 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
>       pmc->dt_power_mgt = pnv_dt_power_mgt;
>   
>       machine_class_allow_dynamic_sysbus_dev(mc, TYPE_PNV_PHB);
> +
> +    object_class_property_add_bool(oc, "big-core",
> +                                   pnv_machine_get_big_core,
> +                                   pnv_machine_set_big_core);
> +    object_class_property_set_description(oc, "big-core",
> +                              "Use big-core (aka fused-core) mode");
>   }
>   
>   static void pnv_machine_p10_common_class_init(ObjectClass *oc, void *data)
> @@ -2531,6 +2649,17 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
>   
>       pnv_machine_p10_common_class_init(oc, data);
>       mc->desc = "IBM PowerNV (Non-Virtualized) POWER10";
> +
> +    /*
> +     * This is the parent of POWER10 Rainier class, so properies go here
> +     * rather than common init (which would add them to both parent and
> +     * child which is invalid).
> +     */
> +    object_class_property_add_bool(oc, "big-core",
> +                                   pnv_machine_get_big_core,
> +                                   pnv_machine_set_big_core);
> +    object_class_property_set_description(oc, "big-core",
> +                              "Use big-core (aka fused-core) mode");
>   }
>   
>   static void pnv_machine_p10_rainier_class_init(ObjectClass *oc, void *data)
> @@ -2543,22 +2672,6 @@ static void pnv_machine_p10_rainier_class_init(ObjectClass *oc, void *data)
>       pmc->i2c_init = pnv_rainier_i2c_init;
>   }
>   
> -static bool pnv_machine_get_hb(Object *obj, Error **errp)
> -{
> -    PnvMachineState *pnv = PNV_MACHINE(obj);
> -
> -    return !!pnv->fw_load_addr;
> -}
> -
> -static void pnv_machine_set_hb(Object *obj, bool value, Error **errp)
> -{
> -    PnvMachineState *pnv = PNV_MACHINE(obj);
> -
> -    if (value) {
> -        pnv->fw_load_addr = 0x8000000;
> -    }
> -}
> -
>   static void pnv_cpu_do_nmi_on_cpu(CPUState *cs, run_on_cpu_data arg)
>   {
>       CPUPPCState *env = cpu_env(cs);
> diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
> index 0f61aabb77..10417d92ae 100644
> --- a/hw/ppc/pnv_core.c
> +++ b/hw/ppc/pnv_core.c
> @@ -58,6 +58,9 @@ static void pnv_core_cpu_reset(PnvCore *pc, PowerPCCPU *cpu)
>       env->nip = 0x10;
>       env->msr |= MSR_HVB; /* Hypervisor mode */
>       env->spr[SPR_HRMOR] = pc->hrmor;
> +    if (pc->big_core) {
> +        env->spr[SPR_PVR] &= ~PPC_BIT(51); /* Clear "small core" bit */
> +    }
>       hreg_compute_hflags(env);
>       ppc_maybe_interrupt(env);
>   
> @@ -252,7 +255,12 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
>       pir_spr->default_value = pir;
>       tir_spr->default_value = tir;
>   
> -    env->core_index = core_hwid;
> +    if (pc->big_core) {
> +        /* 2 "small cores" get the same core index for SMT operations */
> +        env->core_index = core_hwid >> 1;
> +    } else {
> +        env->core_index = core_hwid;
> +    }
>   
>       /* Set time-base frequency to 512 MHz */
>       cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
> @@ -271,6 +279,7 @@ static void pnv_core_reset(void *dev)
>   
>   static void pnv_core_realize(DeviceState *dev, Error **errp)
>   {
> +    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
>       PnvCore *pc = PNV_CORE(OBJECT(dev));
>       PnvCoreClass *pcc = PNV_CORE_GET_CLASS(pc);
>       CPUCore *cc = CPU_CORE(OBJECT(dev));
> @@ -282,14 +291,19 @@ static void pnv_core_realize(DeviceState *dev, Error **errp)
>   
>       assert(pc->chip);
>   
> +    pc->big_core = pnv->big_core;
> +    pc->pnv_tod_tbst.big_core_quirk = pnv->big_core_tbst_quirk;;
> +
>       pc->threads = g_new(PowerPCCPU *, cc->nr_threads);
>       for (i = 0; i < cc->nr_threads; i++) {
>           PowerPCCPU *cpu;
>   
>           obj = object_new(typename);
>           cpu = POWERPC_CPU(obj);
> -
> -        pc->threads[i] = POWERPC_CPU(obj);
> +        pc->threads[i] = cpu;
> +        if (cc->nr_threads > 1) {
> +            cpu->env.has_smt_siblings = true;
> +        }
>   
>           snprintf(name, sizeof(name), "thread[%d]", i);
>           object_property_add_child(OBJECT(pc), name, obj);
> diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> index 059d372c8a..05195527a5 100644
> --- a/hw/ppc/spapr_cpu_core.c
> +++ b/hw/ppc/spapr_cpu_core.c
> @@ -312,12 +312,16 @@ static PowerPCCPU *spapr_create_vcpu(SpaprCpuCore *sc, int i, Error **errp)
>        * and the rest are explicitly started up by the guest using an RTAS call.
>        */
>       qdev_prop_set_bit(DEVICE(obj), "start-powered-off", true);
> -    env->core_index = cc->core_id;
>       cs->cpu_index = cc->core_id + i;
>       if (!spapr_set_vcpu_id(cpu, cs->cpu_index, errp)) {
>           return NULL;
>       }
>   
> +    env->core_index = cc->core_id;
> +    if (cs->nr_threads > 1) {
> +        env->has_smt_siblings = true;
> +    }
> +
>       cpu->node_id = sc->node_id;
>   
>       id = g_strdup_printf("thread[%d]", i);
> diff --git a/target/ppc/misc_helper.c b/target/ppc/misc_helper.c
> index 598c956cdd..1d56d8f85f 100644
> --- a/target/ppc/misc_helper.c
> +++ b/target/ppc/misc_helper.c
> @@ -323,7 +323,11 @@ target_ulong helper_load_sprd(CPUPPCState *env)
>                * prefer to get unimplemented message on POWER10 if it were
>                * used.
>                */
> -            return 0;
> +            if (pc->big_core) {
> +                return PPC_BIT(63);
> +            } else {
> +                return 0;
> +            }
>           }
>           /* fallthru */
>       default:
> diff --git a/target/ppc/timebase_helper.c b/target/ppc/timebase_helper.c
> index abe7b95696..6163d4ecb3 100644
> --- a/target/ppc/timebase_helper.c
> +++ b/target/ppc/timebase_helper.c
> @@ -20,6 +20,7 @@
>   #include "cpu.h"
>   #include "hw/ppc/ppc.h"
>   #include "hw/ppc/pnv_core.h"
> +#include "hw/ppc/pnv_chip.h"
>   #include "exec/helper-proto.h"
>   #include "exec/exec-all.h"
>   #include "qemu/log.h"
> @@ -297,6 +298,14 @@ static struct pnv_tod_tbst *cpu_get_tbst(PowerPCCPU *cpu)
>   {
>       PnvCore *pc = pnv_cpu_state(cpu)->core;
>   
> +    if (pc->big_core && pc->pnv_tod_tbst.big_core_quirk) {
> +        /* Must operate on the even small core */
> +        int core_id = CPU_CORE(pc)->core_id;
> +        if (core_id & 1) {

Comment says must operate on even small core but above check is for odd 
core_id ? Not sure if this is intentional.

Thanks
Harsh
> +            pc = pc->chip->cores[core_id & ~1];
> +        }
> +    }
> +
>       return &pc->pnv_tod_tbst;
>   }
>   

[RFC PATCH 09/10] ppc/pnv: Implement POWER10 PC xscom registers for direct controls

[Nicholas Piggin]

The PC unit in the processor core contains xscom registers that provide
low level status and control of the CPU.

This implements "direct controls" sufficient for OPAL (skiboot) firmware
use, which is to stop threads and send them non-maskable IPIs in the
form of SRESET interrupts.

POWER10 is sufficiently different (particularly QME and special wakeup)
from POWER9 that it is not trivial to implement by reusing the code.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 include/hw/core/cpu.h     |  8 ++++
 include/hw/ppc/pnv.h      |  2 +
 include/hw/ppc/pnv_core.h |  3 ++
 hw/ppc/pnv.c              |  7 +++-
 hw/ppc/pnv_core.c         | 88 ++++++++++++++++++++++++++++++++++++---
 system/cpus.c             | 10 +++++
 6 files changed, 112 insertions(+), 6 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index bb398e8237..52a8fc65cb 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -974,6 +974,14 @@ void cpu_reset_interrupt(CPUState *cpu, int mask);
  */
 void cpu_exit(CPUState *cpu);
 
+/**
+ * cpu_pause:
+ * @cpu: The CPU to pause.
+ *
+ * Resumes CPU, i.e. puts CPU into stopped state.
+ */
+void cpu_pause(CPUState *cpu);
+
 /**
  * cpu_resume:
  * @cpu: The CPU to resume.
diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
index 93ecb062b4..bec603f1a8 100644
--- a/include/hw/ppc/pnv.h
+++ b/include/hw/ppc/pnv.h
@@ -111,6 +111,8 @@ PnvChip *pnv_chip_add_phb(PnvChip *chip, PnvPHB *phb);
 #define PNV_FDT_ADDR          0x01000000
 #define PNV_TIMEBASE_FREQ     512000000ULL
 
+void pnv_cpu_do_nmi(CPUState *cs);
+
 /*
  * BMC helpers
  */
diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
index 39f8f33e6c..9599da15ea 100644
--- a/include/hw/ppc/pnv_core.h
+++ b/include/hw/ppc/pnv_core.h
@@ -109,6 +109,9 @@ OBJECT_DECLARE_TYPE(PnvQuad, PnvQuadClass, PNV_QUAD)
 struct PnvQuad {
     DeviceState parent_obj;
 
+    bool special_wakeup_done;
+    bool special_wakeup[4];
+
     uint32_t quad_id;
     MemoryRegion xscom_regs;
     MemoryRegion xscom_qme_regs;
diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
index 5364c55bbb..765142965f 100644
--- a/hw/ppc/pnv.c
+++ b/hw/ppc/pnv.c
@@ -2700,12 +2700,17 @@ static void pnv_cpu_do_nmi_on_cpu(CPUState *cs, run_on_cpu_data arg)
     }
 }
 
+void pnv_cpu_do_nmi(CPUState *cs)
+{
+    async_run_on_cpu(cs, pnv_cpu_do_nmi_on_cpu, RUN_ON_CPU_NULL);
+}
+
 static void pnv_nmi(NMIState *n, int cpu_index, Error **errp)
 {
     CPUState *cs;
 
     CPU_FOREACH(cs) {
-        async_run_on_cpu(cs, pnv_cpu_do_nmi_on_cpu, RUN_ON_CPU_NULL);
+        pnv_cpu_do_nmi(cs);
     }
 }
 
diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
index 10417d92ae..835c35d90b 100644
--- a/hw/ppc/pnv_core.c
+++ b/hw/ppc/pnv_core.c
@@ -184,16 +184,40 @@ static const MemoryRegionOps pnv_core_power9_xscom_ops = {
  */
 
 #define PNV10_XSCOM_EC_CORE_THREAD_STATE    0x412
+#define PNV10_XSCOM_EC_CORE_THREAD_INFO     0x413
+#define PNV10_XSCOM_EC_CORE_DIRECT_CONTROLS 0x449
+#define PNV10_XSCOM_EC_CORE_RAS_STATUS      0x454
 
 static uint64_t pnv_core_power10_xscom_read(void *opaque, hwaddr addr,
                                            unsigned int width)
 {
+    PnvCore *pc = PNV_CORE(opaque);
+    int nr_threads = CPU_CORE(pc)->nr_threads;
+    int i;
     uint32_t offset = addr >> 3;
     uint64_t val = 0;
 
     switch (offset) {
     case PNV10_XSCOM_EC_CORE_THREAD_STATE:
-        val = 0;
+        for (i = 0; i < nr_threads; i++) {
+            PowerPCCPU *cpu = pc->threads[i];
+            CPUState *cs = CPU(cpu);
+
+            if (cs->halted) {
+                val |= PPC_BIT(56 + i);
+            }
+        }
+        break;
+    case PNV10_XSCOM_EC_CORE_THREAD_INFO:
+        break;
+    case PNV10_XSCOM_EC_CORE_RAS_STATUS:
+        for (i = 0; i < nr_threads; i++) {
+            PowerPCCPU *cpu = pc->threads[i];
+            CPUState *cs = CPU(cpu);
+            if (cs->stopped) {
+                val |= PPC_BIT(0 + 8*i) | PPC_BIT(1 + 8*i);
+            }
+        }
         break;
     default:
         qemu_log_mask(LOG_UNIMP, "%s: unimp read 0x%08x\n", __func__,
@@ -206,9 +230,45 @@ static uint64_t pnv_core_power10_xscom_read(void *opaque, hwaddr addr,
 static void pnv_core_power10_xscom_write(void *opaque, hwaddr addr,
                                          uint64_t val, unsigned int width)
 {
+    PnvCore *pc = PNV_CORE(opaque);
+    int nr_threads = CPU_CORE(pc)->nr_threads;
+    int i;
     uint32_t offset = addr >> 3;
 
     switch (offset) {
+    case PNV10_XSCOM_EC_CORE_DIRECT_CONTROLS:
+        for (i = 0; i < nr_threads; i++) {
+            PowerPCCPU *cpu = pc->threads[i];
+            CPUState *cs = CPU(cpu);
+
+            if (val & PPC_BIT(7 + 8*i)) { /* stop */
+                val &= ~PPC_BIT(7 + 8*i);
+                cpu_pause(cs);
+            }
+            if (val & PPC_BIT(6 + 8*i)) { /* start */
+                val &= ~PPC_BIT(6 + 8*i);
+                cpu_resume(cs);
+            }
+            if (val & PPC_BIT(4 + 8*i)) { /* sreset */
+                val &= ~PPC_BIT(4 + 8*i);
+                pnv_cpu_do_nmi(cs);
+            }
+            if (val & PPC_BIT(3 + 8*i)) { /* clear maint */
+                /*
+                 * Hardware has very particular cases for where clear maint
+                 * must be used and where start must be used to resume a
+                 * thread. These are not modelled exactly, just treat
+                 * this and start the same.
+                 */
+                val &= ~PPC_BIT(3 + 8*i);
+                cpu_resume(cs);
+            }
+        }
+        if (val) {
+            qemu_log_mask(LOG_UNIMP, "%s: unimp bits in DIRECT_CONTROLS 0x%016lx\n", __func__, val);
+        }
+        break;
+
     default:
         qemu_log_mask(LOG_UNIMP, "%s: unimp write 0x%08x\n", __func__,
                       offset);
@@ -525,6 +585,7 @@ static const MemoryRegionOps pnv_quad_power10_xscom_ops = {
 static uint64_t pnv_qme_power10_xscom_read(void *opaque, hwaddr addr,
                                             unsigned int width)
 {
+    PnvQuad *eq = PNV_QUAD(opaque);
     uint32_t offset = addr >> 3;
     uint64_t val = -1;
 
@@ -532,10 +593,14 @@ static uint64_t pnv_qme_power10_xscom_read(void *opaque, hwaddr addr,
      * Forth nibble selects the core within a quad, mask it to process read
      * for any core.
      */
-    switch (offset & ~0xf000) {
-    case P10_QME_SPWU_HYP:
+    switch (offset & ~PPC_BITMASK32(16, 19)) {
     case P10_QME_SSH_HYP:
-        return 0;
+        val = 0;
+        if (eq->special_wakeup_done) {
+            val |= PPC_BIT(1); /* SPWU DONE */
+            val |= PPC_BIT(4); /* SSH SPWU DONE */
+        }
+        break;
     default:
         qemu_log_mask(LOG_UNIMP, "%s: unimp read 0x%08x\n", __func__,
                       offset);
@@ -547,9 +612,22 @@ static uint64_t pnv_qme_power10_xscom_read(void *opaque, hwaddr addr,
 static void pnv_qme_power10_xscom_write(void *opaque, hwaddr addr,
                                          uint64_t val, unsigned int width)
 {
+    PnvQuad *eq = PNV_QUAD(opaque);
     uint32_t offset = addr >> 3;
+    bool set;
+    int i;
 
-    switch (offset) {
+    switch (offset & ~PPC_BITMASK32(16, 19)) {
+    case P10_QME_SPWU_HYP:
+        set = !!(val & PPC_BIT(0));
+        eq->special_wakeup_done = set;
+        for (i = 0; i < 4; i++) {
+            /* These bits select cores in the quad */
+            if (offset & PPC_BIT32(16 + i)) {
+                eq->special_wakeup[i] = set;
+            }
+        }
+        break;
     default:
         qemu_log_mask(LOG_UNIMP, "%s: unimp write 0x%08x\n", __func__,
                       offset);
diff --git a/system/cpus.c b/system/cpus.c
index 68d161d96b..1ca18ae7ea 100644
--- a/system/cpus.c
+++ b/system/cpus.c
@@ -602,6 +602,16 @@ void pause_all_vcpus(void)
     bql_lock();
 }
 
+void cpu_pause(CPUState *cpu)
+{
+    if (qemu_cpu_is_self(cpu)) {
+        qemu_cpu_stop(cpu, true);
+    } else {
+        cpu->stop = true;
+        qemu_cpu_kick(cpu);
+    }
+}
+
 void cpu_resume(CPUState *cpu)
 {
     cpu->stop = false;
-- 
2.43.0

Re: [RFC PATCH 09/10] ppc/pnv: Implement POWER10 PC xscom registers for direct controls

[Cédric Le Goater]

On 5/26/24 14:26, Nicholas Piggin wrote:
> The PC unit in the processor core contains xscom registers that provide
> low level status and control of the CPU.
> 
> This implements "direct controls" sufficient for OPAL (skiboot) firmware
> use, which is to stop threads and send them non-maskable IPIs in the
> form of SRESET interrupts.
> 
> POWER10 is sufficiently different (particularly QME and special wakeup)
> from POWER9 that it is not trivial to implement by reusing the code.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   include/hw/core/cpu.h     |  8 ++++
>   include/hw/ppc/pnv.h      |  2 +
>   include/hw/ppc/pnv_core.h |  3 ++
>   hw/ppc/pnv.c              |  7 +++-
>   hw/ppc/pnv_core.c         | 88 ++++++++++++++++++++++++++++++++++++---
>   system/cpus.c             | 10 +++++
>   6 files changed, 112 insertions(+), 6 deletions(-)
> 
> diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
> index bb398e8237..52a8fc65cb 100644
> --- a/include/hw/core/cpu.h
> +++ b/include/hw/core/cpu.h
> @@ -974,6 +974,14 @@ void cpu_reset_interrupt(CPUState *cpu, int mask);
>    */
>   void cpu_exit(CPUState *cpu);
>   
> +/**
> + * cpu_pause:
> + * @cpu: The CPU to pause.
> + *
> + * Resumes CPU, i.e. puts CPU into stopped state.
> + */
> +void cpu_pause(CPUState *cpu);
> +
>   /**
>    * cpu_resume:
>    * @cpu: The CPU to resume.
> diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
> index 93ecb062b4..bec603f1a8 100644
> --- a/include/hw/ppc/pnv.h
> +++ b/include/hw/ppc/pnv.h
> @@ -111,6 +111,8 @@ PnvChip *pnv_chip_add_phb(PnvChip *chip, PnvPHB *phb);
>   #define PNV_FDT_ADDR          0x01000000
>   #define PNV_TIMEBASE_FREQ     512000000ULL
>   
> +void pnv_cpu_do_nmi(CPUState *cs);
> +
>   /*
>    * BMC helpers
>    */
> diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
> index 39f8f33e6c..9599da15ea 100644
> --- a/include/hw/ppc/pnv_core.h
> +++ b/include/hw/ppc/pnv_core.h
> @@ -109,6 +109,9 @@ OBJECT_DECLARE_TYPE(PnvQuad, PnvQuadClass, PNV_QUAD)
>   struct PnvQuad {
>       DeviceState parent_obj;
>   
> +    bool special_wakeup_done;
> +    bool special_wakeup[4];
> +
>       uint32_t quad_id;
>       MemoryRegion xscom_regs;
>       MemoryRegion xscom_qme_regs;
> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
> index 5364c55bbb..765142965f 100644
> --- a/hw/ppc/pnv.c
> +++ b/hw/ppc/pnv.c
> @@ -2700,12 +2700,17 @@ static void pnv_cpu_do_nmi_on_cpu(CPUState *cs, run_on_cpu_data arg)
>       }
>   }
>   
> +void pnv_cpu_do_nmi(CPUState *cs)
> +{
> +    async_run_on_cpu(cs, pnv_cpu_do_nmi_on_cpu, RUN_ON_CPU_NULL);
> +}
> +
>   static void pnv_nmi(NMIState *n, int cpu_index, Error **errp)
>   {
>       CPUState *cs;
>   
>       CPU_FOREACH(cs) {
> -        async_run_on_cpu(cs, pnv_cpu_do_nmi_on_cpu, RUN_ON_CPU_NULL);
> +        pnv_cpu_do_nmi(cs);
>       }
>   }

What about ?

https://lore.kernel.org/qemu-devel/20240424093048.180966-1-clg@redhat.com/


Thanks,

C.



> diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
> index 10417d92ae..835c35d90b 100644
> --- a/hw/ppc/pnv_core.c
> +++ b/hw/ppc/pnv_core.c
> @@ -184,16 +184,40 @@ static const MemoryRegionOps pnv_core_power9_xscom_ops = {
>    */
>   
>   #define PNV10_XSCOM_EC_CORE_THREAD_STATE    0x412
> +#define PNV10_XSCOM_EC_CORE_THREAD_INFO     0x413
> +#define PNV10_XSCOM_EC_CORE_DIRECT_CONTROLS 0x449
> +#define PNV10_XSCOM_EC_CORE_RAS_STATUS      0x454
>   
>   static uint64_t pnv_core_power10_xscom_read(void *opaque, hwaddr addr,
>                                              unsigned int width)
>   {
> +    PnvCore *pc = PNV_CORE(opaque);
> +    int nr_threads = CPU_CORE(pc)->nr_threads;
> +    int i;
>       uint32_t offset = addr >> 3;
>       uint64_t val = 0;
>   
>       switch (offset) {
>       case PNV10_XSCOM_EC_CORE_THREAD_STATE:
> -        val = 0;
> +        for (i = 0; i < nr_threads; i++) {
> +            PowerPCCPU *cpu = pc->threads[i];
> +            CPUState *cs = CPU(cpu);
> +
> +            if (cs->halted) {
> +                val |= PPC_BIT(56 + i);
> +            }
> +        }
> +        break;
> +    case PNV10_XSCOM_EC_CORE_THREAD_INFO:
> +        break;
> +    case PNV10_XSCOM_EC_CORE_RAS_STATUS:
> +        for (i = 0; i < nr_threads; i++) {
> +            PowerPCCPU *cpu = pc->threads[i];
> +            CPUState *cs = CPU(cpu);
> +            if (cs->stopped) {
> +                val |= PPC_BIT(0 + 8*i) | PPC_BIT(1 + 8*i);
> +            }
> +        }
>           break;
>       default:
>           qemu_log_mask(LOG_UNIMP, "%s: unimp read 0x%08x\n", __func__,
> @@ -206,9 +230,45 @@ static uint64_t pnv_core_power10_xscom_read(void *opaque, hwaddr addr,
>   static void pnv_core_power10_xscom_write(void *opaque, hwaddr addr,
>                                            uint64_t val, unsigned int width)
>   {
> +    PnvCore *pc = PNV_CORE(opaque);
> +    int nr_threads = CPU_CORE(pc)->nr_threads;
> +    int i;
>       uint32_t offset = addr >> 3;
>   
>       switch (offset) {
> +    case PNV10_XSCOM_EC_CORE_DIRECT_CONTROLS:
> +        for (i = 0; i < nr_threads; i++) {
> +            PowerPCCPU *cpu = pc->threads[i];
> +            CPUState *cs = CPU(cpu);
> +
> +            if (val & PPC_BIT(7 + 8*i)) { /* stop */
> +                val &= ~PPC_BIT(7 + 8*i);
> +                cpu_pause(cs);
> +            }
> +            if (val & PPC_BIT(6 + 8*i)) { /* start */
> +                val &= ~PPC_BIT(6 + 8*i);
> +                cpu_resume(cs);
> +            }
> +            if (val & PPC_BIT(4 + 8*i)) { /* sreset */
> +                val &= ~PPC_BIT(4 + 8*i);
> +                pnv_cpu_do_nmi(cs);
> +            }
> +            if (val & PPC_BIT(3 + 8*i)) { /* clear maint */
> +                /*
> +                 * Hardware has very particular cases for where clear maint
> +                 * must be used and where start must be used to resume a
> +                 * thread. These are not modelled exactly, just treat
> +                 * this and start the same.
> +                 */
> +                val &= ~PPC_BIT(3 + 8*i);
> +                cpu_resume(cs);
> +            }
> +        }
> +        if (val) {
> +            qemu_log_mask(LOG_UNIMP, "%s: unimp bits in DIRECT_CONTROLS 0x%016lx\n", __func__, val);
> +        }
> +        break;
> +
>       default:
>           qemu_log_mask(LOG_UNIMP, "%s: unimp write 0x%08x\n", __func__,
>                         offset);
> @@ -525,6 +585,7 @@ static const MemoryRegionOps pnv_quad_power10_xscom_ops = {
>   static uint64_t pnv_qme_power10_xscom_read(void *opaque, hwaddr addr,
>                                               unsigned int width)
>   {
> +    PnvQuad *eq = PNV_QUAD(opaque);
>       uint32_t offset = addr >> 3;
>       uint64_t val = -1;
>   
> @@ -532,10 +593,14 @@ static uint64_t pnv_qme_power10_xscom_read(void *opaque, hwaddr addr,
>        * Forth nibble selects the core within a quad, mask it to process read
>        * for any core.
>        */
> -    switch (offset & ~0xf000) {
> -    case P10_QME_SPWU_HYP:
> +    switch (offset & ~PPC_BITMASK32(16, 19)) {
>       case P10_QME_SSH_HYP:
> -        return 0;
> +        val = 0;
> +        if (eq->special_wakeup_done) {
> +            val |= PPC_BIT(1); /* SPWU DONE */
> +            val |= PPC_BIT(4); /* SSH SPWU DONE */
> +        }
> +        break;
>       default:
>           qemu_log_mask(LOG_UNIMP, "%s: unimp read 0x%08x\n", __func__,
>                         offset);
> @@ -547,9 +612,22 @@ static uint64_t pnv_qme_power10_xscom_read(void *opaque, hwaddr addr,
>   static void pnv_qme_power10_xscom_write(void *opaque, hwaddr addr,
>                                            uint64_t val, unsigned int width)
>   {
> +    PnvQuad *eq = PNV_QUAD(opaque);
>       uint32_t offset = addr >> 3;
> +    bool set;
> +    int i;
>   
> -    switch (offset) {
> +    switch (offset & ~PPC_BITMASK32(16, 19)) {
> +    case P10_QME_SPWU_HYP:
> +        set = !!(val & PPC_BIT(0));
> +        eq->special_wakeup_done = set;
> +        for (i = 0; i < 4; i++) {
> +            /* These bits select cores in the quad */
> +            if (offset & PPC_BIT32(16 + i)) {
> +                eq->special_wakeup[i] = set;
> +            }
> +        }
> +        break;
>       default:
>           qemu_log_mask(LOG_UNIMP, "%s: unimp write 0x%08x\n", __func__,
>                         offset);
> diff --git a/system/cpus.c b/system/cpus.c
> index 68d161d96b..1ca18ae7ea 100644
> --- a/system/cpus.c
> +++ b/system/cpus.c
> @@ -602,6 +602,16 @@ void pause_all_vcpus(void)
>       bql_lock();
>   }
>   
> +void cpu_pause(CPUState *cpu)
> +{
> +    if (qemu_cpu_is_self(cpu)) {
> +        qemu_cpu_stop(cpu, true);
> +    } else {
> +        cpu->stop = true;
> +        qemu_cpu_kick(cpu);
> +    }
> +}
> +
>   void cpu_resume(CPUState *cpu)
>   {
>       cpu->stop = false;

Re: [RFC PATCH 09/10] ppc/pnv: Implement POWER10 PC xscom registers for direct controls

[Nicholas Piggin]

On Wed May 29, 2024 at 5:00 PM AEST, Cédric Le Goater wrote:
> On 5/26/24 14:26, Nicholas Piggin wrote:
> > The PC unit in the processor core contains xscom registers that provide
> > low level status and control of the CPU.
> > 
> > This implements "direct controls" sufficient for OPAL (skiboot) firmware
> > use, which is to stop threads and send them non-maskable IPIs in the
> > form of SRESET interrupts.
> > 
> > POWER10 is sufficiently different (particularly QME and special wakeup)
> > from POWER9 that it is not trivial to implement by reusing the code.
> > 
> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> > ---
> >   include/hw/core/cpu.h     |  8 ++++
> >   include/hw/ppc/pnv.h      |  2 +
> >   include/hw/ppc/pnv_core.h |  3 ++
> >   hw/ppc/pnv.c              |  7 +++-
> >   hw/ppc/pnv_core.c         | 88 ++++++++++++++++++++++++++++++++++++---
> >   system/cpus.c             | 10 +++++
> >   6 files changed, 112 insertions(+), 6 deletions(-)
> > 
> > diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
> > index bb398e8237..52a8fc65cb 100644
> > --- a/include/hw/core/cpu.h
> > +++ b/include/hw/core/cpu.h
> > @@ -974,6 +974,14 @@ void cpu_reset_interrupt(CPUState *cpu, int mask);
> >    */
> >   void cpu_exit(CPUState *cpu);
> >   
> > +/**
> > + * cpu_pause:
> > + * @cpu: The CPU to pause.
> > + *
> > + * Resumes CPU, i.e. puts CPU into stopped state.
> > + */
> > +void cpu_pause(CPUState *cpu);
> > +
> >   /**
> >    * cpu_resume:
> >    * @cpu: The CPU to resume.
> > diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
> > index 93ecb062b4..bec603f1a8 100644
> > --- a/include/hw/ppc/pnv.h
> > +++ b/include/hw/ppc/pnv.h
> > @@ -111,6 +111,8 @@ PnvChip *pnv_chip_add_phb(PnvChip *chip, PnvPHB *phb);
> >   #define PNV_FDT_ADDR          0x01000000
> >   #define PNV_TIMEBASE_FREQ     512000000ULL
> >   
> > +void pnv_cpu_do_nmi(CPUState *cs);
> > +
> >   /*
> >    * BMC helpers
> >    */
> > diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
> > index 39f8f33e6c..9599da15ea 100644
> > --- a/include/hw/ppc/pnv_core.h
> > +++ b/include/hw/ppc/pnv_core.h
> > @@ -109,6 +109,9 @@ OBJECT_DECLARE_TYPE(PnvQuad, PnvQuadClass, PNV_QUAD)
> >   struct PnvQuad {
> >       DeviceState parent_obj;
> >   
> > +    bool special_wakeup_done;
> > +    bool special_wakeup[4];
> > +
> >       uint32_t quad_id;
> >       MemoryRegion xscom_regs;
> >       MemoryRegion xscom_qme_regs;
> > diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
> > index 5364c55bbb..765142965f 100644
> > --- a/hw/ppc/pnv.c
> > +++ b/hw/ppc/pnv.c
> > @@ -2700,12 +2700,17 @@ static void pnv_cpu_do_nmi_on_cpu(CPUState *cs, run_on_cpu_data arg)
> >       }
> >   }
> >   
> > +void pnv_cpu_do_nmi(CPUState *cs)
> > +{
> > +    async_run_on_cpu(cs, pnv_cpu_do_nmi_on_cpu, RUN_ON_CPU_NULL);
> > +}
> > +
> >   static void pnv_nmi(NMIState *n, int cpu_index, Error **errp)
> >   {
> >       CPUState *cs;
> >   
> >       CPU_FOREACH(cs) {
> > -        async_run_on_cpu(cs, pnv_cpu_do_nmi_on_cpu, RUN_ON_CPU_NULL);
> > +        pnv_cpu_do_nmi(cs);
> >       }
> >   }
>
> What about ?
>
> https://lore.kernel.org/qemu-devel/20240424093048.180966-1-clg@redhat.com/

I haven't forgotten it. I just didn't put it in the first PR since
there was quite a lot of pnv patches to do so I thought I will collect
most of them for another PR.

Thanks,
Nick

[RFC PATCH 10/10] ppc/pnv: Add an LPAR per core machine option

[Nicholas Piggin]

Recent POWER CPUs can operate in "LPAR per core" or "LPAR per thread"
modes. In per-core mode, some SPRs and IPI doorbells are shared between
threads in a core. In per-thread mode, supervisor and user state is
not shared between threads.

OpenPOWER systems use LPAR per thread mode as it is required for KVM.
Enterprise systems use LPAR per core mode, as they partition the
machine by core.

Implement a lpar-per-core machine option for powernv machines. This
is supported on POWER9 and POWER10.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 include/hw/ppc/pnv.h      |  1 +
 include/hw/ppc/pnv_core.h |  1 +
 hw/ppc/pnv.c              | 29 +++++++++++++++++++++++++++++
 hw/ppc/pnv_core.c         |  8 ++++++++
 target/ppc/cpu_init.c     |  3 ++-
 5 files changed, 41 insertions(+), 1 deletion(-)

diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
index bec603f1a8..8f75c715d8 100644
--- a/include/hw/ppc/pnv.h
+++ b/include/hw/ppc/pnv.h
@@ -103,6 +103,7 @@ struct PnvMachineState {
 
     bool         big_core;
     bool         big_core_tbst_quirk;
+    bool         lpar_per_core;
 };
 
 PnvChip *pnv_get_chip(PnvMachineState *pnv, uint32_t chip_id);
diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
index 9599da15ea..e41b6347ea 100644
--- a/include/hw/ppc/pnv_core.h
+++ b/include/hw/ppc/pnv_core.h
@@ -57,6 +57,7 @@ struct PnvCore {
     /*< public >*/
     PowerPCCPU **threads;
     bool big_core;
+    bool lpar_per_core;
     uint32_t pir;
     uint32_t hwid;
     uint64_t hrmor;
diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
index 765142965f..0d830ad731 100644
--- a/hw/ppc/pnv.c
+++ b/hw/ppc/pnv.c
@@ -1108,12 +1108,17 @@ static void pnv_init(MachineState *machine)
 
 static void pnv_power8_init(MachineState *machine)
 {
+    PnvMachineState *pnv = PNV_MACHINE(machine);
+
     if (machine->smp.threads > 8) {
         error_report("Cannot support more than 8 threads/core "
                      "on a powernv POWER8 machine");
         exit(1);
     }
 
+    /* POWER8 is always lpar-per-core. */
+    pnv->lpar_per_core = true;
+
     pnv_init(machine);
 }
 
@@ -2541,6 +2546,18 @@ static void pnv_machine_set_big_core(Object *obj, bool value, Error **errp)
     pnv->big_core = value;
 }
 
+static bool pnv_machine_get_1lpar(Object *obj, Error **errp)
+{
+    PnvMachineState *pnv = PNV_MACHINE(obj);
+    return pnv->lpar_per_core;
+}
+
+static void pnv_machine_set_1lpar(Object *obj, bool value, Error **errp)
+{
+    PnvMachineState *pnv = PNV_MACHINE(obj);
+    pnv->lpar_per_core = value;
+}
+
 static bool pnv_machine_get_hb(Object *obj, Error **errp)
 {
     PnvMachineState *pnv = PNV_MACHINE(obj);
@@ -2614,6 +2631,12 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
                                    pnv_machine_set_big_core);
     object_class_property_set_description(oc, "big-core",
                               "Use big-core (aka fused-core) mode");
+
+    object_class_property_add_bool(oc, "lpar-per-core",
+                                   pnv_machine_get_1lpar,
+                                   pnv_machine_set_1lpar);
+    object_class_property_set_description(oc, "lpar-per-core",
+                              "Use 1 LPAR per core mode");
 }
 
 static void pnv_machine_p10_common_class_init(ObjectClass *oc, void *data)
@@ -2660,6 +2683,12 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
                                    pnv_machine_set_big_core);
     object_class_property_set_description(oc, "big-core",
                               "Use big-core (aka fused-core) mode");
+
+    object_class_property_add_bool(oc, "lpar-per-core",
+                                   pnv_machine_get_1lpar,
+                                   pnv_machine_set_1lpar);
+    object_class_property_set_description(oc, "lpar-per-core",
+                              "Use 1 LPAR per core mode");
 }
 
 static void pnv_machine_p10_rainier_class_init(ObjectClass *oc, void *data)
diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
index 835c35d90b..e510909db1 100644
--- a/hw/ppc/pnv_core.c
+++ b/hw/ppc/pnv_core.c
@@ -207,6 +207,9 @@ static uint64_t pnv_core_power10_xscom_read(void *opaque, hwaddr addr,
                 val |= PPC_BIT(56 + i);
             }
         }
+        if (pc->lpar_per_core) {
+            val |= PPC_BIT(62);
+        }
         break;
     case PNV10_XSCOM_EC_CORE_THREAD_INFO:
         break;
@@ -322,6 +325,11 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
         env->core_index = core_hwid;
     }
 
+    if (pnv->lpar_per_core) {
+        pc->lpar_per_core = true;
+        cpu_ppc_set_1lpar(cpu);
+    }
+
     /* Set time-base frequency to 512 MHz */
     cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
 }
diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
index e71ee008ed..f0065c2e37 100644
--- a/target/ppc/cpu_init.c
+++ b/target/ppc/cpu_init.c
@@ -6785,7 +6785,8 @@ void cpu_ppc_set_1lpar(PowerPCCPU *cpu)
 
     /*
      * pseries SMT means "LPAR per core" mode, e.g., msgsndp is usable
-     * between threads.
+     * between threads. powernv be in either mode, and it mostly affects
+     * supervisor visible registers and instructions.
      */
     if (env->flags & POWERPC_FLAG_SMT) {
         env->flags |= POWERPC_FLAG_SMT_1LPAR;
-- 
2.43.0

Re: [RFC PATCH 10/10] ppc/pnv: Add an LPAR per core machine option

[Cédric Le Goater]

On 5/26/24 14:26, Nicholas Piggin wrote:
> Recent POWER CPUs can operate in "LPAR per core" or "LPAR per thread"
> modes. In per-core mode, some SPRs and IPI doorbells are shared between
> threads in a core. In per-thread mode, supervisor and user state is
> not shared between threads.
> 
> OpenPOWER systems use LPAR per thread mode as it is required for KVM.
> Enterprise systems use LPAR per core mode, as they partition the
> machine by core.
> 
> Implement a lpar-per-core machine option for powernv machines. This
> is supported on POWER9 and POWER10.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>

LGTM,


Reviewed-by: Cédric Le Goater <clg@redhat.com>

Thanks,

C.


> ---
>   include/hw/ppc/pnv.h      |  1 +
>   include/hw/ppc/pnv_core.h |  1 +
>   hw/ppc/pnv.c              | 29 +++++++++++++++++++++++++++++
>   hw/ppc/pnv_core.c         |  8 ++++++++
>   target/ppc/cpu_init.c     |  3 ++-
>   5 files changed, 41 insertions(+), 1 deletion(-)
> 
> diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
> index bec603f1a8..8f75c715d8 100644
> --- a/include/hw/ppc/pnv.h
> +++ b/include/hw/ppc/pnv.h
> @@ -103,6 +103,7 @@ struct PnvMachineState {
>   
>       bool         big_core;
>       bool         big_core_tbst_quirk;
> +    bool         lpar_per_core;
>   };
>   
>   PnvChip *pnv_get_chip(PnvMachineState *pnv, uint32_t chip_id);
> diff --git a/include/hw/ppc/pnv_core.h b/include/hw/ppc/pnv_core.h
> index 9599da15ea..e41b6347ea 100644
> --- a/include/hw/ppc/pnv_core.h
> +++ b/include/hw/ppc/pnv_core.h
> @@ -57,6 +57,7 @@ struct PnvCore {
>       /*< public >*/
>       PowerPCCPU **threads;
>       bool big_core;
> +    bool lpar_per_core;
>       uint32_t pir;
>       uint32_t hwid;
>       uint64_t hrmor;
> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
> index 765142965f..0d830ad731 100644
> --- a/hw/ppc/pnv.c
> +++ b/hw/ppc/pnv.c
> @@ -1108,12 +1108,17 @@ static void pnv_init(MachineState *machine)
>   
>   static void pnv_power8_init(MachineState *machine)
>   {
> +    PnvMachineState *pnv = PNV_MACHINE(machine);
> +
>       if (machine->smp.threads > 8) {
>           error_report("Cannot support more than 8 threads/core "
>                        "on a powernv POWER8 machine");
>           exit(1);
>       }
>   
> +    /* POWER8 is always lpar-per-core. */
> +    pnv->lpar_per_core = true;
> +
>       pnv_init(machine);
>   }
>   
> @@ -2541,6 +2546,18 @@ static void pnv_machine_set_big_core(Object *obj, bool value, Error **errp)
>       pnv->big_core = value;
>   }
>   
> +static bool pnv_machine_get_1lpar(Object *obj, Error **errp)
> +{
> +    PnvMachineState *pnv = PNV_MACHINE(obj);
> +    return pnv->lpar_per_core;
> +}
> +
> +static void pnv_machine_set_1lpar(Object *obj, bool value, Error **errp)
> +{
> +    PnvMachineState *pnv = PNV_MACHINE(obj);
> +    pnv->lpar_per_core = value;
> +}
> +
>   static bool pnv_machine_get_hb(Object *obj, Error **errp)
>   {
>       PnvMachineState *pnv = PNV_MACHINE(obj);
> @@ -2614,6 +2631,12 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
>                                      pnv_machine_set_big_core);
>       object_class_property_set_description(oc, "big-core",
>                                 "Use big-core (aka fused-core) mode");
> +
> +    object_class_property_add_bool(oc, "lpar-per-core",
> +                                   pnv_machine_get_1lpar,
> +                                   pnv_machine_set_1lpar);
> +    object_class_property_set_description(oc, "lpar-per-core",
> +                              "Use 1 LPAR per core mode");
>   }
>   
>   static void pnv_machine_p10_common_class_init(ObjectClass *oc, void *data)
> @@ -2660,6 +2683,12 @@ static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
>                                      pnv_machine_set_big_core);
>       object_class_property_set_description(oc, "big-core",
>                                 "Use big-core (aka fused-core) mode");
> +
> +    object_class_property_add_bool(oc, "lpar-per-core",
> +                                   pnv_machine_get_1lpar,
> +                                   pnv_machine_set_1lpar);
> +    object_class_property_set_description(oc, "lpar-per-core",
> +                              "Use 1 LPAR per core mode");
>   }
>   
>   static void pnv_machine_p10_rainier_class_init(ObjectClass *oc, void *data)
> diff --git a/hw/ppc/pnv_core.c b/hw/ppc/pnv_core.c
> index 835c35d90b..e510909db1 100644
> --- a/hw/ppc/pnv_core.c
> +++ b/hw/ppc/pnv_core.c
> @@ -207,6 +207,9 @@ static uint64_t pnv_core_power10_xscom_read(void *opaque, hwaddr addr,
>                   val |= PPC_BIT(56 + i);
>               }
>           }
> +        if (pc->lpar_per_core) {
> +            val |= PPC_BIT(62);
> +        }
>           break;
>       case PNV10_XSCOM_EC_CORE_THREAD_INFO:
>           break;
> @@ -322,6 +325,11 @@ static void pnv_core_cpu_realize(PnvCore *pc, PowerPCCPU *cpu, Error **errp,
>           env->core_index = core_hwid;
>       }
>   
> +    if (pnv->lpar_per_core) {
> +        pc->lpar_per_core = true;
> +        cpu_ppc_set_1lpar(cpu);
> +    }
> +
>       /* Set time-base frequency to 512 MHz */
>       cpu_ppc_tb_init(env, PNV_TIMEBASE_FREQ);
>   }
> diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
> index e71ee008ed..f0065c2e37 100644
> --- a/target/ppc/cpu_init.c
> +++ b/target/ppc/cpu_init.c
> @@ -6785,7 +6785,8 @@ void cpu_ppc_set_1lpar(PowerPCCPU *cpu)
>   
>       /*
>        * pseries SMT means "LPAR per core" mode, e.g., msgsndp is usable
> -     * between threads.
> +     * between threads. powernv be in either mode, and it mostly affects
> +     * supervisor visible registers and instructions.
>        */
>       if (env->flags & POWERPC_FLAG_SMT) {
>           env->flags |= POWERPC_FLAG_SMT_1LPAR;

Re: [RFC PATCH 00/10] ppc/pnv: Better big-core model, lpar-per-core, PC unit

[Cédric Le Goater]

On 5/26/24 14:26, Nicholas Piggin wrote:
> Primary motivation for this series is to improve big-core support.
> Other things like SPR indirect, timebase state, PC xscom, are required
> for minimal big core support.
> 
> I'm still not 100% happy with the big-core topology model after this.
> Maybe one day we add pnv big core and pnv small core structures. But

I haven't look at the proposal yet, but indeed, we could introduce
a new TYPE_PNV_CORE type for big cores only.

> nothing is completely clean because big core mode still has certain
> small core restrictions. I think for now we take a bit of mostly
> abstracted ugliness in TCG code for the benefit of not spreading
> hacks through pervasive (xscom) core addressing.
> 
> After this series, power9 and power10 get through skiboot/Linux boot
s

Have you tried SMT8 on powernv8 ? I remember seeing a hang if I am correct.
I don't think POWER8 deserves much attention anymore, we could deprecate
POWER8E and POWER8NVL. However, we should at least report an error if we
know a setup is broken.

Thanks,

C.


> Not all big core registers are modeled
> exactly (some are not shared between small core halves), but that
> mostly doesn't matter for OPAL and it can be improved later.
> 
> Thanks,
> Nick
> 
> Nicholas Piggin (10):
>    ppc/pnv: Add pointer from PnvCPUState to PnvCore
>    ppc/pnv: Move timebase state into PnvCore
>    target/ppc: Improve SPR indirect registers
>    ppc/pnv: specialise init for powernv8/9/10 machines
>    ppc/pnv: Extend chip_pir class method to TIR as well
>    ppc: Add a core_index to CPUPPCState for SMT vCPUs
>    target/ppc: Add helpers to check for SMT sibling threads
>    ppc/pnv: Invert the design for big-core machine modelling
>    ppc/pnv: Implement POWER10 PC xscom registers for direct controls
>    ppc/pnv: Add an LPAR per core machine option
> 
>   include/hw/core/cpu.h        |   8 +
>   include/hw/ppc/pnv.h         |   6 +
>   include/hw/ppc/pnv_chip.h    |   3 +-
>   include/hw/ppc/pnv_core.h    |  31 ++++
>   target/ppc/cpu.h             |  37 ++---
>   hw/ppc/pnv.c                 | 297 ++++++++++++++++++++++++++++-------
>   hw/ppc/pnv_chiptod.c         |   6 +-
>   hw/ppc/pnv_core.c            | 129 +++++++++++++--
>   hw/ppc/spapr_cpu_core.c      |   7 +
>   system/cpus.c                |  10 ++
>   target/ppc/cpu_init.c        |  26 +--
>   target/ppc/excp_helper.c     |  16 +-
>   target/ppc/misc_helper.c     |  98 ++++++------
>   target/ppc/timebase_helper.c |  82 +++++-----
>   14 files changed, 548 insertions(+), 208 deletions(-)
> 

Re: [RFC PATCH 00/10] ppc/pnv: Better big-core model, lpar-per-core, PC unit

[Nicholas Piggin]

On Mon May 27, 2024 at 4:25 PM AEST, Cédric Le Goater wrote:
> On 5/26/24 14:26, Nicholas Piggin wrote:
> > Primary motivation for this series is to improve big-core support.
> > Other things like SPR indirect, timebase state, PC xscom, are required
> > for minimal big core support.
> > 
> > I'm still not 100% happy with the big-core topology model after this.
> > Maybe one day we add pnv big core and pnv small core structures. But
>
> I haven't look at the proposal yet, but indeed, we could introduce
> a new TYPE_PNV_CORE type for big cores only.

Yeah. It's still tricky because big-core structure contains the CPUs
if you are running in small core mode. So it would really have to be
a PnvCPUCore and PnvPervasiveCore or something, where the former is
either SMT4 and 1:1 with the latter or SMT8 and 1:2 depending on mode.

And some of the "CPU" type operations in big core mode still need to
operate on the small core.

For now, the accessors and helpers seem to be not too bad.

> > nothing is completely clean because big core mode still has certain
> > small core restrictions. I think for now we take a bit of mostly
> > abstracted ugliness in TCG code for the benefit of not spreading
> > hacks through pervasive (xscom) core addressing.
> > 
> > After this series, power9 and power10 get through skiboot/Linux boot
> s
>
> Have you tried SMT8 on powernv8 ? I remember seeing a hang if I am correct.
> I don't think POWER8 deserves much attention anymore, we could deprecate
> POWER8E and POWER8NVL. However, we should at least report an error if we
> know a setup is broken.

Yeah it does have some problem. Maybe should just disable SMT unless
someone finds time to work it out.

Thanks,
Nick

Re: [RFC PATCH 00/10] ppc/pnv: Better big-core model, lpar-per-core, PC unit

[Cédric Le Goater]

On 5/27/24 09:32, Nicholas Piggin wrote:
> On Mon May 27, 2024 at 4:25 PM AEST, Cédric Le Goater wrote:
>> On 5/26/24 14:26, Nicholas Piggin wrote:
>>> Primary motivation for this series is to improve big-core support.
>>> Other things like SPR indirect, timebase state, PC xscom, are required
>>> for minimal big core support.
>>>
>>> I'm still not 100% happy with the big-core topology model after this.
>>> Maybe one day we add pnv big core and pnv small core structures. But
>>
>> I haven't look at the proposal yet, but indeed, we could introduce
>> a new TYPE_PNV_CORE type for big cores only.
> 
> Yeah. It's still tricky because big-core structure contains the CPUs
> if you are running in small core mode. So it would really have to be
> a PnvCPUCore and PnvPervasiveCore or something, where the former is
> either SMT4 and 1:1 with the latter or SMT8 and 1:2 depending on mode.
> 
> And some of the "CPU" type operations in big core mode still need to
> operate on the small core.
> 
> For now, the accessors and helpers seem to be not too bad.
> 
>>> nothing is completely clean because big core mode still has certain
>>> small core restrictions. I think for now we take a bit of mostly
>>> abstracted ugliness in TCG code for the benefit of not spreading
>>> hacks through pervasive (xscom) core addressing.
>>>
>>> After this series, power9 and power10 get through skiboot/Linux boot
>> s
>>
>> Have you tried SMT8 on powernv8 ? I remember seeing a hang if I am correct.
>> I don't think POWER8 deserves much attention anymore, we could deprecate
>> POWER8E and POWER8NVL. However, we should at least report an error if we
>> know a setup is broken.
> 
> Yeah it does have some problem. Maybe should just disable SMT unless
> someone finds time to work it out.


The new _init routines would be a good place to do that.

Thanks,

C.


> 
> Thanks,
> Nick