Re: [PATCH v4 4/5] ppc/pnv: Introduce user creatable pnv-phb4 devices

[Daniel Henrique Barboza <danielhb413@gmail.com>]

On 1/11/22 06:56, Cédric Le Goater wrote:
> On 1/11/22 01:55, Daniel Henrique Barboza wrote:
>> This patch introduces pnv-phb4 user creatable devices that are created
>> in a similar manner as pnv-phb3 devices, allowing the user to interact
>> with the PHBs directly instead of creating PCI Express Controllers that
>> will create a certain amount of PHBs per controller index.
>>
>> We accomplish this by doing the following:
>>
>> - add a pnv_phb4_get_stack() helper to retrieve which stack an user
>> created phb4 would occupy;
>>
>> - when dealing with an user created pnv-phb4 (detected by checking if
>> phb->stack is NULL at the start of phb4_realize()), retrieve its stack
>> and initialize its properties as done in stk_realize();
>>
>> - use 'defaults_enabled()' in stk_realize() to avoid creating and
>> initializing a 'stack->phb' qdev that might be overwritten by an user
>> created pnv-phb4 device.
>>
>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>> ---
>>   hw/pci-host/pnv_phb4.c     | 75 +++++++++++++++++++++++++++++++++++++-
>>   hw/pci-host/pnv_phb4_pec.c |  5 +++
>>   hw/ppc/pnv.c               |  2 +
>>   3 files changed, 80 insertions(+), 2 deletions(-)
>>
>> diff --git a/hw/pci-host/pnv_phb4.c b/hw/pci-host/pnv_phb4.c
>> index 3ffa8f51e9..10f8d6a919 100644
>> --- a/hw/pci-host/pnv_phb4.c
>> +++ b/hw/pci-host/pnv_phb4.c
>> @@ -1487,15 +1487,86 @@ static void pnv_phb4_instance_init(Object *obj)
>>       object_initialize_child(obj, "source", &phb->xsrc, TYPE_XIVE_SOURCE);
>>   }
>> +static PnvPhb4PecStack *pnv_phb4_get_stack(int chip_id, int index,
>> +                                           Error **errp)
>> +{
>> +    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
>> +    PnvChip *chip = pnv_get_chip(pnv, chip_id);
>> +    Pnv9Chip *chip9 = PNV9_CHIP(chip);
>> +    int i, j;
>> +
>> +    for (i = 0; i < chip->num_pecs; i++) {
>> +        /*
>> +         * For each PEC, check the amount of stacks it supports
>> +         * and see if the given phb4 index matches a stack.
>> +         */
>> +        PnvPhb4PecState *pec = &chip9->pecs[i];
>> +
>> +        for (j = 0; j < pec->num_stacks; j++) {
>> +            if (index == pnv_phb4_pec_get_phb_id(pec, j)) {
>> +                return &pec->stacks[j];
>> +            }
>> +        }
>> +    }
>> +
>> +    error_setg(errp,
>> +               "pnv-phb4 chip-id %d index %d didn't match any existing PEC",
>> +               chip_id, index);
>> +
>> +    return NULL;
>> +}
>> +
>>   static void pnv_phb4_realize(DeviceState *dev, Error **errp)
>>   {
>>       PnvPHB4 *phb = PNV_PHB4(dev);
>>       PCIHostState *pci = PCI_HOST_BRIDGE(dev);
>>       XiveSource *xsrc = &phb->xsrc;
>> +    Error *local_err = NULL;
>>       int nr_irqs;
>>       char name[32];
>> -    assert(phb->stack);
>> +    /* User created PHB */
>> +    if (!phb->stack) {
>> +        PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
>> +        PnvChip *chip = pnv_get_chip(pnv, phb->chip_id);
> 
> Arg. my comment on the pnv_phb4_get_stack() prototype was clearly
> a mistake ! We are calling twice pnv_get_chip() to get the chip
> and there is no reason for it.
> 
> Can you changed it back ? Sorry about that ... You can flame m
hehehe don't worry about. I'll change it back.

I'll also take the opportunity to test the changes proposed in patch 2 in my env
to make sure we're good there as well.


Thanks,


Daniel


> 
>> +        PnvPhb4PecClass *pecc;
>> +        BusState *s;
>> +
>> +        if (!chip) {
>> +            error_setg(errp, "invalid chip id: %d", phb->chip_id);
>> +            return;
>> +        }
>> +
>> +        phb->stack = pnv_phb4_get_stack(phb->chip_id, phb->phb_id,
>> +                                        &local_err);
>> +        if (local_err) {
>> +            error_propagate(errp, local_err);
>> +            return;
>> +        }
>> +
>> +        /* All other phb properties but 'version' are already set */
>> +        pecc = PNV_PHB4_PEC_GET_CLASS(phb->stack->pec);
>> +        object_property_set_int(OBJECT(phb), "version", pecc->version,
>> +                                &error_fatal);
> 
> Yes :/ This version is a constant and should probably be a class attribute.
> 
> I am not sure we need a property anymore. we could change pnv_phb4_reg_read()
> with :
>     switch (off) {
>     case PHB_VERSION:
>          return PNV_PHB4_PEC_GET_CLASS(phb->stack->pec)->version;
> 
> 
> Another small problem to handle is the root port type :
> 
>      /* Add a single Root port if running with defaults */
>      if (defaults_enabled()) {
>          pnv_phb_attach_root_port(PCI_HOST_BRIDGE(phb),
>                                   TYPE_PNV_PHB4_ROOT_PORT);
>      }
> 
> 
> With PHB5, we should use another type. This can come later.
> 
>> +
>> +        /*
>> +         * Assign stack->phb since pnv_phb4_update_regions() uses it
>> +         * to access the phb.
>> +         */
>> +        phb->stack->phb = phb;
>> +
>> +        /*
>> +         * Reparent user created devices to the chip to build
>> +         * correctly the device tree.
>> +         */
>> +        pnv_chip_parent_fixup(chip, OBJECT(phb), phb->phb_id);
>> +
>> +        s = qdev_get_parent_bus(DEVICE(chip));
>> +        if (!qdev_set_parent_bus(DEVICE(phb), s, &local_err)) {
>> +            error_propagate(errp, local_err);
>> +            return;
>> +        }
>> +    }
>>       pnv_phb4_XSCOM_init(phb);
>> @@ -1600,7 +1671,7 @@ static void pnv_phb4_class_init(ObjectClass *klass, void *data)
>>       dc->realize         = pnv_phb4_realize;
>>       device_class_set_props(dc, pnv_phb4_properties);
>>       set_bit(DEVICE_CATEGORY_BRIDGE, dc->categories);
>> -    dc->user_creatable  = false;
>> +    dc->user_creatable  = true;
>>       xfc->notify         = pnv_phb4_xive_notify;
>>   }
>> diff --git a/hw/pci-host/pnv_phb4_pec.c b/hw/pci-host/pnv_phb4_pec.c
>> index d4c52a5d28..dfd25831d5 100644
>> --- a/hw/pci-host/pnv_phb4_pec.c
>> +++ b/hw/pci-host/pnv_phb4_pec.c
>> @@ -19,6 +19,7 @@
>>   #include "hw/pci/pci_bus.h"
>>   #include "hw/ppc/pnv.h"
>>   #include "hw/qdev-properties.h"
>> +#include "sysemu/sysemu.h"
>>   #include <libfdt.h>
>> @@ -282,6 +283,10 @@ static void pnv_pec_stk_realize(DeviceState *dev, Error **errp)
>>       PnvPhb4PecClass *pecc = PNV_PHB4_PEC_GET_CLASS(pec);
>>       int phb_id = pnv_phb4_pec_get_phb_id(pec, stack->stack_no);
>> +    if (!defaults_enabled()) {> +        return;
> 
> May be move all the code doing the PHB4 device realize in a helper routine
> and call it under if (defaults_enabled()) { ... }
> 
> Thanks,
> 
> C.
> 
> 
>> +    }
>> +
>>       stack->phb = PNV_PHB4(qdev_new(TYPE_PNV_PHB4));
>>
>>       object_property_set_int(OBJECT(stack->phb), "chip-id", pec->chip_id,
>> diff --git a/hw/ppc/pnv.c b/hw/ppc/pnv.c
>> index fe7e67e73a..837146a2fb 100644
>> --- a/hw/ppc/pnv.c
>> +++ b/hw/ppc/pnv.c
>> @@ -1960,6 +1960,8 @@ static void pnv_machine_power9_class_init(ObjectClass *oc, void *data)
>>       pmc->compat = compat;
>>       pmc->compat_size = sizeof(compat);
>>       pmc->dt_power_mgt = pnv_dt_power_mgt;
>> +
>> +    machine_class_allow_dynamic_sysbus_dev(mc, TYPE_PNV_PHB4);
>>   }
>>   static void pnv_machine_power10_class_init(ObjectClass *oc, void *data)
>>
>