Re: [Qemu-devel] [PATCH for 2.13 v3 1/2] spapr: Add ibm, max-associativity-domains property

[Serhii Popovych <spopovyc@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 3257 bytes --]

David Gibson wrote:
> On Tue, Apr 17, 2018 at 09:28:42AM +0530, Bharata B Rao wrote:
>> On Mon, Apr 16, 2018 at 07:47:29PM +0300, Serhii Popovych wrote:
>>> Bharata B Rao wrote:
>>>> On Wed, Apr 11, 2018 at 02:41:59PM -0400, Serhii Popovych wrote:
>>>>> Now recent kernels (i.e. since linux-stable commit a346137e9142
>>>>> ("powerpc/numa: Use ibm,max-associativity-domains to discover possible nodes")
>>>>> support this property to mark initially memory-less NUMA nodes as "possible"
>>>>> to allow further memory hot-add to them.
>>>>>
>>>>> Advertise this property for pSeries machines to let guest kernels detect
>>>>> maximum supported node configuration and benefit from kernel side change
>>>>> when hot-add memory to specific, possibly empty before, NUMA node.
>>>>>
>>>>> Signed-off-by: Serhii Popovych <spopovyc@redhat.com>
>>>>> ---
>>>>>  hw/ppc/spapr.c | 10 ++++++++++
>>>>>  1 file changed, 10 insertions(+)
>>>>>
>>>>> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
>>>>> index a81570e..c05bbad 100644
>>>>> --- a/hw/ppc/spapr.c
>>>>> +++ b/hw/ppc/spapr.c
>>>>> @@ -910,6 +910,13 @@ static void spapr_dt_rtas(sPAPRMachineState *spapr, void *fdt)
>>>>>          0, cpu_to_be32(SPAPR_MEMORY_BLOCK_SIZE),
>>>>>          cpu_to_be32(max_cpus / smp_threads),
>>>>>      };
>>>>> +    uint32_t maxdomains[] = {
>>>>> +        cpu_to_be32(4),
>>>>> +        cpu_to_be32(0),
>>>>> +        cpu_to_be32(0),
>>>>> +        cpu_to_be32(0),
>>>>> +        cpu_to_be32(nb_numa_nodes - 1),
>>>>> +    };
>>>>>
>>>>>      _FDT(rtas = fdt_add_subnode(fdt, 0, "rtas"));
>>>>>
>>>>> @@ -946,6 +953,9 @@ static void spapr_dt_rtas(sPAPRMachineState *spapr, void *fdt)
>>>>>      _FDT(fdt_setprop(fdt, rtas, "ibm,associativity-reference-points",
>>>>>                       refpoints, sizeof(refpoints)));
>>>>>
>>>>> +    _FDT(fdt_setprop(fdt, rtas, "ibm,max-associativity-domains",
>>>>> +                     maxdomains, sizeof(maxdomains)));
>>>>> +
>>>>>      _FDT(fdt_setprop_cell(fdt, rtas, "rtas-error-log-max",
>>>>>                            RTAS_ERROR_LOG_MAX));
>>>>>      _FDT(fdt_setprop_cell(fdt, rtas, "rtas-event-scan-rate",
>>>>
>>>> This commit causes hash guest with latest guest kernel to hang at early boot.
>>>
>>> I use v4.16 tag from stable and can't reproduce on P8 machine reported
>>> issue.
>>>
>>> Could you please share more details about your setup, kernel commit id
>>> you spot problem?
>>
>> I am on 4.16.0-rc7 (commit id: 0b412605ef5f)
>>
>> BTW this happens only for non-NUMA guest.
> 
> Ah, that might explain it.  With no NUMA nodes specified, I think this
> code will put a -1 into the max-associativity-domains property, which
> is probably causing the mess.  If we don't have NUMA (nb_numa_nodes ==
> 0) we probably want to either omit the property entirely, or clamp
> that 5th cell to 0.
> 
Ok, proposed fix already posted to qemu-devel. Sorry forgot to CC.

Mail subject contains (w/o quotas):
"spapr: Correct max associativity domains value for non-NUMA configs"

Tested with v4.16 tag:

  o Before : non-NUMA configs: able to reproduce, stall during boot
  o After  : non-NUMA configs: not reproducible, boot is ok

-- 
Thanks,
Serhii


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 490 bytes --]