offline CPUs shown in sysfs core_siblings mask

offline CPUs shown in sysfs core_siblings mask

[Brice Goglin]

Hello

We have a report from a user of hwloc/lstopo on ThunderX2 that complains
that offline CPUs are shown in sysfs cpu core_siblings files.

$ cat sys/devices/system/cpu/*/topology/core_siblings
000000,0fffffff
000000,0fffffff
000000,0fffffff
000000,0fffffff
ffffff,f0000000
ffffff,f0000000
ffffff,f0000000
ffffff,f0000000

Only 8 online logicial CPUs, but 56 are shown in these masks. This is on
RHEL7 with a kernel 4.14.0-115.2.2.el7a.aarch64 but I couldn't find any
significant change in Linux git.

On x86, only online CPUs are shown, you would get:

$ cat sys/devices/system/cpu/*/topology/core_siblings
000000,0000000f
000000,0000000f
000000,0000000f
000000,0000000f
000000,f0000000
000000,f0000000
000000,f0000000
000000,f0000000

I guess thread_siblings has the same behavior but I couldn't test it.

Do you ever plan to align the ARM behavior with x86? I couldn't find
anything in the doc stating whether offline CPUs are supposed to be
shown there, but it would be nice to have the same behavior across
different architectures. I couldn't find any such code for POWER either.

Brice



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: offline CPUs shown in sysfs core_siblings mask

[James Morse]

Hi Brice,

(CC: +Jeremy)

On 9/11/19 2:23 PM, Brice Goglin wrote:
> We have a report from a user of hwloc/lstopo on ThunderX2 that complains
> that offline CPUs are shown in sysfs cpu core_siblings files.

Hmmm, this doesn't happen on my TX2, running mainline:
root@eglon:/sys/devices/system/cpu# cat cpu1/topology/core_siblings
00000000,00000000,00000000,00000000,ffffffff,ffffffff,ffffffff,ffffffff
root@eglon:/sys/devices/system/cpu# echo 0 > cpu0/online
root@eglon:/sys/devices/system/cpu# cat cpu1/topology/core_siblings
00000000,00000000,00000000,00000000,ffffffff,ffffffff,ffffffff,fffffffe
root@eglon:/sys/devices/system/cpu# echo 1 > cpu0/online
root@eglon:/sys/devices/system/cpu# cat cpu1/topology/core_siblings
00000000,00000000,00000000,00000000,ffffffff,ffffffff,ffffffff,ffffffff


> Only 8 online logicial CPUs, but 56 are shown in these masks. This is on
> RHEL7 with a kernel 4.14.0-115.2.2.el7a.aarch64 but I couldn't find any
> significant change in Linux git.

Could you try a recent mainline kernel?

Does your system have an ACPI PPTT table? (I assume its ACPI)

Prior to the ACPI PPTT table handling, it wasn't possible for an arm64 
ACPI system to know about packages and threads.


> I guess thread_siblings has the same behavior but I couldn't test it.

(this would depend on the firmware table too)


Thanks,

James

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: offline CPUs shown in sysfs core_siblings mask

[Sudeep Holla]

On Wed, Sep 11, 2019 at 05:06:24PM +0100, James Morse wrote:
> Hi Brice,
> 
> (CC: +Jeremy)
> 
> On 9/11/19 2:23 PM, Brice Goglin wrote:
> > We have a report from a user of hwloc/lstopo on ThunderX2 that complains
> > that offline CPUs are shown in sysfs cpu core_siblings files.
> 
> Hmmm, this doesn't happen on my TX2, running mainline:
> root@eglon:/sys/devices/system/cpu# cat cpu1/topology/core_siblings
> 00000000,00000000,00000000,00000000,ffffffff,ffffffff,ffffffff,ffffffff
> root@eglon:/sys/devices/system/cpu# echo 0 > cpu0/online
> root@eglon:/sys/devices/system/cpu# cat cpu1/topology/core_siblings
> 00000000,00000000,00000000,00000000,ffffffff,ffffffff,ffffffff,fffffffe
> root@eglon:/sys/devices/system/cpu# echo 1 > cpu0/online
> root@eglon:/sys/devices/system/cpu# cat cpu1/topology/core_siblings
> 00000000,00000000,00000000,00000000,ffffffff,ffffffff,ffffffff,ffffffff
> 

Commit 7f9545aa1a91 ("arm64: smp: remove cpu and numa topology information when hotplugging out CPU")
is the patch that introduced that changes to update the sibling masks when
CPUs are hotplugged out. Before that patch the behaviour was same as you
describe. AFAIK, they were not marked stable and never backported.

> 
> > Only 8 online logicial CPUs, but 56 are shown in these masks. This is on
> > RHEL7 with a kernel 4.14.0-115.2.2.el7a.aarch64 but I couldn't find any
> > significant change in Linux git.
> 
> Could you try a recent mainline kernel?
> 
> Does your system have an ACPI PPTT table? (I assume its ACPI)
> 
> Prior to the ACPI PPTT table handling, it wasn't possible for an arm64 ACPI
> system to know about packages and threads.
>

Indeed yes, but not sure if that fixes the above mentioned issue.

> 
> > I guess thread_siblings has the same behavior but I couldn't test it.
> 
> (this would depend on the firmware table too)
>

If the above commit doesn't fix it, then it's a bug. I didn't test them on
SMT systems for sure.

--
Regards,
Sudeep

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: offline CPUs shown in sysfs core_siblings mask

[Brice Goglin]

Le 11/09/2019 à 18:06, James Morse a écrit :
> On 9/11/19 2:23 PM, Brice Goglin wrote:
>> We have a report from a user of hwloc/lstopo on ThunderX2 that complains
>> that offline CPUs are shown in sysfs cpu core_siblings files.
>
> Hmmm, this doesn't happen on my TX2, running mainline:
> root@eglon:/sys/devices/system/cpu# cat cpu1/topology/core_siblings
> 00000000,00000000,00000000,00000000,ffffffff,ffffffff,ffffffff,ffffffff
> root@eglon:/sys/devices/system/cpu# echo 0 > cpu0/online
> root@eglon:/sys/devices/system/cpu# cat cpu1/topology/core_siblings
> 00000000,00000000,00000000,00000000,ffffffff,ffffffff,ffffffff,fffffffe
> root@eglon:/sys/devices/system/cpu# echo 1 > cpu0/online
> root@eglon:/sys/devices/system/cpu# cat cpu1/topology/core_siblings
> 00000000,00000000,00000000,00000000,ffffffff,ffffffff,ffffffff,ffffffff
>
>
>> Only 8 online logicial CPUs, but 56 are shown in these masks. This is on
>> RHEL7 with a kernel 4.14.0-115.2.2.el7a.aarch64 but I couldn't find any
>> significant change in Linux git.
>
> Could you try a recent mainline kernel?
>
> Does your system have an ACPI PPTT table? (I assume its ACPI)
>
> Prior to the ACPI PPTT table handling, it wasn't possible for an arm64
> ACPI system to know about packages and threads.
>

Yes the machine has an ACPI PPTT table. I am working with admins to
verify that recent kernels with 7f9545aa1a91 work better.

Thanks

Brice



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel