public inbox for linux-arm-kernel@lists.infradead.org
 help / color / mirror / Atom feed
From: Jinjie Ruan <ruanjinjie@huawei.com>
To: Catalin Marinas <catalin.marinas@arm.com>
Cc: Thomas Gleixner <tglx@kernel.org>, <peterz@infradead.org>,
	<sudeep.holla@kernel.org>, <yangyicong@hisilicon.com>,
	<dietmar.eggemann@arm.com>, Jonathan Cameron <jic23@kernel.org>,
	<linux-kernel@vger.kernel.org>, James Morse <james.morse@arm.com>,
	<linux-arm-kernel@lists.infradead.org>
Subject: Re: [PATCH] cpu/hotplug: Fix NULL kobject warning in cpuhp_smt_enable()
Date: Sat, 25 Apr 2026 10:05:30 +0800	[thread overview]
Message-ID: <b4f97652-5cde-4ef7-8987-6f9b134d50ae@huawei.com> (raw)
In-Reply-To: <aetnTT51ucm2azGG@arm.com>



On 4/24/2026 8:51 PM, Catalin Marinas wrote:
> (updating Jonathan's email address to match MAINTAINERS)
> 
> On Fri, Apr 24, 2026 at 09:56:24AM +0800, Jinjie Ruan wrote:
>> On 4/24/2026 4:11 AM, Catalin Marinas wrote:
>>> On Thu, Apr 23, 2026 at 08:32:34PM +0800, Jinjie Ruan wrote:
>>>> On 4/23/2026 6:08 PM, Thomas Gleixner wrote:
>>>>> On Sat, Apr 18 2026 at 12:55, Catalin Marinas wrote:
>>>>>> Another option would have been to avoid marking such CPUs present but I
>>>>>> think this will break other things. Yet another option is to register
>>>>>> all CPU devices even if they never come up (like maxcpus greater than
>>>>>> actual CPUs).
>>>>>>
>>>>>> Opinions? It might be an arm64+ACPI-only thing.
>>>>>
>>>>> I think so. The proper thing to do is to apply sane limits:
>>>>>
>>>>>  1) The possible CPUs enumerated by firmware N_POSSIBLE_FW
>>>>>
>>>>>  2) The maxcpus limit on the command line N_MAXCPUS_CL
>>>>>
>>>>> So the actual possible CPUs evaluates to:
>>>>>
>>>>>    num_possible = min(N_POSSIBLE_FW, N_MAXCPUS_CL, CONFIG_NR_CPUS);
>>>>>
>>>>> The evaluation of the firmware should not mark CPUs present which are
>>>>> actually not. ACPI gives you that information. See:
>>>>>
>>>>>          5.2.12.14 GIC CPU Interface (GICC) Structure
>>>>>
>>>>> in the ACPI spec. That has two related bits:
>>>>>
>>>>> Enabled:
>>>>>
>>>>>    If this bit is set, the processor is ready for use. If this bit is
>>>>>    clear and the Online Capable bit is set, the system supports enabling
>>>>>    this processor during OS runtime. If this bit is clear and the Online
>>>>>    Capable bit is also clear, this processor is un- usable, and the
>>>>>    operating system support will not attempt to use it.
>>>>>
>>>>> Online Capable:
>>>>>
>>>>>    The information conveyed by this bit depends on the value of the
>>>>>    Enabled bit. If the Enabled bit is set, this bit is reserved and must
>>>>>    be zero. Otherwise, if this bit is set, the system supports enabling
>>>>>    this processor later during OS runtime
>>>>>
>>>>> So the combination of those gives you the right answer:
>>>>>
>>>>>    Enabled	Online
>>>>>    	        Capable
>>>>>    0            0        Not present, not possible
>>>>>    0            1        Not present, but possible to "hotplug" layter
>>>>>    1            0        Present
>>>>>    1            1        Invalid
>>>>
>>>> On x86, it seems that all CPUs with the ACPI_MADT_ENABLED bit set will
>>>> be marked as present.
>>>>
>>>> acpi_parse_x2apic()
>>>>   -> enabled = processor->lapic_flags & ACPI_MADT_ENABLED
>>>>   -> topology_register_apic(enabled)
>>>>      -> topo_register_apic(enabled)
>>>>         -> set_cpu_present(cpu, true)
>>>
>>> Yes but arm64 marks all CPUs present even if !ACPI_MADT_ENABLED as we
>>> don't have the notion of hardware CPU hotplug.
>>>
>>> I need to dig some more into the original vCPU hotplug support and why
>>> we ended up with all CPUs marked as present even if not calling
>>> register_cpu():
>>>
>>> https://lore.kernel.org/linux-arm-kernel/20240529133446.28446-1-Jonathan.Cameron@huawei.com/
>>>
>>> What's the MADT GICC provided by qemu with "-smp cpus=4,maxcpus=8"? If
>>> it says Enabled for the first 4 and Online Capable for the rest, maybe
>>> we can try something like below:
>>
>> Yes, you are absolutely right,Enabled for the first 4(with GIC Flags:
>> 0x1, bit0 set) and Online Capable for the rest(with GIC Flags: 0x8, bit3
>> set). The ACPI MADT disassembly result is as follows:
> 
> That's great, thanks for checking.
> 
> I'd like to get some feedback from Jonathan and James as they
> contributed the vCPU hotplug support. The reason was for kubernetes to
> add vCPUs to an existing VM. Hopefully no-one relied on
> /sys/devices/system/cpu/present to report 0-7 in the above
> configuration.

Yes, only cpu 0~3 present.

# cat /sys/devices/system/cpu/possible
0-7
# cat /sys/devices/system/cpu/enabled
0-3
# cat /sys/devices/system/cpu/present
0-3
# cat /sys/devices/system/cpu/online
0-3
# cat /sys/devices/system/cpu/offline
4-7

> 
> Have you tried the vCPU hotplug with this patch (the original use-case)?

Yes,the basic vCPU hotplug has no problem as below:

Add the last four CPUs:

(qemu) device_add host-arm-cpu,id=cpu4,core-id=2,thread-id=0
(qemu) [  678.984281] ACPI: CPU4 has been hot-added

(qemu) device_add host-arm-cpu,id=cpu5,core-id=2,thread-id=1
(qemu) [  686.593536] ACPI: CPU5 has been hot-added

(qemu) device_add host-arm-cpu,id=cpu6,core-id=3,thread-id=0
(qemu) [  699.493530] ACPI: CPU6 has been hot-added

(qemu) device_add host-arm-cpu,id=cpu7,core-id=3,thread-id=1
(qemu) [  706.165770] ACPI: CPU7 has been hot-added

# cat /sys/devices/system/cpu/cpu*/online
1
1
1
1
0
0
0
0
# echo 1 > /sys/devices/system/cpu/cpu4/online
[  868.423001] Detected PIPT I-cache on CPU4
[  868.437103] GICv3: CPU4: found redistributor 4 region
0:0x0000000008120000
[  868.437193] GICv3: CPU4: using allocated LPI pending table
@0x0000000040120000
[  868.437330] CPU4: Booted secondary processor 0x0000000004 [0x410fd082]
# echo 1 > /sys/devices/system/cpu/cpu5/online
[  871.855694] Detected PIPT I-cache on CPU5
[  871.869725] GICv3: CPU5: found redistributor 5 region
0:0x0000000008140000
[  871.869811] GICv3: CPU5: using allocated LPI pending table
@0x0000000040130000
[  871.869938] CPU5: Booted secondary processor 0x0000000005 [0x410fd082]
# echo 1 > /sys/devices/system/cpu/cpu6/online
[  874.756497] Detected PIPT I-cache on CPU6
[  874.770521] GICv3: CPU6: found redistributor 6 region
0:0x0000000008160000
[  874.770606] GICv3: CPU6: using allocated LPI pending table
@0x0000000040140000
[  874.770748] CPU6: Booted secondary processor 0x0000000006 [0x410fd082]
# echo 1 > /sys/devices/system/cpu/cpu7/online
[  878.212505] Detected PIPT I-cache on CPU7
[  878.226560] GICv3: CPU7: found redistributor 7 region
0:0x0000000008180000
[  878.226646] GICv3: CPU7: using allocated LPI pending table
@0x0000000040150000
[  878.226783] CPU7: Booted secondary processor 0x0000000007 [0x410fd082]
# cat /sys/devices/system/cpu/cpu*/online
1
1
1
1
1
1
1
1
# cat /sys/devices/system/cpu/enabled
0-7
# cat /sys/devices/system/cpu/present
0-7
# cat /sys/devices/system/cpu/online
0-7

Unplug the last four CPUs.

# (qemu) device_del cpu4
(qemu) [  977.336251] psci: CPU4 killed (polled 0 ms)

(qemu) device_del cpu5
(qemu) [  979.948212] psci: CPU5 killed (polled 0 ms)

(qemu) device_del cpu6
(qemu) [  981.976337] psci: CPU6 killed (polled 0 ms)

(qemu) device_del cpu7
(qemu) [  984.476253] psci: CPU7 killed (polled 0 ms)


# cat /sys/devices/system/cpu/cpu*/online
1
1
1
1
# cat /sys/devices/system/cpu/possible
0-7
# cat /sys/devices/system/cpu/enabled
0-3
# cat /sys/devices/system/cpu/present
0-3
# cat /sys/devices/system/cpu/online
0-3
# cat /sys/devices/system/cpu/offline
4-7

> 
> Anyway, feel free to post a v2 with the above proposal, cc Jonathan (on
> kernel.org) and James Morse and we'll take it from there. You can add a
> suggested-by me.
> 
> Thanks.
> 



  parent reply	other threads:[~2026-04-25  2:05 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20260417075534.3745793-1-ruanjinjie@huawei.com>
2026-04-18 11:55 ` [PATCH] cpu/hotplug: Fix NULL kobject warning in cpuhp_smt_enable() Catalin Marinas
2026-04-18 15:05   ` Catalin Marinas
2026-04-20  1:29     ` Jinjie Ruan
2026-04-23 12:46     ` Jinjie Ruan
2026-04-23 10:08   ` Thomas Gleixner
2026-04-23 12:32     ` Jinjie Ruan
2026-04-23 20:11       ` Catalin Marinas
2026-04-24  1:56         ` Jinjie Ruan
2026-04-24 12:51           ` Catalin Marinas
2026-04-24 18:27             ` Jonathan Cameron
2026-04-25  2:05             ` Jinjie Ruan [this message]
2026-04-24  2:47         ` Jinjie Ruan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=b4f97652-5cde-4ef7-8987-6f9b134d50ae@huawei.com \
    --to=ruanjinjie@huawei.com \
    --cc=catalin.marinas@arm.com \
    --cc=dietmar.eggemann@arm.com \
    --cc=james.morse@arm.com \
    --cc=jic23@kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=peterz@infradead.org \
    --cc=sudeep.holla@kernel.org \
    --cc=tglx@kernel.org \
    --cc=yangyicong@hisilicon.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox