From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47CE6C54E4A for ; Fri, 8 Mar 2024 14:51:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 787206B0397; Fri, 8 Mar 2024 09:51:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 736816B0398; Fri, 8 Mar 2024 09:51:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5FDB96B0399; Fri, 8 Mar 2024 09:51:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5123D6B0397 for ; Fri, 8 Mar 2024 09:51:56 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id EE48D1C0EEA for ; Fri, 8 Mar 2024 14:51:55 +0000 (UTC) X-FDA: 81874161390.01.EEB504E Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf20.hostedemail.com (Postfix) with ESMTP id E4A961C0009 for ; Fri, 8 Mar 2024 14:51:52 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf20.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709909513; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r/8Bxm2e4x68X3XrF4BYg3LiQyeafC4vLuOeMUOmTig=; b=2+hpkCJj29yyzF9NQ9wRl7A0O7r5S2yHR2fmQ6hi8XmBXuhCq+xbV4WX71X9w8fwA+jwVh 6g1OW8mgak3wceBVKmLyTWRxAcdAOXU5RWeVbjcOTKN9zVTT8IQ5JBJPDnry6jUvHgXqWg jUE6ty8bONQbXTxZuWYZBgk/uSTao5c= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf20.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709909513; a=rsa-sha256; cv=none; b=5jBWpQBx5MNSMOjGQBnYWpbyhnBxMiBCgQNnFQ9Zy2lhjZk7jwFGKzGhh0HIaVUHoSD3PQ 2gvXSnfsqf7uADnoDH7BDKIApBI4QPWB4x/fnNNgdfykw+xUAKt7Elq9+a1yuffseEldU/ tRC+ZqNBb/ccLYhh/oKxBA3cTT+0Xg8= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 6190160A29; Fri, 8 Mar 2024 14:51:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9F0DDC433F1; Fri, 8 Mar 2024 14:51:46 +0000 (UTC) Date: Fri, 8 Mar 2024 14:51:44 +0000 From: Catalin Marinas To: Marek Szyprowski Cc: "Christoph Lameter (Ampere)" , Mark Rutland , "linux-pm@vger.kernel.org" , "Rafael J. Wysocki" , Viresh Kumar , Will Deacon , Jonathan.Cameron@huawei.com, Matteo.Carlini@arm.com, Valentin.Schneider@arm.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, Eric Mackay , dave.kleikamp@oracle.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux@armlinux.org.uk, robin.murphy@arm.com, vanshikonda@os.amperecomputing.com, yang@os.amperecomputing.com, Nishanth Menon , Stephen Boyd Subject: Re: [PATCH v3] ARM64: Dynamically allocate cpumasks and increase supported CPUs to 512 Message-ID: References: <37099a57-b655-3b3a-56d0-5f7fbd49d7db@gentwo.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: E4A961C0009 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: ty94kq4fackdf1xx6ohfk4usnr98zuqw X-HE-Tag: 1709909512-577706 X-HE-Meta: U2FsdGVkX18mJpqBqTsnuI8iBM/JpohP3T5Wvvd5TPkWvCep7V2vRv46rQ7cXmVrzPDym9rX+mxo5skrJ9qbq1WbPPl5CuLK3rIuVEujqPaXy3rq8AVnEdiL/HE5KivJIwmTVjwjQflNl7BFV+z5Aqw53mzp4oJqNnLMRAV5+y7/yjdqe8HkwLZ7hul+alaeMXHrGlbqvrJU90JCEznr3oThvnlE4dyvfU9NJV388dCpllDA0TkUA2e6vxggjoNuOLV39Q44//j1VPkGgjC6sRdefM3VEWf6G9c1uz8HdCBjKoUFAhNsfdShs9TQIQW2MmfI4P7cHfSinpqmKwaaGY6OXyAEFGUTpvjoHZvMFQEPDq0BgkWlisU4z+NXL/F9L7AyTrj0vhSPJhfW+Zv1aLmOKWTfqQ5K12oG6zQhpgG93f9hOvx6hHgFquy8Qlbf3ixMOpeowTjMOqmu32OJhnL61T0wl2H8m1LNV7xT87f5RrKVB4HfTsD3f5Ewb7/AqSlWacv3k+sCWpauQdQLNmoYAiyZqRmgeo1E/fy34BTzr9EhkbQaznT+laWNihOcpZcBG1V0moo5rOAYB9eNjAXOzNKKWcS4vZNv+bUVvnMWxcVvGthJTqOh8GBQMLW7vDB63ukxWE2reFeGpq8WJ6UFRtpokWD0dAR24tv1g3S8zbjmiWeBz9aC90VWVhrSzsH9hqwmOxECgveaDr0jDH08mg1C0804opF0IRjes6CCC+HlHYMHL0qS5FOAJNd+bd3Wdg523V3n1mKI7h5U6hd2zw4KZL8EGSe4OGzuSOH308i+SuCukY60M+B/Ib9zMzx5cAzgAczN7R1G7qT1rAQk+t76TCbe5TsxEvoMx2OZtKh/6IeFVfXlgumUiRyfnFS1u7fUIexmtrn+/LKveSpfpsEbS8DYuiNT3wKSQw0cNzNDj/liqjE5/wqPL/Wl7iYZ59+i/mfPoliY4Z3 Y5zxPIrz lLxgxEOXP1RFECctOwc+vp06fDKnMvFvoPmI2av9bj8i/liQKXumPQgwYFiq/brVydbLYwkQ1oEONHZF+yRSurj7KCA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Mar 08, 2024 at 03:01:28PM +0100, Marek Szyprowski wrote: > This patch landed in today's linux-next as commit 0499a78369ad ("ARM64: > Dynamically allocate cpumasks and increase supported CPUs to 512"). > Unfortunately it triggers the following warning during boot on most of > my ARM64-based test boards. Here is an example from Odroid-N2 board: > >  ------------[ cut here ]------------ >  WARNING: CPU: 4 PID: 63 at drivers/opp/core.c:2554 > dev_pm_opp_set_config+0x390/0x710 >  Modules linked in: dw_hdmi_i2s_audio meson_gxl smsc onboard_usb_hub(+) > rtc_pcf8563 panfrost snd_soc_meson_axg_sound_card drm_shmem_helper > crct10dif_ce dwmac_generic snd_soc_meson_card_utils gpu_sched > snd_soc_meson_g12a_toacodec snd_soc_meson_g12a_tohdmitx rc_odroid > snd_soc_meson_codec_glue pwm_meson ao_cec_g12a meson_gxbb_wdt meson_ir > snd_soc_meson_axg_frddr snd_soc_meson_axg_toddr snd_soc_meson_axg_tdmin > meson_vdec(C) v4l2_mem2mem videobuf2_dma_contig snd_soc_meson_axg_tdmout > videobuf2_memops axg_audio videobuf2_v4l2 sclk_div videodev > reset_meson_audio_arb snd_soc_meson_axg_fifo clk_phase dwmac_meson8b > stmmac_platform videobuf2_common mdio_mux_meson_g12a meson_drm > meson_dw_hdmi rtc_meson_vrtc stmmac meson_ddr_pmu_g12 mc dw_hdmi > drm_display_helper pcs_xpcs snd_soc_meson_t9015 meson_canvas gpio_fan > display_connector snd_soc_meson_axg_tdm_interface > snd_soc_simple_amplifier snd_soc_meson_axg_tdm_formatter nvmem_meson_efuse >  hub 1-1:1.0: USB hub found >  CPU: 4 PID: 63 Comm: kworker/u12:5 Tainted: G         C 6.8.0-rc3+ #14677 >  Hardware name: Hardkernel ODROID-N2 (DT) >  Workqueue: events_unbound deferred_probe_work_func >  pstate: 80000005 (Nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) >  pc : dev_pm_opp_set_config+0x390/0x710 >  lr : dev_pm_opp_set_config+0x5c/0x710 >  ... >  Call trace: >   dev_pm_opp_set_config+0x390/0x710 >   dt_cpufreq_probe+0x268/0x480 >   platform_probe+0x68/0xd8 >   really_probe+0x148/0x2b4 >   __driver_probe_device+0x78/0x12c >   driver_probe_device+0xdc/0x164 >   __device_attach_driver+0xb8/0x138 >   bus_for_each_drv+0x84/0xe0 >   __device_attach+0xa8/0x1b0 >   device_initial_probe+0x14/0x20 >   bus_probe_device+0xb0/0xb4 >   deferred_probe_work_func+0x8c/0xc8 >   process_one_work+0x1ec/0x53c >   worker_thread+0x298/0x408 >   kthread+0x124/0x128 >   ret_from_fork+0x10/0x20 >  irq event stamp: 317178 >  hardirqs last  enabled at (317177): [] > ktime_get_coarse_real_ts64+0x10c/0x110 >  hardirqs last disabled at (317178): [] el1_dbg+0x24/0x8c >  softirqs last  enabled at (315802): [] > __do_softirq+0x4a0/0x4e8 >  softirqs last disabled at (315793): [] > ____do_softirq+0x10/0x1c >  ---[ end trace 0000000000000000 ]--- >  cpu cpu2: error -EBUSY: failed to set regulators >  cpufreq-dt: probe of cpufreq-dt failed with error -16 > > It looks that cpufreq-dt and/or opp drivers needs some adjustments > related with this change. That's strange. Is this with defconfig? I wonder whether NR_CPUS being larger caused the issue with this specific code. Otherwise CPUMASK_OFFSTACK may not work that well on arm64. > > > > --- > > > > > > Original post: https://www.spinics.net/lists/linux-mm/msg369701.html > > V2: https://lkml.org/lkml/2024/2/7/505 > > > > > > V1->V2 > > > > - Keep quotation marks > > - Remove whiltespace damage > > - Add tested by > > > > V2->V3: > > - Add test results > > - Rework descriptions > > > > > >  arch/arm64/Kconfig | 16 +++++++++++++++- > >  1 file changed, 15 insertions(+), 1 deletion(-) > > > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig > > index aa7c1d435139..4e767dede47d 100644 > > --- a/arch/arm64/Kconfig > > +++ b/arch/arm64/Kconfig > > @@ -1427,7 +1427,21 @@ config SCHED_SMT > >  config NR_CPUS > >      int "Maximum number of CPUs (2-4096)" > >      range 2 4096 > > -    default "256" > > +    default "512" > > + > > +# > > +# Determines the placement of cpumasks. > > +# > > +# With CPUMASK_OFFSTACK the cpumasks are dynamically allocated. > > +# Useful for machines with lots of core because it avoids increasing > > +# the size of many of the data structures in the kernel. > > +# > > +# If this is off then the cpumasks have a static sizes and are > > +# embedded within data structures. > > +# > > +    config CPUMASK_OFFSTACK > > +    def_bool y > > +    depends on NR_CPUS > 256 > > > >  config HOTPLUG_CPU > >      bool "Support for hot-pluggable CPUs" > > Best regards > -- > Marek Szyprowski, PhD > Samsung R&D Institute Poland > -- Catalin