From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 25B63C47DDF for ; Thu, 25 Jan 2024 07:33:23 +0000 (UTC) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=o/qlP90b; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4TLCH55Df4z3cVR for ; Thu, 25 Jan 2024 18:33:21 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=o/qlP90b; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=139.178.84.217; helo=dfw.source.kernel.org; envelope-from=rppt@kernel.org; receiver=lists.ozlabs.org) Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4TLCG92BNNz2yPq for ; Thu, 25 Jan 2024 18:32:33 +1100 (AEDT) Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id EF3176211E; Thu, 25 Jan 2024 07:32:29 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5D9E8C433C7; Thu, 25 Jan 2024 07:32:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706167949; bh=4YJZl/HB/FeUJBFx4g+Hq4GWbhTEDrW8F8ywC05k2CU=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=o/qlP90bWA8vs1DoAh96IyHbdPjr4YyuZvDq0onY5ZuuTV08XQml8GYs0ScKoZDdk kRNaiyxLgYIpzg1fYukUi9EtRFIRc727kYqe3wtWXR5gtzs35c/p1Y0J1fwyQZK1Ap 6AYBkhNKcCN91vd4Pq89IdOJXzTo/DfX9ptcP+9MEhnDjZshaFfmldWDYiRB/53swx +1IA6GzmPSyirfoR+LyXczKV2zT5RxfrhLZ1jFEvHaBZKOwVLKW3QWPCviK1bmr2b8 GoXu6/VAAQ3DwKjghCy5ELrIOAk+v3Yh5RTjDRV/vZkGVh46K5kitkUo/oNKs++SiR aXnmU6ZKh3AJg== Date: Thu, 25 Jan 2024 09:31:57 +0200 From: Mike Rapoport To: "Lameter, Christopher" Subject: Re: [PATCH v2] NUMA: Early use of cpu_to_node() returns 0 instead of the correct node id Message-ID: References: <20240123045843.75969-1-shijie@os.amperecomputing.com> <4a13353c-cf4b-a388-5776-389c61c63ec0@os.amperecomputing.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4a13353c-cf4b-a388-5776-389c61c63ec0@os.amperecomputing.com> X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, rafael@kernel.org, catalin.marinas@arm.com, jiaxun.yang@flygoat.com, mikelley@microsoft.com, linux-riscv@lists.infradead.org, will@kernel.org, mingo@kernel.org, vschneid@redhat.com, arnd@arndb.de, chenhuacai@kernel.org, linux-arm-kernel@lists.infradead.org, kuba@kernel.org, patches@amperecomputing.com, linux-mips@vger.kernel.org, aou@eecs.berkeley.edu, yury.norov@gmail.com, paul.walmsley@sifive.com, tglx@linutronix.de, jpoimboe@kernel.org, vbabka@suse.cz, Huang Shijie , gregkh@linuxfoundation.org, ndesaulniers@google.com, linux-kernel@vger.kernel.org, palmer@dabbelt.com, mhiramat@kernel.org, akpm@linux-foundation.org, linuxppc-dev@lists.ozlabs.org Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Wed, Jan 24, 2024 at 09:19:00AM -0800, Lameter, Christopher wrote: > On Tue, 23 Jan 2024, Huang Shijie wrote: > > > During the kernel booting, the generic cpu_to_node() is called too early in > > arm64, powerpc and riscv when CONFIG_NUMA is enabled. > > > > For arm64/powerpc/riscv, there are at least four places in the common code > > where the generic cpu_to_node() is called before it is initialized: > > 1.) early_trace_init() in kernel/trace/trace.c > > 2.) sched_init() in kernel/sched/core.c > > 3.) init_sched_fair_class() in kernel/sched/fair.c > > 4.) workqueue_init_early() in kernel/workqueue.c > > > > In order to fix the bug, the patch changes generic cpu_to_node to > > function pointer, and export it for kernel modules. > > Introduce smp_prepare_boot_cpu_start() to wrap the original > > smp_prepare_boot_cpu(), and set cpu_to_node with early_cpu_to_node. > > Introduce smp_prepare_cpus_done() to wrap the original smp_prepare_cpus(), > > and set the cpu_to_node to formal _cpu_to_node(). > > Would you please fix this cleanly without a function pointer? > > What I think needs to be done is a patch series. > > 1. Instrument cpu_to_node so that some warning is issued if it is used too > early. Preloading the array with NUMA_NO_NODE would allow us to do that. > > 2. Implement early_cpu_to_node on platforms that currently do not have it. > > 3. A series of patches that fix each place where cpu_to_node is used too > early. I think step 3 can be simplified with a generic function that sets per_cpu(numa_node) using early_cpu_to_node(). It can be called right after setup_per_cpu_areas(). -- Sincerely yours, Mike.