From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E467DC5475B for ; Mon, 11 Mar 2024 18:55:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 616736B0102; Mon, 11 Mar 2024 14:55:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5C6B06B0103; Mon, 11 Mar 2024 14:55:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4402E6B0104; Mon, 11 Mar 2024 14:55:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 325B76B0102 for ; Mon, 11 Mar 2024 14:55:55 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id F2DD780250 for ; Mon, 11 Mar 2024 18:55:54 +0000 (UTC) X-FDA: 81885662628.20.C496C57 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf07.hostedemail.com (Postfix) with ESMTP id 84B0140010 for ; Mon, 11 Mar 2024 18:55:53 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710183353; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ySBg4pfu3LftaTYH2zHW1RxYEaQqC68iF18YV2LoezU=; b=iMu/Qy6uvWYh/Ryivrunj9stJXhERijrqMDt4i7sXCnyydGwuuNIaqbcQJieBRzIgS+x/g hygoBAwX9c4tuYyjAG7VVNtbGwsrHTQtlWQklgbUQEqtubxwyonDp50g7ByyTn5cJGsdYH AOEL8U62qQWqscICE08YxTNJWOY9ups= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of cmarinas@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=cmarinas@kernel.org; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710183353; a=rsa-sha256; cv=none; b=xCxXVYqjQhksyHWpi2eWvroekpn8zKJeSMIdkf5dpnJBpD4DFlBMnUcNdYK6AItARz6fKl eTVaG3XsA6UUNHs9IIbJKEW7BCe9W1zrgUmkBt/AtCd6hnBLdjrX7ADlbrJ12XlpUmRyO6 uHYFD3S488zQKQQCQrtv6VtALGMnFHQ= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 70F0560F70; Mon, 11 Mar 2024 18:55:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B6058C433F1; Mon, 11 Mar 2024 18:55:47 +0000 (UTC) Date: Mon, 11 Mar 2024 18:55:45 +0000 From: Catalin Marinas To: Marek Szyprowski Cc: "Christoph Lameter (Ampere)" , Mark Rutland , "linux-pm@vger.kernel.org" , "Rafael J. Wysocki" , Viresh Kumar , Will Deacon , Jonathan.Cameron@huawei.com, Matteo.Carlini@arm.com, Valentin.Schneider@arm.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, Eric Mackay , dave.kleikamp@oracle.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux@armlinux.org.uk, robin.murphy@arm.com, vanshikonda@os.amperecomputing.com, yang@os.amperecomputing.com, Nishanth Menon , Stephen Boyd Subject: Re: [PATCH v3] ARM64: Dynamically allocate cpumasks and increase supported CPUs to 512 Message-ID: References: <37099a57-b655-3b3a-56d0-5f7fbd49d7db@gentwo.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 84B0140010 X-Rspam-User: X-Stat-Signature: qxsmynny9qbmg5pp1ijtknm4fiz56dfp X-Rspamd-Server: rspam01 X-HE-Tag: 1710183353-48636 X-HE-Meta: U2FsdGVkX1+tRdbr6QCW5enMqM0giDMZVFvrZ9xoepFEd3P6BItSkyMS+HByikM74zb5KEaOdt93soBb1XUhkLwaz7lcL6t8xKGqypkFuXk3v2mf0uVcd0BcG8MGn+zy4pxrwx3vc4Z1ACnjbPULffZkIW1uwap7kaTkjg/rtD9kc0+5zlQbzbiIgp4On3Tn+WpKCT7FP51Umo4TNw7bNu3HbaZGFG44EYRbj72vSIRAsyL27IF23OhIEndX1gzqsWECDEb6tgU1sKF84JMGVndJ+RDO3kzOZeCNb1qkwfOLPlteIe09u3MlNQeznYxpP/oJHzlPUS6Qw6a+YpuukeZ1asL5wBZh4CdeBWgCrDX+Z+mNSJAoT9yuL0q5qjzV0gj2W4JnHlM0KcezYNFhFUEoDaFgIO4SCd8ag4GIebxfTTdxtAiXOYviIZhvbkHOakRyZ0WjFhpYXRBCcTZatM1HFN4oznF35r7EmqmBhBD24HAlVXtbpYSUXT3KX0s1Wa3rGW11YgbrPsHPTlh9IZrSub72eMA9v0u2lS9vI9xI2Dkfb+VARqyCXpmUEd43lGkorOisAjc7LrK9kCp66Qm344vqoTo+aYdlyi4/tkfdGbHqIy7IblzyaqUFuOQ5Q7e3sjLcLeBA2sqOGOoGHGP2ww8XxXVg4dfNOG9yRj4SUuOrV91tBFmr1F0z9hv6z1+BZwZRQD8MT8Q9QXN2ASe0zKyTSCkPVHx3onOHDX4LHjg1OdTrrGQ9ASY/h4hBozfSUBdWDXp6MuZZAm/KpdMajmSToPH71hxUrV/n3ezMHtk+KdEUi5OeB2s070Z9GNLbcrxa56HS/P7EGegzSvYufiUHK8YuMdMlwnTfepp45UtkzilGskxHEmHb0ARLqiroNB8JIfER5xWYwz7P3cyaAo8sfjdp05+8SU3L7U+XHXcAEdmGRnQ8zPQR2x5r/dK4du4wrOrxb2pBOrJ 24UiuJbj ovDLpV3tPImGFX3rDwvLEAjF9iAPq9wxxS63CZHiWafqq0PYAIM+W8kht1fVFG3som7DLrW0JZm8WjYXfM86122Cszg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Mar 08, 2024 at 03:01:28PM +0100, Marek Szyprowski wrote: > On 07.03.2024 02:45, Christoph Lameter (Ampere) wrote: > > Currently defconfig selects NR_CPUS=256, but some vendors (e.g. Ampere > > Computing) are planning to ship systems with 512 CPUs. So that all CPUs on > > these systems can be used with defconfig, we'd like to bump NR_CPUS to 512. > > Therefore this patch increases the default NR_CPUS from 256 to 512. > > > > As increasing NR_CPUS will increase the size of cpumasks, there's a fear that > > this might have a significant impact on stack usage due to code which places > > cpumasks on the stack. To mitigate that concern, we can select > > CPUMASK_OFFSTACK. As that doesn't seem to be a problem today with > > NR_CPUS=256, we only select this when NR_CPUS > 256. > > > > CPUMASK_OFFSTACK configures the cpumasks in the kernel to be > > dynamically allocated. This was used in the X86 architecture in the > > past to enable support for larger CPU configurations up to 8k cpus. [...] > This patch landed in today's linux-next as commit 0499a78369ad ("ARM64: > Dynamically allocate cpumasks and increase supported CPUs to 512"). > Unfortunately it triggers the following warning during boot on most of > my ARM64-based test boards. Here is an example from Odroid-N2 board: I spent a big part of this afternoon going through the code paths but there's nothing obvious that triggered this problem. My suspicion is some memory corruption, algorithmically I can't see anything that could go wrong with CPUMASK_OFFSTACK. Unfortunately I could not reproduce it yet to be able to add some debug info. So I decided to revert this patch. If we get to the bottom of it during the merging window, I can still revive it. Otherwise we'll add it to linux-next post -rc1. Thanks for reporting it and subsequent debugging. -- Catalin