From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2045.outbound.protection.outlook.com [40.107.220.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A0F2212D1F1 for ; Mon, 20 Jan 2025 03:27:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.220.45 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737343628; cv=fail; b=OCxrEGFccjU235qklSYMKaW1H2i029qa/iLnNOR8WeU+iSRQX2GIQw8aq4h0i1QdxJgvGY2KVZ0XeUsiUD/WlS9IvbSBZisGh6dNn7VoSDV/BMRi3ekoR5RRUXSFyLzrDA5kz7GRXiR9bbQwq0nx+HIcKutoQHDp0ggCWpwlct4= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737343628; c=relaxed/simple; bh=XGAUTe68tj0lNpuOjYoqkNdnzkSq5SS/Tc0i4LgO/fc=; h=Content-Type:Date:Message-Id:From:Subject:Cc:To:References: In-Reply-To:MIME-Version; b=OqyuUB/DFtaKNyEKpvN/7FUCSaMy7uxRYif1/QUX4m0c+h1mqCV8eyzzVvlxVbmBxrXFXlro27KHdsN2N172VXHr940aSVYK1pMNR8I+SEu9Q/E5oPMYfOjOesR7tDllyZzYFnpl5vfn0UnQ9lcqVtLUk5jYmp45Te2i7YROUE0= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=nwCw8Vtg; arc=fail smtp.client-ip=40.107.220.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="nwCw8Vtg" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BjMY0wLw/V9l0hgTesNapl6quNReiAb2QYPlyA64GfMbhUw89I+He4dvWJIGefXAhzv6/UIskjd2Jn/bQiS80d0Zmr7wgoDe43kq0Km617bQwwlpI29SJPMtMA7COKYWURq5W7heB7qlNIjglyoSg5Op9/cqYHYTgMSIVnGxmzkRaW48axxeGz8pI4gb669RcLHFEYTl/k2aqRl14FzTrPbpwHaqfLwrhXj8ExtEM5aiKMram2wZNl5faBzsfH4yoOKXgnVyYsu+HCB59icZMeUNGeCHNYuRtMt27o7BeZll72USUat9CY7fFLJOlwB024hpFOCoJChNMKOj90MFSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rZJeyIoilnB7V/UDcBT36JRmpWKlzdbgy8G4YLLXR+4=; b=hhysP6135FrmR0DbEe8H8nhLybhUMMbmuUG9hYNN79YeZ/jPShE7FmA6WmtFY898FwNNcg46MSBCpk0rcjsDWflGkChxHOozOZnMw9cnCrTVp2SJPhtKvbXw0zAbefhfTTEilXigcxgr1tIizHNCzQ5dGVzenAiHQ/7kcnOGYDxSl/cqrHLc5Lrnind9U5rqegH2EOci4JxEQfW3smrXAgMGKE0vOE0iL89IW6k0hgeALuKrg2KSAn3AC2bCE6xrg9QhW8old+uF1x6A3Ts5RBH+NUREZnTvBN0OJtD83H0jmhxXyzFbd641Us1UEGKTdbyzXo8PqbH0h6xQ+5SRBA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rZJeyIoilnB7V/UDcBT36JRmpWKlzdbgy8G4YLLXR+4=; b=nwCw8VtgsIHjjb+y1KGPwuqOchLVUvg060Duj+symuIOInu2PwT7eaNkpObm3mmOHLTg7UeDwVHd2imgASm2wvs/ZfD4ld1HtAVea+uYuEIz6f5eeWpuiYNbm6tDvdABkV2PBtebOEvuA64GZs2N4qckgz2etlOB5aJHfPJvVw2ViUYTopcf0W1DqbBKz2wJHAJfVcXuEF2++c2oEzXxv7bv3AKpMMkCJIqmkUnrd+g0fij0iw9KBg9XRGOE8qwamt+3I2Px9i5g8IE1vDKJNEw1Xw/TE05/3nm7c+if420of6klMmYVXU/fcx/WHUCl5aD3dYN3eKfi19/023otxg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by SA1PR12MB7317.namprd12.prod.outlook.com (2603:10b6:806:2ba::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8356.17; Mon, 20 Jan 2025 03:27:04 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%3]) with mapi id 15.20.8356.020; Mon, 20 Jan 2025 03:27:03 +0000 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=UTF-8 Date: Sun, 19 Jan 2025 22:27:01 -0500 Message-Id: From: "Zi Yan" Subject: Re: [fake numa not working] [PATCH 1/1] mm/fake-numa: allow later numa node hotplug Cc: , <42.hyeyoo@gmail.com>, "Bruno Faccini" , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , "akpm@linux-foundation.org" , "rppt@kernel.org" , "david@redhat.com" , "jhubbard@nvidia.com" , "mrusiniak@nvidia.com" To: "Hyeonggon Yoo" X-Mailer: aerc 0.18.2 References: <20250106120659.359610-1-bfaccini@nvidia.com> <20250106120659.359610-2-bfaccini@nvidia.com> <1DC02706-256B-4B61-B309-7D86595F4B22@nvidia.com> <7a641617-a4e5-45cd-bb1f-628bacd44046@sk.com> In-Reply-To: X-ClientProxiedBy: BN9PR03CA0941.namprd03.prod.outlook.com (2603:10b6:408:108::16) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|SA1PR12MB7317:EE_ X-MS-Office365-Filtering-Correlation-Id: 5d906e23-c84e-44ea-342c-08dd39024f06 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: =?utf-8?B?ZHNTUzN4TEZCblZmSlN3R01TVnRUSXdoN2lrMEtBLzE4Wjd6NHRrMEk5bU5y?= =?utf-8?B?clJJdU0vaWJsQnF6eTdPN3dhNXFpeWtTdm52UDVkbmJTK2xkVHFFUDNIRVl1?= =?utf-8?B?TitWNnBXdXd3ZHQ2M1NSQmU2VHU4VXk5emM4M0VkN0pNakxITkhkdWtGUmlr?= =?utf-8?B?cVBxU0c5dDlReFR2dDUvUHlNVGIrN1ZkUWJlVDlLOHRNVlA4NFFRZXNHSlEy?= =?utf-8?B?Y2JURWtIVHY1OU5QZ3JXVDREWjAxM2V6MERNR0hGVkt1OHdqd2pqWmxLNXpx?= =?utf-8?B?UEFyWmRFc095bmdyWjl6Nk1waFNYYmxxK0I2dUl2WGFYY2tiVGhVU1BXc0NS?= =?utf-8?B?VGgyQldRYWxwYUNIbHRmY2dldU1iM3hrUEFyZW5YdjhMSW1kSUxXWllyby8r?= =?utf-8?B?RDRmT1hUeEpVSTZJMmMzWEg5TlFyN3BFRXlsRDZYSlBadUQ0b0U2OG5yay9r?= =?utf-8?B?T01XQmpqd3JuMkoyVFF5UHgvZ1RkcEdFRHNpVy94ZHhiMVZZSVBOVFYyZFFN?= =?utf-8?B?UzNIRUNNQkk2RWo5R0tYMnRLd1NDNWRveVlmaHNoN29BTXhEam5lTzdhSmUx?= =?utf-8?B?c3ZuR1pvcWhoeUR1ZG41R1JLanZtTDYwYlpSb3FRbEJMb3h0d3prRktsME93?= =?utf-8?B?cDhOVTQ0MmNlbThxb1BNaTNPcjdzUHhFR3UrQW5TOC93L0F2UzVUbnkyUGdT?= =?utf-8?B?WVBjM2pQRGdWS3VINGNwd1E2cE9HYTNVd1RET3FBYXJoNjJLaCtwSnVXMjh5?= =?utf-8?B?cHhnT3pmL2Q2OThwTEIwMmJPZ0IrK0sxVWg4T2RmZFZXZU8xcG1Vd2hwL1JY?= =?utf-8?B?WlNXNHFyd1E5Ulcyb2xSdG9ybFZEdXZwTVRuZHRxbjRpZ3dsRVlqckQ5dnQv?= =?utf-8?B?NGM3Y3hrZUY3bVlZRXg5clFVL1NYMDhZcEwrbXhrTWU2R1l1cTJZeENHTitC?= =?utf-8?B?dUNWMVdBU1Y4VWlISUhmeWZFbWRWNlIzZWd4b1JNTlMzdGh0ZTNIZUdTbDU5?= =?utf-8?B?VEZ2anpEL3V1V2ROci9zUnVWcjRHWTdIcUVkTEpnall1TGlCditWT3R1Z1RJ?= =?utf-8?B?dU4xQ2dIQnJJN3E0aHc2ZEFFb3RyenNIS21xc1ZmVUtpRWVRNldHdjluQXpG?= =?utf-8?B?aVJsdDhjNURYVXNVaVg4dDh0VS9CZWVKdlJ3bUpMZ0tFdnpqN3dSa3pLcjRV?= =?utf-8?B?TEpoYTRObjdHUDRTYjUyOUE2RjBmQk5qdjZ1K1MyTlFIQ2ZpQU9rK3Z0Z1NG?= =?utf-8?B?dUpndlV2K3ova1dmVkJJaFpKRHhxOVFjTE5PbDhpVzd5Q1NqR1JnbmZnOGEv?= =?utf-8?B?RVc2SmhwWDUySzdwdTNtMGZqUjE2QUkrVWRRZzltZmc5T3pEWGRRZkl4bm1r?= =?utf-8?B?c2lxT2MrWjkrODV2OGZnVmJOSE1BOWRxQzVYakVBQk00M2MvMGM5WDQ2R1RC?= =?utf-8?B?ZHhxbHhWSEdMQWlpUE1HYWVCZmYwc0VOYmlobUl1a0VBNmtrK3Q5OEVHd1dH?= =?utf-8?B?b2NLdjc1eWloR0Q1MEJ1V1hMbkkyaFYwNmNtbW9aN216enNxZGpVS3Q1VGRK?= =?utf-8?B?bHpua21Rd25IMlNnM0VNNTd6cVIwTFdMQmZxN0hvcnA0ZTBBUVYxaUpkV1BW?= =?utf-8?B?TmVyUXFVSTd2eDJWZ2FtM2ZYM2tVUGZOeUZlekczUFovU284cFhsNDJDRndH?= =?utf-8?B?em5TRXBMa0ZtNnJKdWttSTUwNDFrY0x0dk1ScW1vUytrN3c2dllDSGRUc1Vn?= =?utf-8?B?QzNSZ2FiYk1LRDUraHVrNkJVSkJ0R3NOM0I0Y3Y0SmdmVE5pZTJQMlltcWQ0?= =?utf-8?B?Z3NvaFcvNmVMQkZaNXFWZXBocXcycU4rOXpMcjBLWVFBMFoyRHNYZmMvRUxm?= =?utf-8?Q?qxmsbSHVjOsUt?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aG05OWNDVjBGekNwaTdWenNwaVp1WVVpYklzSUx4a1ltMTgzUG1lSUF5WDdk?= =?utf-8?B?bFM5bU14ZU5OcDJyWmtKT2NnSXVLNDZIVlEvcGJwRG9ZZXAzdGFsS1VrbS9X?= =?utf-8?B?MnR3enlzODJLOUJkVkNYNWIrTW92S2FiYldadURKd0Z4WmhUUTZPWEdIMFRB?= =?utf-8?B?UHdVeXd2MUx1anRCWmZBeVJhQkprejl5WGlkZ3hVWXR2RkcrQjh2cjZmamhN?= =?utf-8?B?S2xkZGM2a0djcHovaGZEV3pGaXdVdis2NFdpdUVFQk1zYnhjUkhnajFiY01W?= =?utf-8?B?SDV3dC8xZmNLVHVGMTl6RWRKTTJ5cTdCdUplb20xMHZUc0xLSk9GZGQxelN6?= =?utf-8?B?Ni9PaDhKbExEeURlSE9tSlFHNzZZdVkyN0hmUTNPcjJyd2RDQ1RsWUo1ZXky?= =?utf-8?B?dWVaNGMxZVhzTTdZejhCM0Fvdk5leENSVDh2cWZxcEJwVlFNZm5RZ1FPWWta?= =?utf-8?B?aVIxcEVNdlhiQ3g3TVNoZGFoNFRYOXo2dWo3UXpKbDFJWTQ4Z09xVFkyVlMz?= =?utf-8?B?MDB5cW9OcHVsNUNxQVJ1aHgydU1mWTM1YUxGcFFjZGl5NTJkUERkQUpQZUlU?= =?utf-8?B?MXp3ZmkyK2E1ZFQ1aHphK2dac1lWZG5vNXpsdmh3ZXo4U1ZMWFVGTTNDSUU2?= =?utf-8?B?UlExSEM1Qm1scWkzQjYyZ3Bza1hzN3Q1QWo1b3RMVEhGMk14bjBXNVRQNzJy?= =?utf-8?B?c0RPWkhNZHZld0JsZXpZam1ZaGdMWGtwelB6ZXVHaG9laE1acUI1eHl1OW5E?= =?utf-8?B?eEJYOGU1L1U3eFVQVGlKeW5MdzNSN05JQi9LRXg2ZTdXMWpRUitDajkwajNj?= =?utf-8?B?SDlqZ2t3bmtvSHZ1QmdaYkRVN3p5Rkdublh4K3JQZ0hoY3lBeWM0RkpndWVo?= =?utf-8?B?Rkh1cFhPMUpCY3dvMk1VMFcyK0pPMU1OSXFJVFVublAwTjdINGU1aUtPVktQ?= =?utf-8?B?TUR3UFJQd0RHTjRkTlFSWWZQZ01rYmo0UCtiVVRMNGxzZ3RLb2thWDVrUUpB?= =?utf-8?B?YVhtVm1pNC9LWWMxWHE2bWd0MmhETXBLY1RpajM1dStzOVJ3Mktoa092WVdk?= =?utf-8?B?bzV3VmhNc0taT3lLeVErUENGYkN2N3RFUnhwUy83UnQrL1NJdnFQR0U1dFlw?= =?utf-8?B?Wm9Ld0tLNXRqVlJFd21YQ1liMjU5SHBwL3p1djlqZElFbWsxS3pZWnlvWGNl?= =?utf-8?B?TEZOWlcwbGxrRWZBWGFJZzlOUlRXT3NMajlkbEpkNzZ3V1VkWUNmYTUxVVI5?= =?utf-8?B?K1IrWVRrUE1HVEtxM0hMSGliTmV1RnZlL2tQdTVCTWR6Z1h3UGw5dXdrdXc3?= =?utf-8?B?c1hXSmFONW1lblF1a2RPVElzRFZ3QUJMVVRJR1NpMGhXZ3VlZU5wc3lwL0Zr?= =?utf-8?B?eEhqcFViQWN4ZDV2VzVlMHVaU3JzMnY4emZXTk1GenNrRTZZbDJ4T3oyV3N2?= =?utf-8?B?RGl5WHF0alBTVkVmZGZkL0dzTkppNjlCQmZrdFFuTU1ZZVREbmdFQzE5cThw?= =?utf-8?B?YlRpTi9GeVRxK2dhYnlmZ3lDazNyTU54YitmL3crOHE2QXlNc05zUkF4aTVs?= =?utf-8?B?aTQvMlpEUEp2dlhFb2ttdUMxV1l0R0VBejlMdVdDdjJ1TVFxZnlBWEkrMTFV?= =?utf-8?B?VHFyQnY5cFk4RmtHSXpaSkhrWEpJeEFkZmt1Z1FKSU5Ob01UWVJ4elVPQWpk?= =?utf-8?B?SXNXU2s2cW5Tcm53alpLUzdRYzlZZ3lZNGY2M2t5VTNDbFRpQXRldGdlZXQw?= =?utf-8?B?WDV2bUtvREt4VXNjM2VaZ2FtekYwQ295ano3U2hzVVRSREtlZGd0Q1FkcCsw?= =?utf-8?B?WngzejZHNm1TcUU2azcrbDBsQzQ4cXZXNC9oVjJJd2owb1FzeWNRb2F5dUo4?= =?utf-8?B?Zjg3b2xHYkMxaWFWL3gxTTNUdEgvTzFucjY3a2xkVlhwSG1KK1BEN1dEOGdU?= =?utf-8?B?RnExMmtxcGI3MDk1NWtoOWpOblZ3S3JYRHdOc3FGQkFDdnV2ZWVMa0xsN01j?= =?utf-8?B?Z1lpUkdjYmNSVS9xaVpaeTBOdEs4ZkNvMjdMa0EyVGk4dy9oM0NxVjR6L1ZR?= =?utf-8?B?cVJCQ3pNNzkwS1pkM25PVWQ2dU4yNkl4eXZCRVNHSzlJYitERTRrK21Wazcz?= =?utf-8?Q?ghue2kF/FAelM9IC/QaDwiqZm?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5d906e23-c84e-44ea-342c-08dd39024f06 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jan 2025 03:27:03.6436 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2ZSIpjx5laEqbbWBM31jsge/dpgLiNQYPQ/K+9JE+JYtO6rOTRjPCrXNBt1XjCJR X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7317 On Sun Jan 19, 2025 at 9:37 PM EST, Hyeonggon Yoo wrote: > > > On 1/20/2025 11:16 AM, Zi Yan wrote: > > On Sun Jan 19, 2025 at 7:37 PM EST, Hyeonggon Yoo wrote: > >> > >> > >> On 1/18/2025 4:09 AM, Zi Yan wrote: > >>> On 17 Jan 2025, at 0:00, Hyeonggon Yoo wrote: > >>> > >>>> On 1/6/2025 9:06 PM, Bruno Faccini wrote: > >>>>> Current fake-numa implementation prevents new Numa > >>>>> nodes to be later hot-plugged by drivers. > >>>>> A common symptom of this limitation is the > >>>>> "node was absent from the node_possible_map" > >>>>> message by associated warning in mm/memory_hotplug.c: > >>>>> add_memory_resource(). > >>>>> This comes from the lack of remapping in both > >>>>> pxm_to_node_map[] and node_to_pxm_map[] tables > >>>>> to take fake-numa nodes into account and thus > >>>>> triggers collisions with original and physical nodes > >>>>> only-mapping that had been determined from BIOS tables. > >>>>> This patch fixes this by doing the necessary node-ids > >>>>> translation in both pxm_to_node_map[]/node_to_pxm_map[] > >>>>> tables. > >>>>> node_distance[] table has also been fixed accordingly. > >>>>> > >>>>> Signed-off-by: Bruno Faccini > >>>> > >>>> Hi Bruno, > >>>> > >>>> This commit causes WARN() and disables fakenuma > >>>> when I pass "numa=3Dfake=3D4U" on my QEMU setup. > >>>> > >>>> Attaching WARN() log and git bisect log to help debugging. > >>> > >>> Is your VM getting 4 NUMA nodes at the end? > >> > >> No, it only gets node 0 at the end. > >=20 > > Got it. > >=20 > >> > >>> Can you also share your QEMU command line and your kernel config? > >> > >> The config is attached (fakenuma.config) > >> > >> QEMU command line: > >> $ qemu-system-x86_64 \ > >> -cpu host \ > >> -enable-kvm \ > >> -smp $(nproc) \ > >> -m 32G \ > >> -nographic \ > >> -kernel arch/x86/boot/bzImage \ > >> -initrd ../../rootfs.cpio.gz \ > >> -nic user,model=3Dvirtio-net-pci \ > >> -append "console=3DttyS0 numa=3Dfake=3D4U" > >> > >>> This can help Bruno reproduce the issue locally. At least, I > >>> cannot reproduce it on mm-everything-2025-01-16-23-18. > >> > >> Still reproduced on mm-everything-2025-01-18-04-55 with my setup. > >> > >>> I also > >>> notice that my VM does not have =E2=80=9CNo NUMA configuration found= =E2=80=9D > >>> (in your log) in its kernel log and that might explain > >>> why I could not reproduce it. > >> > >> Did you do NUMA configuration (-numa in QEMU) for your VM? > >> I didn't, but it's still expected to work 'cause it's fake numa. > >=20 > > No. It seems that the "No NUMA configuration found" is caused by your > > qemu, where dummy_numa_init() is used (maybe ACPI SRAT is not > > populated?) > > FYI, Yes. ACPI SRAT is not populated. > > $ qemu-system-x86_64 --version > QEMU emulator version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.24) > Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers Basically, fix_pxm_node_maps() breaks fakenuma when it cannot fix the pxm_to_node_map() due to ACPI SRAT is missing. IMHO, it should let fakenuma proceed with a warning. The diff below fixed the issue locally, but Bruno can tell if I miss anything. diff --git a/drivers/acpi/numa/srat.c b/drivers/acpi/numa/srat.c index 59fffe34c9d0..18d04f054e93 100644 --- a/drivers/acpi/numa/srat.c +++ b/drivers/acpi/numa/srat.c @@ -95,9 +95,12 @@ int __init fix_pxm_node_maps(int max_nid) int i, j, index =3D -1, count =3D 0; nodemask_t nodes_to_enable; =20 - if (numa_off || srat_disabled()) + if (numa_off) return -1; =20 + if (srat_disabled()) + return 0; + /* find fake nodes PXM mapping */ for (i =3D 0; i < MAX_NUMNODES; i++) { if (node_to_pxm_map[i] !=3D PXM_INVAL) { @@ -117,9 +120,9 @@ int __init fix_pxm_node_maps(int max_nid) } } } - if (WARN(index !=3D max_nid, "%d max nid when expected %d\n", - index, max_nid)) - return -1; + /* No srat data */ + if (index !=3D max_nid) + return 0; =20 nodes_clear(nodes_to_enable); =20 > > > I used the patch below to emulate the situation and > > reproduced the issue. Since dummy_numa_init() is used when there is > > no underlying NUMA architecture, NUMA initialization fails, or NUMA > > is disabled on the command line (the last does not apply here), > > it should be properly handled. I will let Bruno look into it. > >=20 > > Thank you for the info. > >=20 > > diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c > > index 64e5cdb2460a..70dfd9bfb23f 100644 > > --- a/arch/x86/mm/numa.c > > +++ b/arch/x86/mm/numa.c > > @@ -223,7 +223,7 @@ static int __init dummy_numa_init(void) > > */ > > void __init x86_numa_init(void) > > { > > - if (!numa_off) { > > + if (0 && !numa_off) { > > #ifdef CONFIG_ACPI_NUMA > > if (!numa_init(x86_acpi_numa_init)) > > return; > >=20 > >>>> > >>>> [WARN() splat] > >>>> > >>>> [ 0.009676] No NUMA configuration found > >>>> [ 0.009677] Faking a node at [mem 0x0000000000000000-0x000000083f= ffffff] > >>>> [ 0.009705] Fake node size 16895MB too small, increasing to 16896= MB > >>>> [ 0.009708] Faking node 0 at [mem 0x0000000000001000-0x0000000420= 000fff] (16896MB) > >>>> [ 0.009711] Faking node 1 at [mem 0x0000000420001000-0x000000083f= ffffff] (16895MB) > >>>> [ 0.009738] ------------[ cut here ]------------ > >>>> [ 0.009739] -1 max nid when expected 1 > >>>> [ 0.009760] WARNING: CPU: 0 PID: 0 at drivers/acpi/numa/srat.c:12= 0 fix_pxm_node_maps+0x577/0x810 > >>>> [ 0.009766] Modules linked in: > >>>> [ 0.009769] CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 6.13.0= -rc6+ #31 > >>>> [ 0.009771] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996)= , BIOS 1.15.0-1 04/01/2014 > >>>> [ 0.009772] RIP: 0010:fix_pxm_node_maps+0x577/0x810 > >>>> [ 0.009774] Code: 8b 85 40 ff ff ff 44 8b 8d 44 ff ff ff 8b 85 48= ff ff ff e9 83 fb ff ff 44 89 f2 44 89 ce 48 c7 c7 aa 2f 84 8d e8 b9 71 01= fe <0f> 0b 41 b8 ff ff ff ff e9 ba fb ff ff 48 c7 c7 a0 d8 d1 8d 44 89 > >>>> [ 0.009776] RSP: 0000:ffffffff8da03be0 EFLAGS: 00010046 ORIG_RAX:= 0000000000000000 > >>>> [ 0.009778] RAX: 0000000000000000 RBX: 0000000000000400 RCX: 0000= 000000000000 > >>>> [ 0.009779] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000= 000000000000 > >>>> [ 0.009780] RBP: ffffffff8da03ca8 R08: 0000000000000000 R09: 0000= 000000000000 > >>>> [ 0.009781] R10: 0000000000000000 R11: 0000000000000000 R12: 0000= 000000000001 > >>>> [ 0.009782] R13: 00000000000007ff R14: 0000000000000001 R15: 0000= 000000000000 > >>>> [ 0.009783] FS: 0000000000000000(0000) GS:ffffffff8de8c000(0000)= knlGS:0000000000000000 > >>>> [ 0.009784] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > >>>> [ 0.009785] CR2: ffff90e4bde01000 CR3: 00000001bd03e000 CR4: 0000= 0000000200b0 > >>>> [ 0.009788] Call Trace: > >>>> [ 0.009790] > >>>> [ 0.009792] ? show_regs+0x71/0x90 > >>>> [ 0.009797] ? __warn+0x8d/0x150 > >>>> [ 0.009799] ? fix_pxm_node_maps+0x577/0x810 > >>>> [ 0.009802] ? report_bug+0x1ab/0x1c0 > >>>> [ 0.009805] ? fixup_exception+0x27/0x380 > >>>> [ 0.009809] ? early_fixup_exception+0xa2/0xf0 > >>>> [ 0.009813] ? do_early_exception+0x28/0x90 > >>>> [ 0.009816] ? early_idt_handler_common+0x2f/0x3a > >>>> [ 0.009819] ? fix_pxm_node_maps+0x577/0x810 > >>>> [ 0.009822] ? numa_cleanup_meminfo+0x8c/0x5b0 > >>>> [ 0.009828] numa_emulation+0x4e3/0xad0 > >>>> [ 0.009830] ? __pfx_dummy_numa_init+0x10/0x10 > >>>> [ 0.009833] ? __pfx_dummy_numa_init+0x10/0x10 > >>>> [ 0.009836] numa_memblks_init+0x10c/0x2c0 > >>>> [ 0.009838] ? __pfx_dummy_numa_init+0x10/0x10 > >>>> [ 0.009840] numa_init+0x61/0x3e0 > >>>> [ 0.009842] ? topology_register_boot_apic+0x2a/0x40 > >>>> [ 0.009846] x86_numa_init+0x65/0x80 > >>>> [ 0.009848] initmem_init+0xe/0x20 > >>>> [ 0.009850] setup_arch+0x9d8/0xfa0 > >>>> [ 0.009852] ? _printk+0x58/0x90 > >>>> [ 0.009855] start_kernel+0x5f/0xb50 > >>>> [ 0.009860] x86_64_start_reservations+0x18/0x30 > >>>> [ 0.009863] x86_64_start_kernel+0xc0/0x110 > >>>> [ 0.009864] ? setup_ghcb+0xe/0x140 > >>>> [ 0.009867] common_startup_64+0x13e/0x141 > >>>> [ 0.009871] > >>>> [ 0.009871] ---[ end trace 0000000000000000 ]--- > >>>> > >>>> [Git bisect log] > >>>> > >>>> # bad: [f378252a2168c2fbf8fc08b635061e5f6748c1f2] kasan: sw_tags: us= e str_on_off() helper in kasan_init_sw_tags() > >>>> git bisect bad f378252a2168c2fbf8fc08b635061e5f6748c1f2 > >>>> # good: [cbc5dde0a461240046e8a41c43d7c3b76d5db952] fs/proc: fix soft= lockup in __read_vmcore (part 2) > >>>> git bisect good cbc5dde0a461240046e8a41c43d7c3b76d5db952 > >>>> # good: [fb73203263021f2ee9dd54d280b1c543d10acd76] mm: move common p= art of pagetable_*_ctor to helper > >>>> git bisect good fb73203263021f2ee9dd54d280b1c543d10acd76 > >>>> # bad: [8fd966c9310f14d5bbe653cf087853582170aad6] mm, swap: remove o= ld allocation path for HDD > >>>> git bisect bad 8fd966c9310f14d5bbe653cf087853582170aad6 > >>>> # bad: [8646971f02651146554233d63fdd721f5f060973] mm: shmem: skip sw= apcache for swapin of synchronous swap device > >>>> git bisect bad 8646971f02651146554233d63fdd721f5f060973 > >>>> # good: [b8ba614eaa8dd31702c61e7df7231b1b18b99259] mm/damon/paddr: r= eport filter-passed bytes back for DAMOS_STAT action > >>>> git bisect good b8ba614eaa8dd31702c61e7df7231b1b18b99259 > >>>> # good: [fd0935b8e9e8c2edf05a3d0ffa09d0aa3e9cf2dd] Docs/ABI/damon: d= ocument per-region DAMOS filter-passed bytes stat file > >>>> git bisect good fd0935b8e9e8c2edf05a3d0ffa09d0aa3e9cf2dd > >>>> # good: [3ffd3cd7e2e8793d3b5fbb83c03668c47ab6d599] selftests/damon: = remove tests for DAMON debugfs interface > >>>> git bisect good 3ffd3cd7e2e8793d3b5fbb83c03668c47ab6d599 > >>>> # good: [bf9f93f0b7bff8af780330f5094775a39caf3612] mm/damon: remove = DAMON debugfs interface > >>>> git bisect good bf9f93f0b7bff8af780330f5094775a39caf3612 > >>>> # bad: [1746744be6ff271e29345a5be4cc144aa33b10ab] mm/memmap: prevent= double scanning of memmap by kmemleak > >>>> git bisect bad 1746744be6ff271e29345a5be4cc144aa33b10ab > >>>> # bad: [ca19a5ab8756ef78c34d02d2b8c266a9cc7bf57d] mm/fake-numa: allo= w later numa node hotplug > >>>> git bisect bad ca19a5ab8756ef78c34d02d2b8c266a9cc7bf57d > >>>> # first bad commit: [ca19a5ab8756ef78c34d02d2b8c266a9cc7bf57d] mm/fa= ke-numa: allow later numa node hotplug > >>>> > >>>>> --- > >>>>> drivers/acpi/numa/srat.c | 86 +++++++++++++++++++++++++++++= +++++++ > >>>>> include/acpi/acpi_numa.h | 5 +++ > >>>>> include/linux/numa_memblks.h | 3 ++ > >>>>> mm/numa_emulation.c | 45 ++++++++++++++++--- > >>>>> mm/numa_memblks.c | 2 +- > >>>>> 5 files changed, 133 insertions(+), 8 deletions(-) > >>>>> > >>>>> diff --git a/drivers/acpi/numa/srat.c b/drivers/acpi/numa/srat.c > >>>>> index bec0dcd1f9c3..59fffe34c9d0 100644 > >>>>> --- a/drivers/acpi/numa/srat.c > >>>>> +++ b/drivers/acpi/numa/srat.c > >>>>> @@ -81,6 +81,92 @@ int acpi_map_pxm_to_node(int pxm) > >>>>> } > >>>>> EXPORT_SYMBOL(acpi_map_pxm_to_node); > >>>>> +#ifdef CONFIG_NUMA_EMU > >>>>> +/* > >>>>> + * Take max_nid - 1 fake-numa nodes into account in both > >>>>> + * pxm_to_node_map()/node_to_pxm_map[] tables. > >>>>> + */ > >>>>> +int __init fix_pxm_node_maps(int max_nid) > >>>>> +{ > >>>>> + static int pxm_to_node_map_copy[MAX_PXM_DOMAINS] __initdata > >>>>> + =3D { [0 ... MAX_PXM_DOMAINS - 1] =3D NUMA_NO_NODE }; > >>>>> + static int node_to_pxm_map_copy[MAX_NUMNODES] __initdata > >>>>> + =3D { [0 ... MAX_NUMNODES - 1] =3D PXM_INVAL }; > >>>>> + int i, j, index =3D -1, count =3D 0; > >>>>> + nodemask_t nodes_to_enable; > >>>>> + > >>>>> + if (numa_off || srat_disabled()) > >>>>> + return -1; > >>>>> + > >>>>> + /* find fake nodes PXM mapping */ > >>>>> + for (i =3D 0; i < MAX_NUMNODES; i++) { > >>>>> + if (node_to_pxm_map[i] !=3D PXM_INVAL) { > >>>>> + for (j =3D 0; j <=3D max_nid; j++) { > >>>>> + if ((emu_nid_to_phys[j] =3D=3D i) && > >>>>> + WARN(node_to_pxm_map_copy[j] !=3D PXM_INVAL, > >>>>> + "Node %d is already binded to PXM %d\n", > >>>>> + j, node_to_pxm_map_copy[j])) > >>>>> + return -1; > >>>>> + if (emu_nid_to_phys[j] =3D=3D i) { > >>>>> + node_to_pxm_map_copy[j] =3D > >>>>> + node_to_pxm_map[i]; > >>>>> + if (j > index) > >>>>> + index =3D j; > >>>>> + count++; > >>>>> + } > >>>>> + } > >>>>> + } > >>>>> + } > >>>>> + if (WARN(index !=3D max_nid, "%d max nid when expected %d\n", > >>>>> + index, max_nid)) > >>>>> + return -1; > >>>>> + > >>>>> + nodes_clear(nodes_to_enable); > >>>>> + > >>>>> + /* map phys nodes not used for fake nodes */ > >>>>> + for (i =3D 0; i < MAX_NUMNODES; i++) { > >>>>> + if (node_to_pxm_map[i] !=3D PXM_INVAL) { > >>>>> + for (j =3D 0; j <=3D max_nid; j++) > >>>>> + if (emu_nid_to_phys[j] =3D=3D i) > >>>>> + break; > >>>>> + /* fake nodes PXM mapping has been done */ > >>>>> + if (j <=3D max_nid) > >>>>> + continue; > >>>>> + /* find first hole */ > >>>>> + for (j =3D 0; > >>>>> + j < MAX_NUMNODES && > >>>>> + node_to_pxm_map_copy[j] !=3D PXM_INVAL; > >>>>> + j++) > >>>>> + ; > >>>>> + if (WARN(j =3D=3D MAX_NUMNODES, > >>>>> + "Number of nodes exceeds MAX_NUMNODES\n")) > >>>>> + return -1; > >>>>> + node_to_pxm_map_copy[j] =3D node_to_pxm_map[i]; > >>>>> + node_set(j, nodes_to_enable); > >>>>> + count++; > >>>>> + } > >>>>> + } > >>>>> + > >>>>> + /* creating reverse mapping in pxm_to_node_map[] */ > >>>>> + for (i =3D 0; i < MAX_NUMNODES; i++) > >>>>> + if (node_to_pxm_map_copy[i] !=3D PXM_INVAL && > >>>>> + pxm_to_node_map_copy[node_to_pxm_map_copy[i]] =3D=3D NUMA_NO= _NODE) > >>>>> + pxm_to_node_map_copy[node_to_pxm_map_copy[i]] =3D i; > >>>>> + > >>>>> + /* overwrite with new mapping */ > >>>>> + for (i =3D 0; i < MAX_NUMNODES; i++) { > >>>>> + node_to_pxm_map[i] =3D node_to_pxm_map_copy[i]; > >>>>> + pxm_to_node_map[i] =3D pxm_to_node_map_copy[i]; > >>>>> + } > >>>>> + > >>>>> + /* enable other nodes found in PXM for hotplug */ > >>>>> + nodes_or(numa_nodes_parsed, nodes_to_enable, numa_nodes_parsed); > >>>>> + > >>>>> + pr_debug("found %d total number of nodes\n", count); > >>>>> + return 0; > >>>>> +} > >>>>> +#endif > >>>>> + > >>>>> static void __init > >>>>> acpi_table_print_srat_entry(struct acpi_subtable_header *header= ) > >>>>> { > >>>>> diff --git a/include/acpi/acpi_numa.h b/include/acpi/acpi_numa.h > >>>>> index b5f594754a9e..99b960bd473c 100644 > >>>>> --- a/include/acpi/acpi_numa.h > >>>>> +++ b/include/acpi/acpi_numa.h > >>>>> @@ -17,11 +17,16 @@ extern int node_to_pxm(int); > >>>>> extern int acpi_map_pxm_to_node(int); > >>>>> extern unsigned char acpi_srat_revision; > >>>>> extern void disable_srat(void); > >>>>> +extern int fix_pxm_node_maps(int max_nid); > >>>>> extern void bad_srat(void); > >>>>> extern int srat_disabled(void); > >>>>> #else /* CONFIG_ACPI_NUMA */ > >>>>> +static inline int fix_pxm_node_maps(int max_nid) > >>>>> +{ > >>>>> + return 0; > >>>>> +} > >>>>> static inline void disable_srat(void) > >>>>> { > >>>>> } > >>>>> diff --git a/include/linux/numa_memblks.h b/include/linux/numa_memb= lks.h > >>>>> index cfad6ce7e1bd..dd85613cdd86 100644 > >>>>> --- a/include/linux/numa_memblks.h > >>>>> +++ b/include/linux/numa_memblks.h > >>>>> @@ -29,7 +29,10 @@ int __init numa_cleanup_meminfo(struct numa_memi= nfo *mi); > >>>>> int __init numa_memblks_init(int (*init_func)(void), > >>>>> bool memblock_force_top_down); > >>>>> +extern int numa_distance_cnt; > >>>>> + > >>>>> #ifdef CONFIG_NUMA_EMU > >>>>> +extern int emu_nid_to_phys[MAX_NUMNODES]; > >>>>> int numa_emu_cmdline(char *str); > >>>>> void __init numa_emu_update_cpu_to_node(int *emu_nid_to_phys, > >>>>> unsigned int nr_emu_nids); > >>>>> diff --git a/mm/numa_emulation.c b/mm/numa_emulation.c > >>>>> index 031fb9961bf7..9d55679d99ce 100644 > >>>>> --- a/mm/numa_emulation.c > >>>>> +++ b/mm/numa_emulation.c > >>>>> @@ -8,11 +8,12 @@ > >>>>> #include > >>>>> #include > >>>>> #include > >>>>> +#include > >>>>> #define FAKE_NODE_MIN_SIZE ((u64)32 << 20) > >>>>> #define FAKE_NODE_MIN_HASH_MASK (~(FAKE_NODE_MIN_SIZE - 1UL)) > >>>>> -static int emu_nid_to_phys[MAX_NUMNODES]; > >>>>> +int emu_nid_to_phys[MAX_NUMNODES]; > >>>>> static char *emu_cmdline __initdata; > >>>>> int __init numa_emu_cmdline(char *str) > >>>>> @@ -379,6 +380,7 @@ void __init numa_emulation(struct numa_meminfo = *numa_meminfo, int numa_dist_cnt) > >>>>> size_t phys_size =3D numa_dist_cnt * numa_dist_cnt * sizeof(ph= ys_dist[0]); > >>>>> int max_emu_nid, dfl_phys_nid; > >>>>> int i, j, ret; > >>>>> + nodemask_t physnode_mask =3D numa_nodes_parsed; > >>>>> if (!emu_cmdline) > >>>>> goto no_emu; > >>>>> @@ -395,7 +397,6 @@ void __init numa_emulation(struct numa_meminfo = *numa_meminfo, int numa_dist_cnt) > >>>>> * split the system RAM into N fake nodes. > >>>>> */ > >>>>> if (strchr(emu_cmdline, 'U')) { > >>>>> - nodemask_t physnode_mask =3D numa_nodes_parsed; > >>>>> unsigned long n; > >>>>> int nid =3D 0; > >>>>> @@ -465,9 +466,6 @@ void __init numa_emulation(struct numa_memin= fo *numa_meminfo, int numa_dist_cnt) > >>>>> */ > >>>>> max_emu_nid =3D setup_emu2phys_nid(&dfl_phys_nid); > >>>>> - /* commit */ > >>>>> - *numa_meminfo =3D ei; > >>>>> - > >>>>> /* Make sure numa_nodes_parsed only contains emulated nodes */ > >>>>> nodes_clear(numa_nodes_parsed); > >>>>> for (i =3D 0; i < ARRAY_SIZE(ei.blk); i++) > >>>>> @@ -475,10 +473,21 @@ void __init numa_emulation(struct numa_meminf= o *numa_meminfo, int numa_dist_cnt) > >>>>> ei.blk[i].nid !=3D NUMA_NO_NODE) > >>>>> node_set(ei.blk[i].nid, numa_nodes_parsed); > >>>>> - numa_emu_update_cpu_to_node(emu_nid_to_phys, ARRAY_SIZE(emu_ni= d_to_phys)); > >>>>> + /* fix pxm_to_node_map[] and node_to_pxm_map[] to avoid collision > >>>>> + * with faked numa nodes, particularly during later memory hotplu= g > >>>>> + * handling, and also update numa_nodes_parsed accordingly. > >>>>> + */ > >>>>> + ret =3D fix_pxm_node_maps(max_emu_nid); > >>>>> + if (ret < 0) > >>>>> + goto no_emu; > >>>>> + > >>>>> + /* commit */ > >>>>> + *numa_meminfo =3D ei; > >>>>> + > >>>>> + numa_emu_update_cpu_to_node(emu_nid_to_phys, max_emu_nid + 1); > >>>>> /* make sure all emulated nodes are mapped to a physical node= */ > >>>>> - for (i =3D 0; i < ARRAY_SIZE(emu_nid_to_phys); i++) > >>>>> + for (i =3D 0; i < max_emu_nid + 1; i++) > >>>>> if (emu_nid_to_phys[i] =3D=3D NUMA_NO_NODE) > >>>>> emu_nid_to_phys[i] =3D dfl_phys_nid; > >>>>> @@ -501,12 +510,34 @@ void __init numa_emulation(struct numa_mem= info *numa_meminfo, int numa_dist_cnt) > >>>>> numa_set_distance(i, j, dist); > >>>>> } > >>>>> } > >>>>> + for (i =3D 0; i < numa_distance_cnt; i++) { > >>>>> + for (j =3D 0; j < numa_distance_cnt; j++) { > >>>>> + int physi, physj; > >>>>> + u8 dist; > >>>>> + > >>>>> + /* distance between fake nodes is already ok */ > >>>>> + if (emu_nid_to_phys[i] !=3D NUMA_NO_NODE && > >>>>> + emu_nid_to_phys[j] !=3D NUMA_NO_NODE) > >>>>> + continue; > >>>>> + if (emu_nid_to_phys[i] !=3D NUMA_NO_NODE) > >>>>> + physi =3D emu_nid_to_phys[i]; > >>>>> + else > >>>>> + physi =3D i - max_emu_nid; > >>>>> + if (emu_nid_to_phys[j] !=3D NUMA_NO_NODE) > >>>>> + physj =3D emu_nid_to_phys[j]; > >>>>> + else > >>>>> + physj =3D j - max_emu_nid; > >>>>> + dist =3D phys_dist[physi * numa_dist_cnt + physj]; > >>>>> + numa_set_distance(i, j, dist); > >>>>> + } > >>>>> + } > >>>>> /* free the copied physical distance table */ > >>>>> memblock_free(phys_dist, phys_size); > >>>>> return; > >>>>> no_emu: > >>>>> + numa_nodes_parsed =3D physnode_mask; > >>>>> /* No emulation. Build identity emu_nid_to_phys[] for numa_ad= d_cpu() */ > >>>>> for (i =3D 0; i < ARRAY_SIZE(emu_nid_to_phys); i++) > >>>>> emu_nid_to_phys[i] =3D i; > >>>>> diff --git a/mm/numa_memblks.c b/mm/numa_memblks.c > >>>>> index a3877e9bc878..ff4054f4334d 100644 > >>>>> --- a/mm/numa_memblks.c > >>>>> +++ b/mm/numa_memblks.c > >>>>> @@ -7,7 +7,7 @@ > >>>>> #include > >>>>> #include > >>>>> -static int numa_distance_cnt; > >>>>> +int numa_distance_cnt; > >>>>> static u8 *numa_distance; > >>>>> nodemask_t numa_nodes_parsed __initdata; > >>> > >>> > >>> Best Regards, > >>> Yan, Zi > >>> > >=20 > >=20 > >=20 > >=20 --=20 Best Regards, Yan, Zi