From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 92270C36013 for ; Thu, 27 Mar 2025 13:41:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To: Content-Type:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=S/5SQq/xjhzkHH9hfiyRiKpnvfrnQTAW4c+eZ1ETT9w=; b=FmLjaRBe44AXjv4sZ1AZ8IEFJL UU9Xu9EiQAYgkyRjA5XMMBcXhVxvqPH7jw+XUYyxgppz6qxZ0ZxIkbHqLLO2QFt/u/2FCNDC/MIDg oG70gSuNUP3qgrlSSTW7mHinJxtFVmc7mLtH0qFRuvTr11cSUQ1jWwbNYgbMvAV79Rojs8DE5LADY 4+P2s+tOp+vnq5yOENc4pxVseK6MdPWrTZiquxMzINKc+7PQEg3cx82jRP7YxQ2FOKQktb9ycL3Vn tgYQXINwDze0Q/HWpviPIUMWtc6ZFS/NEGG7Mvo/C6A/bgxEckfAsijcAI7LxP8kxbQ93IlBKqWaf vwy6z+8Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1txnU2-0000000B6Ih-0UM1; Thu, 27 Mar 2025 13:41:22 +0000 Received: from mail-bn8nam04on2062a.outbound.protection.outlook.com ([2a01:111:f403:2408::62a] helo=NAM04-BN8-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98.1 #2 (Red Hat Linux)) id 1txnKG-0000000B4vQ-46UG; Thu, 27 Mar 2025 13:31:18 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=uYuewtkl8nDGtQRQlmrJK0nKbqHiTq+5x6Klt83j3J/u+aoLgM5lGxUTimJ6AtU2Nh0lL073qR9XTb0WMq64zJWRsg/VA1dRts7zB+0/HfOdYMvoWCW0/gcRZoAiW+1LdGbyc00hbiQLBDqGCdZZ/FwdsxQsiIxC2s3NjW38w+YMcFzbnQ3rCtZ6UJXo3Opv2oTtoUEhe0dt25sBq8uE2IDTT/5jgskEotEfYu+YkgJJIPSvOvS0ObV8sKLHupaJMspnvqSBw3E8veRlTQQujBi22V7u7acxKqCKPOQocpOzL8LZRrElpymDjKzGjC0zCY7ryZEw8zYi80+qNUy39A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=S/5SQq/xjhzkHH9hfiyRiKpnvfrnQTAW4c+eZ1ETT9w=; b=Bz3NUZaU182oworoiY5ml5+PEY0uf+53wNj5nOK2ivV1M1LikMCy9s6ocsm8Xi/7UQbWfmkmwh9RD+jeWDgbbsLX01tb7bRpj2D9c92TM6p+CIWVLMJgJeWyeAEZ1u/9FutXzhEQ0jdK5AxNdZ2h1TO8BaZurqLdRqzPjSlOL/3qQepy6oDJVHCGkIhyaiMER5mNHuPYxFApIcYBQCurbx8AA6Tg3A/a+fUTiQjzcMR26n6swAIqBD2xpi0mJTBjiW6nA5q8YP8zoA1M90dgYgAgTJQg5qp5aRMtkIAhl3Ac3tj0PCq/wbS9hZyFQjhkBYZtPBTADdoeiE6LcgBVpg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S/5SQq/xjhzkHH9hfiyRiKpnvfrnQTAW4c+eZ1ETT9w=; b=m9fqeJPRO0C3zXAwd89fRlfHZ/A3hsBODr3Qy9HIV7oevyhrtKXq/VpTvNHpfea23oJg2MVEkuhjjoAzutYX4rAjU3xC1mZ17TwmY4q9j34wSll2itTl5tvMicsMt2KRGvm1lMXtL2ldOxmIleG/LmC4xttFXqNzGu4JpTiQ8+cdFiSu5/eNFcUhe/w2ZjRmEzxGrE2GlucxbG1hifPgmgaHRk4wovXSiAWLh9jOmLqAZFHD8LeJkJ3Tl1jQrV7ceDdJwD1dCZaqHTPYgW/7/xa07sNbowA6q/F6vhp++IqZpe549obv1iC9zWbxtuN1pi/UOa42U2sJNakYVwj8Lg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by IA0PR12MB8695.namprd12.prod.outlook.com (2603:10b6:208:485::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.44; Thu, 27 Mar 2025 13:31:12 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8534.042; Thu, 27 Mar 2025 13:31:12 +0000 Date: Thu, 27 Mar 2025 10:31:10 -0300 From: Jason Gunthorpe To: Pratyush Yadav Cc: Changyuan Lyu , linux-kernel@vger.kernel.org, graf@amazon.com, akpm@linux-foundation.org, luto@kernel.org, anthony.yznaga@oracle.com, arnd@arndb.de, ashish.kalra@amd.com, benh@kernel.crashing.org, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dwmw2@infradead.org, ebiederm@xmission.com, mingo@redhat.com, jgowans@amazon.com, corbet@lwn.net, krzk@kernel.org, rppt@kernel.org, mark.rutland@arm.com, pbonzini@redhat.com, pasha.tatashin@soleen.com, hpa@zytor.com, peterz@infradead.org, robh+dt@kernel.org, robh@kernel.org, saravanak@google.com, skinsburskii@linux.microsoft.com, rostedt@goodmis.org, tglx@linutronix.de, thomas.lendacky@amd.com, usama.arif@bytedance.com, will@kernel.org, devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Subject: Re: [PATCH v5 09/16] kexec: enable KHO support for memory preservation Message-ID: References: <20250320015551.2157511-1-changyuanl@google.com> <20250320015551.2157511-10-changyuanl@google.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: BL1PR13CA0309.namprd13.prod.outlook.com (2603:10b6:208:2c1::14) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|IA0PR12MB8695:EE_ X-MS-Office365-Filtering-Correlation-Id: a2db3c6a-5ca7-4235-a489-08dd6d33a3e4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?EqQ6pBvxvwsUCpQYD0zp6OrFYTwneIG2dMdJeVj8yJcXvylyojGVOfWO6Gcq?= =?us-ascii?Q?PujtfjK4BwVv80ybVX5HliJ+YMiIp59DC9NkUY/1UPOzd0E9lA01SbLhA/mU?= =?us-ascii?Q?+9s5WPdkJGFRpzSLm19O7TXpFQaPuI3ySOU+u5vf0QgGl6eL5ljKEI7cSSrt?= =?us-ascii?Q?9pRj8cd99dQn4zwZHgaiHcWLflLL48Km2XWL2rd4ZkX3SSBq/f7MIJRX5Wnw?= =?us-ascii?Q?ZJGp11Ggwvtoj0bp3Ywad33jADD+dDa0MC7pGlsJyI2DfMoM43tNp5YwtJGY?= =?us-ascii?Q?dqYAiT38xXq2jItGUmhIPTY3T8PlB8N3c7Bf+jWrsyHwqvM4gqPPYUpgLnmi?= =?us-ascii?Q?MsTERM715/zk0r3L4LDlu5IFBnW+EwE0hqm2bOv7ODPOc1V2M55bMUwMiYZU?= =?us-ascii?Q?7IAfbbVRTYz4O4ugmcmi5UZL9v8BfIeTbrsY89dpYf92x8uLQRJJ4RWu+onR?= =?us-ascii?Q?TJCHBl129QnrN54PcNvrDaWqhaRUW8yHzuZ1OFgq55ltzjeAMEn0yYui/49v?= =?us-ascii?Q?bi76fpFYB6Fq4CY1H6jaDnLJVkb1nFaWl7c5cW3dlBwUWZzhUpl3O08dH2hC?= =?us-ascii?Q?WJG0r5IN8qJ5LN2sJo5OwGRdLATu20xGDCNWDEf1Sw6zBayTEiQg+6VQGwXd?= =?us-ascii?Q?Mgt8kPeawWYCu/ENIxosBW2r+Dr+2xmBMLVFhYkuqVWtvYGfqATnlgYHPxy/?= =?us-ascii?Q?ThmNHb75O8EuRLSsaGaTekfZrbQsCjeR8IvD0mJdcieiSap2IwlNd5QHjhx2?= =?us-ascii?Q?7ZvxF+t7p6Fqf2zIAvp/olcfsU4pi7nh38ha+IEohPleXcoKVyZzpV2VPxCZ?= =?us-ascii?Q?f1+s6dGjrAW21zAstNQe2xSAorkfaCV5vuXgs/B5XmHAF06jmHVA+hrOok/m?= =?us-ascii?Q?Y1pXnGDoRSkfGzZgJ09G+OH2t2CTZNMgetAeSD7r4TwBDyUYgt0lYI8dLHoP?= =?us-ascii?Q?zbgpwfCuNb4FBYUaW5/t2rmjp7IeZU8BXJhJIrqA9YzfWgabONACfVp6xwGd?= =?us-ascii?Q?VgQIzfsot/S8OcDA5SzKgLDgMT6C3cIZU6mvhln9IAjD6Pmi3CTN+YRdVMfp?= =?us-ascii?Q?xH6Jzc5Yg+6xOKvatjaN0WWUkd/sOhYkWTTqDm5F6juHKkLeF/Nt8weSll3T?= =?us-ascii?Q?v9WJ3+nvSlDoKze7PSz946FvrpnWknX9r5e06VGwapqLmuQ1IrkLMSu1BiOL?= =?us-ascii?Q?yRRFu6zk//CT49Fp++i+Mu6Ngfh0XoTMK6xtWRwiG6CAyXVvxvqbDv5IV1vi?= =?us-ascii?Q?XbSTruIWD2SsQqvXxE/A5usRwFZ+QcpNvAo3vOfyq8KLxjRNlGpOQPa+Ceat?= =?us-ascii?Q?0v5Nah0473nMcIWcGJn+POAI38p+Q8MiT96JjiB96lqAkegb1tZRmeII9wGf?= =?us-ascii?Q?uwGNN0q+MMiLzpaspkuflV3s58YN?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB8659.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?IAySl2/9+iJNtVeRnQI0khuS7jpV8Nctew82mEn77I+FZxlznyrhY3FW9nLj?= =?us-ascii?Q?VsZBEM2/aHZvNJIIb5TyZMzVBjr/bsyJAeQR556Ie6OAjKrpx+FYXeQ4eQrB?= =?us-ascii?Q?zrhJDOVAacB3RYG2VcLlhM8PR25JFnCSC4mINoAPbCSVp8VjH09+7AEwMl3L?= =?us-ascii?Q?14rLpBsxDPoJV+DahguxNxdIP83IyrggJ15bE836Vn5XJLbZL0BSh5dwo0gt?= =?us-ascii?Q?DQoRWAC9hs1B5Ynv3oItfo8X9iLvtgwa9B4q2bRxRWjFAEc9Rwz+ORbkYh02?= =?us-ascii?Q?Gzlw54YefxcupCX67VgnNcpxh5Zzis1s4kRztwY0vPF/LC+qY6eDRQjdi+oe?= =?us-ascii?Q?dLiZ8By6AQO/Jm8ihBMKzVxuiN9XWYdJfZ8620RjbbFO5BbC+RcGbsd0O9E6?= =?us-ascii?Q?3XbpGfK9WWgBqUXdlAXRa4MG/GSYITRDwv5nb+rT5kGJrSJCVU0Py16A3074?= =?us-ascii?Q?mYj8hrpEdAYf1gOx3cyOxh/GGTnLLFmNOPXfpStjjE6s2vD/1FQwrOiZRmC3?= =?us-ascii?Q?zEm0R5vISKIvhXYEiDCzpHYXPX2x0dJEnbV68RuvbdqNoZOJoIgu3ZnFW0F2?= =?us-ascii?Q?U6Rw8Jo/q2U3uHoDzuebWA5OjBS2W2Rre11BWbiCXrFcM25mfrP54/fVBGq5?= =?us-ascii?Q?H7eFBYr5ACU4E0DEtRqYuyvDPJHwk55qs1uDRYgzRnLIuQ+SUTyIY7p+lar5?= =?us-ascii?Q?qlxUbOYtnDyvB5qKvvuMmTXdoDz0sTSRYoWiQ7jw48EpIxPIn+UCpiVi7MnX?= =?us-ascii?Q?N6CDUs3lrCUv82xnhvcyu3gQzOZx6hPz7qEOQfJW/euafHI+/ue/Set8NcZx?= =?us-ascii?Q?HVYmmb17L8Nvttn6eSzLPeoIQxzZrqoEIi92J58SaUTx0snjNv7R49K3eUq/?= =?us-ascii?Q?QXN98rstwNvKuP477R8j3Yd0kdFpRyaGPBpK3UXSRbfFDtan/Ggrs8oIaFJk?= =?us-ascii?Q?iqaoqjtCsMYC4Etlhf3NC5MI1OPEKIQeja4StIGOCRHbJMaLDNwSasEq3Gqw?= =?us-ascii?Q?FYakNAjzdL2Go3Uxbkg8m94H1gCPxC0DD5roh6tyKIl5+3awys1oXAp21wjq?= =?us-ascii?Q?nukbluSwOwU7CQID5dryi/z6ePtNmBvNKyeaQd7Mjpnr0VLs0kKNrxd8GbYM?= =?us-ascii?Q?HGPgD0esseLwwZVMRN0kK0trgUssS/MdnZNBKTUxOtMylxv+NJERHHtvRcxY?= =?us-ascii?Q?1TWU5Ba9nMF6KRnDcQilm7euxdK3Xqmje0MyV1+4b8rLP2YFrSADOTgiVy7y?= =?us-ascii?Q?r780D4ZmGyl/Y0UQHjZRgXy1JbYxLB33RKYQHjgjbYYFeM0JBS2KIf/DampZ?= =?us-ascii?Q?IS6YceALqVvfJhNDXiql7pudgFJPLGsjHerXzDGrzbv/3qnmhds1+mJ6VS1x?= =?us-ascii?Q?2zSvIJnrR6wxgBXSEdzWWkSu6M+8EWPZTK0Qh0xTAhfy57Dgzp3Pa551uH8s?= =?us-ascii?Q?/MQegJ9sugDMgcvH8RaiA4wd6+UDZmb+fRbC/qOV8U+/eqycauBl7Orh1TJI?= =?us-ascii?Q?j9BCe0O4gTnXHy9JQ34g7gaqk2tREQg8f8xeAswyWUCrpKDPBfJSYloGpL6P?= =?us-ascii?Q?GkY2UU2wBi2cLRL/aOyxz1oqhuzdmUO7FfhfuM6X?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: a2db3c6a-5ca7-4235-a489-08dd6d33a3e4 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2025 13:31:11.9119 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QKpgEigjXdQV171c4EqRvHccGvjhnuNKBaXhk1whoT1+5vdtyqc25NO5YJFXEa6v X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8695 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250327_063117_026289_DE80C9E5 X-CRM114-Status: GOOD ( 10.52 ) X-Mailman-Approved-At: Thu, 27 Mar 2025 06:41:08 -0700 X-BeenThere: kexec@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "kexec" Errors-To: kexec-bounces+kexec=archiver.kernel.org@lists.infradead.org On Thu, Mar 27, 2025 at 10:03:17AM +0000, Pratyush Yadav wrote: > Of course, with the current linked list structure, this cannot work. But > I don't see why we need to have it. I think having a page-table like > structure would be better -- only instead of having PTEs at the lowest > levels, you have the bitmap. Yes, but there is a trade off here of what I could write in 30 mins and what is maximally possible :) The xarray is providing a page table implementation in a library form. I think this whole thing can be optimized, especially the memblock_reserve side, but the idea here is to get started and once we have some data on what the actual preservation workload is then someone can optimize this. Otherwise we are going to be spending months just polishing this one patch without any actual data on where the performance issues and hot spots actually are. Jason