From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0CF32D3A67B for ; Tue, 29 Oct 2024 18:47:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=cX774osWxdkYZ6WGEjb8Lb3ReCtSI6W8nFCRT0QUpAY=; b=fi51mfMR9mimB1 71hYHNKRmNOqT2Q6sbK6D8221B3FbClTzRMsGIRzcW5m6zmWtd6dRvWauingveCiYs7vJiqfhGEr/ iQ7iUqQsmn7ymwGgnj0zSaWZH24OMEFU76Z0X/de7oYFJ52pwjPZvlOadL+3aG4KXSqiya4oRwVDV /9hXvdoCyf6LtMN/iX2utBYPaFapZo20e7YQkk06gqO4CgzqFxA6Vl6AqBU4YV3z1kvIVG7quFbpY 2MvsruOK1XBevBzPzhT/4qP405K3QH47nkgpI9/Y902Eoyq3++9SlXuwy1Oy2YkrY3KepMgpSHC0H xivIiTrj2hSp8pSgzuMA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t5rFJ-0000000FY5l-3nwx; Tue, 29 Oct 2024 18:47:13 +0000 Received: from nyc.source.kernel.org ([2604:1380:45d1:ec00::3]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1t5pFi-0000000F9On-2SQj for kexec@lists.infradead.org; Tue, 29 Oct 2024 16:39:31 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 857A2A42BAE; Tue, 29 Oct 2024 16:37:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0C4A9C4CECD; Tue, 29 Oct 2024 16:39:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1730219969; bh=CvGZshwjj/Jzn5BjBR9puizjZSCPPPEqIfbTu3B8uME=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=H49B90HSTsjTYtBoUnW/LU4uL0T1hght3SqZoXNwwASlW55bQNW1CinjwhB/prcmu iUoxWppmLaa5HNplX4vWLbPVkAG6YuzUv4H6BG/3/HxAKWJ3DWieR4Mz+tcKIBF0ir NQVLIl7RSw4sDuGvQsRj0rCqRGAXZ/g283OZKWIiYR8ecmEAoPRGpPidbnDOkdFGB8 TeW7p5S3BktQ3dneRvFiYg43r1Fnj+DAgk9v+5LCHzStBuN4DbHRIum3ms39Cckv9g aWohmQHz6h3UAMFJzdhnOZ0KvM+87mJQ2NFc4QKpRpimGJymmCS44YVNbGkyc5H6Fu g9jRKgCFBGhxw== Date: Tue, 29 Oct 2024 18:35:30 +0200 From: Mike Rapoport To: David Rientjes Cc: James Gowans , Dave Hansen , David Hildenbrand , Matthew Wilcox , Mike Rapoport , Pasha Tatashin , Peter Xu , Alexander Graf , Ashish Kalra , Tom Lendacky , David Woodhouse , Anthony Yznaga , Jason Gunthorpe , Andrew Morton , Frank van der Linden , Vipin Sharma , David Matlack , Steve Rutherford , Erdem Aktas , Alper Gun , Vishal Annapurve , Ackerley Tng , Sagi Shahar , linux-mm@kvack.org, kexec@lists.infradead.org Subject: Re: Pmemfs/guestmemfs discussion recap and open questions Message-ID: References: <5b6613bc-beef-79e6-62ed-23de4dfafe51@google.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <5b6613bc-beef-79e6-62ed-23de4dfafe51@google.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241029_093930_776930_1ED6ED1B X-CRM114-Status: GOOD ( 34.47 ) X-BeenThere: kexec@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "kexec" Errors-To: kexec-bounces+kexec=archiver.kernel.org@lists.infradead.org Hi David, On Fri, Oct 25, 2024 at 11:07:27PM -0700, David Rientjes wrote: > On Wed, 16 Oct 2024, David Rientjes wrote: > > > ----->o----- > > My takeaway: based on the feedback that was provided in the discussion: > > > > - we need an allocator abstraction for persistent memory that can return > > memory with various characteristics: 1GB or not, kernel direct map or > > not, HVO or not, etc. > > > > - built on top of that, we need the ability to carve out very large > > ranges of memory (cloud provider use case) with NUMA awareness on the > > kernel command line > > > > Following up on this, I think this physical memory allocator would also be > possible to use as a backend for hugetlb. Hopefully this would be an > allocator that would be generally useful for multiple purposes, something > like a mm/phys_alloc.c. Can you elaborate on this? mm/page_alloc.c already allocates physical memory :) Or you mean an allocator that will deal with memory carved out from what page allocator manages? > Frank van der Linden may also have thoughts on the above? > > > - we also need the ability to be able to dynamically resize this or > > provide hints at allocation time that memory must be persisted across > > kexec to support the non-cloud provider use case > > > > - we need a filesystem abstraction that map memory of the type that is > > requested, including guest_memfd and then deal with all the fun of > > multitenancy since it would be drawing from a finite per-NUMA node > > pool of persistent memory > > > > - absolutely critical to this discussion is defining what is the core > > infrastructure that is required for a generally acceptable solution > > and then what builds off of that to be more special cased (like the > > cloud provider use case or persistent tmpfs use case) > > > > We're looking to continue that discussion here and then come together > > again in a few weeks. > > > > We'll be looking to schedule some more time to talk about this topic in > the Wednesday, November 13 instance of the Linux MM Alignment Session. > > After that, I think it would be quite useful to break out the set of > people that are interested in persisting guest memory across kexec and KHO > into a separate series to accelerate discussion and next stpes. Getting > the requirements and design locked down are critical, so happy to > facilitate that to any extent possible and welcome everybody interested in > discussing it. > > James, for the guestmemfs discussions, would this work for you? > > Alexander, same question for you regarding the KHO work? > > It's a global community, so the timing won't work for eveyrbody. We'd > plan on sending out summaries of these discussions, such as in this email, > to solicit feedback and ideas from everybody. > > If you're not on the To: or Cc: list already, please email me separatel if > you're interested in participating and then we can find a regular time. > > This is exciting! > -- Sincerely yours, Mike. _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21E33D3A670 for ; Tue, 29 Oct 2024 16:39:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A78D76B009A; Tue, 29 Oct 2024 12:39:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A29486B009B; Tue, 29 Oct 2024 12:39:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F1E76B009C; Tue, 29 Oct 2024 12:39:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 672106B009A for ; Tue, 29 Oct 2024 12:39:32 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E250F12026A for ; Tue, 29 Oct 2024 16:39:31 +0000 (UTC) X-FDA: 82727199786.26.22CB6FE Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf06.hostedemail.com (Postfix) with ESMTP id A0A7A180020 for ; Tue, 29 Oct 2024 16:39:12 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=H49B90HS; spf=pass (imf06.hostedemail.com: domain of rppt@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730219810; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=J/uMRDYpoYpxNSW4eOPyTkcdUUyPA4/9WVe/PLR9H1c=; b=nTWlbXCg5qlM2KlVDRF4wdkxyuWSkbolXVPHRWbkTTNTlOJLgyiIyRzxWRlnd2CYmCGppy b2qp64MF/TSqTS8kjFzobr/l0GOH11rOALPZNWYARNnRyvhZI0R/kNoBk8qSjK4vJPY7Hm 2dygzzLbv3CeXIdbiqDE7vmkvrkDROU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730219810; a=rsa-sha256; cv=none; b=K0omghqRqt6095DS1wclczPVN9iLL19ruRHEiK8HPT9vbZU7ZpwiSkLIITbgkWvi+lKQPB Q5s6HMDQKbL1yFq/nGTqwWtbXSiU0beg3hExHJ5OPYaMkgIRXepWsl1itQBps4RDbcm2fu LRoLj+A2KG1MJiE29h1/e3ZSxGTI5tQ= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=H49B90HS; spf=pass (imf06.hostedemail.com: domain of rppt@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 857A2A42BAE; Tue, 29 Oct 2024 16:37:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0C4A9C4CECD; Tue, 29 Oct 2024 16:39:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1730219969; bh=CvGZshwjj/Jzn5BjBR9puizjZSCPPPEqIfbTu3B8uME=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=H49B90HSTsjTYtBoUnW/LU4uL0T1hght3SqZoXNwwASlW55bQNW1CinjwhB/prcmu iUoxWppmLaa5HNplX4vWLbPVkAG6YuzUv4H6BG/3/HxAKWJ3DWieR4Mz+tcKIBF0ir NQVLIl7RSw4sDuGvQsRj0rCqRGAXZ/g283OZKWIiYR8ecmEAoPRGpPidbnDOkdFGB8 TeW7p5S3BktQ3dneRvFiYg43r1Fnj+DAgk9v+5LCHzStBuN4DbHRIum3ms39Cckv9g aWohmQHz6h3UAMFJzdhnOZ0KvM+87mJQ2NFc4QKpRpimGJymmCS44YVNbGkyc5H6Fu g9jRKgCFBGhxw== Date: Tue, 29 Oct 2024 18:35:30 +0200 From: Mike Rapoport To: David Rientjes Cc: James Gowans , Dave Hansen , David Hildenbrand , Matthew Wilcox , Mike Rapoport , Pasha Tatashin , Peter Xu , Alexander Graf , Ashish Kalra , Tom Lendacky , David Woodhouse , Anthony Yznaga , Jason Gunthorpe , Andrew Morton , Frank van der Linden , Vipin Sharma , David Matlack , Steve Rutherford , Erdem Aktas , Alper Gun , Vishal Annapurve , Ackerley Tng , Sagi Shahar , linux-mm@kvack.org, kexec@lists.infradead.org Subject: Re: Pmemfs/guestmemfs discussion recap and open questions Message-ID: References: <5b6613bc-beef-79e6-62ed-23de4dfafe51@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5b6613bc-beef-79e6-62ed-23de4dfafe51@google.com> X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A0A7A180020 X-Stat-Signature: oxkpj3b44thue1oxszdahxbuc5jhjsbk X-HE-Tag: 1730219952-591590 X-HE-Meta: U2FsdGVkX18+VWNr1g74sL9sHQ2p6bFD2Of/qFkuScYTpHno27IK9Nn4mkmxtObQGkQMsjCMNf7sGDDdDDx3bjn499LKpQpqoH6hparsBCXCqeMIne9y8zx8k8xnnKMUnIYkjy0WVnNoHH0NUf6cRSOgtc6Z30CcpiZywOQKrtuP4Swl/Vz97iKqNMEICThJhmJxCwbyVEJarHN4klarLKB8+/z+ecwsIMuRmyincGw0UYSj1cjsgY8Di098+51aBAImlflV1YlDnUQN+kqhJVLbVm6pvk9VglJzUiKI+uSPXnljDQq2g4e5cFHKZKCloQB2ZP1Z0MSgZUWEFEjo64XzCSmv+fjo3GJCDBtQNPYvGf3YOX0aFJho4JuIwyUYJTrHX18FuyWnv0Z+xCrG9lbK0RwnSh3OboWEVpt4W1+eNJ4GednWWBroUgDHl/+p06bnxh8hehl44VptLJfLqUXpOOnkcyJ7kTQh0gi1j2TpI7nG1M1l6HYjUwCoMbByoHfeL7aAJdPHOiQShWIRav3ttIsc0CC/CjQAAcR6+JRfH5TbZwfixure1RqkpC/G5V+XJLuej9+bwljKuUouunkRi/X0tEdhRKRdpp6QPR5vUC/9DccKOPnHyZ3FzzsbfMms/hDvKCjTbpDRcXicYlNoRn04yA00ooR5OZUXWPfaH1OdjD8W3fM60lNis7om0EKvYTD+cJfyg8FhPWXRdmpj0obQ9phKdWHBchjEPmow1goIDblYZlLfTqI7LU/0MveuVysTWIZRbNa8jZQ7vWh7/ORJVvvknVeL+mR9ufhJY25E56QvbwBgWoJ7nDbKGl5s+d927Au7Zcxxmd9nVEdBgxOh/GaN1P0i9iAVU59dpgYfW6nj2N800reWHEv0l5ABXhQHFkLC7hrOtO0VT5rTSsnItFI7bwpLx0kLoSUr361OCeEW3Q8gQWHeLfMByK+Q9zE1sH+p/yKIwC/ 3ifsQ0S7 KedYMOuTthKaDqlqU4mrFHgB4varMccZdBbNYflPu8WO8U2h6JblVXDI1t94wo5goqTwFg67UvS9blxNse3kkSPtuEHp+Yp/MsVMdcNJxm2Q+cYW3WQSew2rfm4k2d4h65iplu6aB7MmEHBCPMmJCP8o4EBWIrj83OtGqUARIz6Cju6KsjXzqZECJ8brLFzNVzLFvT9s/HPxu5rgeSISj16qmyfFPp2dyYR93/5qRCSSYkS2FJ4Vs0gb6PxEbK7tP9Vp0zdinLbrbDRq0KALcY1IvAUkncVOGIV3ONzoXRmkQXEFF8/Cqo2HXu2P6ieWKPR6BQUolx8Vzb+k= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi David, On Fri, Oct 25, 2024 at 11:07:27PM -0700, David Rientjes wrote: > On Wed, 16 Oct 2024, David Rientjes wrote: > > > ----->o----- > > My takeaway: based on the feedback that was provided in the discussion: > > > > - we need an allocator abstraction for persistent memory that can return > > memory with various characteristics: 1GB or not, kernel direct map or > > not, HVO or not, etc. > > > > - built on top of that, we need the ability to carve out very large > > ranges of memory (cloud provider use case) with NUMA awareness on the > > kernel command line > > > > Following up on this, I think this physical memory allocator would also be > possible to use as a backend for hugetlb. Hopefully this would be an > allocator that would be generally useful for multiple purposes, something > like a mm/phys_alloc.c. Can you elaborate on this? mm/page_alloc.c already allocates physical memory :) Or you mean an allocator that will deal with memory carved out from what page allocator manages? > Frank van der Linden may also have thoughts on the above? > > > - we also need the ability to be able to dynamically resize this or > > provide hints at allocation time that memory must be persisted across > > kexec to support the non-cloud provider use case > > > > - we need a filesystem abstraction that map memory of the type that is > > requested, including guest_memfd and then deal with all the fun of > > multitenancy since it would be drawing from a finite per-NUMA node > > pool of persistent memory > > > > - absolutely critical to this discussion is defining what is the core > > infrastructure that is required for a generally acceptable solution > > and then what builds off of that to be more special cased (like the > > cloud provider use case or persistent tmpfs use case) > > > > We're looking to continue that discussion here and then come together > > again in a few weeks. > > > > We'll be looking to schedule some more time to talk about this topic in > the Wednesday, November 13 instance of the Linux MM Alignment Session. > > After that, I think it would be quite useful to break out the set of > people that are interested in persisting guest memory across kexec and KHO > into a separate series to accelerate discussion and next stpes. Getting > the requirements and design locked down are critical, so happy to > facilitate that to any extent possible and welcome everybody interested in > discussing it. > > James, for the guestmemfs discussions, would this work for you? > > Alexander, same question for you regarding the KHO work? > > It's a global community, so the timing won't work for eveyrbody. We'd > plan on sending out summaries of these discussions, such as in this email, > to solicit feedback and ideas from everybody. > > If you're not on the To: or Cc: list already, please email me separatel if > you're interested in participating and then we can find a regular time. > > This is exciting! > -- Sincerely yours, Mike.