From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5382D329C40 for ; Mon, 12 Jan 2026 07:28:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768202913; cv=none; b=u4v183Hlab/p7GEXFIUV8nFiNny8d9YbKqFOt7YX8Ubc28nhUDQYvSt2sFLhNRgiNyHgHfK0WXpu6dfrAFz8mIkfOocxrsVN2ldT+D/nmb0ZM3SVZ6PzqkKUUHzGDS7nuPXvEhidcwK1f9LUmJVNSl85ntnj96axlzd/kD6KUj8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768202913; c=relaxed/simple; bh=GYkqjRP1d1AkJmY5EyndDuNTaaRtF6vBx0u9mrii2mk=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=jXuWM1CVO+G80EEt9WaYiQzvsokRTeeKVui7RhZwTTLtE2qLESVxBPRPin4Sonf21NdaEEJlji9BfI87OFkqbKgFQ0XI+k+H30fBGwe6nKoP6MjHLxy8MUHeApSLPtp60nj6gSfdO60rrJoHoVY0zfylZH9wikzpL+VVIfPy9W0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=n2JBP/2O; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=XZVDXwO4; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=n2JBP/2O; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=XZVDXwO4; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="n2JBP/2O"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="XZVDXwO4"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="n2JBP/2O"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="XZVDXwO4" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 53BB3336AE; Mon, 12 Jan 2026 07:28:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1768202910; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=76p1BKSyNSE6Ia9N32SzQzLhtPo9wRgYQuaf3Xhfo98=; b=n2JBP/2OtM9QK4ca43tBi2yP4+Eik3Fe2wF7bW9L1OsRAu2aeImqeIo+j/JAHOVAhOEl6O F2YoAIAaRbhcrPqfpDmH+gs3E4UMwrQR0d3EJCNGLhXjv/b1TL8n/z1qKifNUecVD+iq1d To+DCvUWuORNjZ7+V9yPDTi1vGDU/5w= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1768202910; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=76p1BKSyNSE6Ia9N32SzQzLhtPo9wRgYQuaf3Xhfo98=; b=XZVDXwO4OuqJvtL+doLnImo8TwFCB2rWqvMgPd+QO3iFiQAGLfJ5bMQNvn7uFGNslIC8lI fnQ9eFyy282ZDFDw== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b="n2JBP/2O"; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=XZVDXwO4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1768202910; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=76p1BKSyNSE6Ia9N32SzQzLhtPo9wRgYQuaf3Xhfo98=; b=n2JBP/2OtM9QK4ca43tBi2yP4+Eik3Fe2wF7bW9L1OsRAu2aeImqeIo+j/JAHOVAhOEl6O F2YoAIAaRbhcrPqfpDmH+gs3E4UMwrQR0d3EJCNGLhXjv/b1TL8n/z1qKifNUecVD+iq1d To+DCvUWuORNjZ7+V9yPDTi1vGDU/5w= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1768202910; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=76p1BKSyNSE6Ia9N32SzQzLhtPo9wRgYQuaf3Xhfo98=; b=XZVDXwO4OuqJvtL+doLnImo8TwFCB2rWqvMgPd+QO3iFiQAGLfJ5bMQNvn7uFGNslIC8lI fnQ9eFyy282ZDFDw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id C97833EA63; Mon, 12 Jan 2026 07:28:29 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id LBuBL52iZGljRQAAD6G6ig (envelope-from ); Mon, 12 Jan 2026 07:28:29 +0000 Message-ID: Date: Mon, 12 Jan 2026 08:28:29 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH] memory,memory_hotplug: allow restricting memory blocks to zone movable To: Gregory Price , "David Hildenbrand (Red Hat)" Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, osalvador@suse.de, gregkh@linuxfoundation.org, rafael@kernel.org, dakr@kernel.org, akpm@linux-foundation.org, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, rppt@kernel.org, surenb@google.com, mhocko@suse.com References: <20260105203611.4079743-1-gourry@gourry.net> <7f053290-6b9a-4d18-936e-0f28006c79c3@kernel.org> <9575e042-39f4-4f01-80db-34aaaa9312e6@kernel.org> <616f97b7-24e0-4134-a08d-5abaf07a8b09@kernel.org> <20baab84-c8b0-4c46-a550-21b26b975d07@suse.de> <65c246bc-fb10-4cef-8163-3a55bd96f326@kernel.org> Content-Language: en-US From: Hannes Reinecke In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Score: -4.51 X-Spamd-Result: default: False [-4.51 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; ARC_NA(0.00)[]; RCPT_COUNT_TWELVE(0.00)[16]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; SPAMHAUS_XBL(0.00)[2a07:de40:b281:104:10:150:64:97:from]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:rdns,imap1.dmz-prg2.suse.org:helo,suse.de:dkim,suse.de:mid,suse.de:email]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[suse.de:+] X-Spam-Level: X-Rspamd-Action: no action X-Rspamd-Queue-Id: 53BB3336AE X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Spam-Flag: NO On 1/9/26 17:41, Gregory Price wrote: > On Thu, Jan 08, 2026 at 03:16:24PM +0100, David Hildenbrand (Red Hat) wrote: >> On 1/8/26 08:31, Hannes Reinecke wrote: >>> On 1/6/26 21:22, David Hildenbrand (Red Hat) wrote: >>>> On 1/6/26 20:59, Gregory Price wrote: >> >>> For hardware-based scenarios memory will always be removed in >>> larger entities (eg the CXL device), and it's always an 'all-or-nothing' >>> scenario; you cannot remove individual memory blocks on a CXL device. >>> So there the memory block abstraction makes less sense, and it >>> would be good to have a single 'knob' to remove the entire CXL >>> device and all memory blocks on it. >>> Sure, it might take some time, but one doesn't need to worry about >>> restoring the original state if the operation on one block fails. >> >> That's not what I was getting at: >> >> offline_and_remove_memory() can be called on large regions, and it properly >> handles whether we have to back out because some offlining failed. >> >> The issue arises once dax would have to call offline_and_remove_memory() >> multiple times, on non-contiguous areas. Of course, we could handle that by >> providing an interface that consumes multiple memory ranges. >> >> For the DAX use case, I thing we'd really want a way to just use >> >> * add_and_online_memory() [does not exist yet, but ppc does something >> similar] >> * offline_and_remove_memory() >> > > I'm starting to think this issue is actually the result of bad patterns > in the cxl driver - namely using dax as a path to hotplug sysram. > > I suppose either we need a `cxl/dax_region/remove` that handles the > whole operation in one go, or > > we want `cxl/region/commit` to handle hot(un)plug as a single action. > > tl;dr: Split the dax use case from the sysram use case, and make a > cxl sysram driver directly manage hotplug rather than use dax. > Well ... not sure. We are doing fine even currently during boot up; we can align policies and everything to ensure the system comes up with the 'correct' setting Things start to get iffy if one is reconfiguring memory to move from daxdev to system ram and vice versa. Currently we can do this with a simple memory online/offline; with your suggestion we would need to remove the memory, too, when doing that. Might be getting even more awkward as this most likely involves calling the hotplug functions for the CXL device itself ... So not sure if it's a win. But one should try and see where we end up. Cheers, Hannes -- Dr. Hannes Reinecke Kernel Storage Architect hare@suse.de +49 911 74053 688 SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich