From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C90DEB64DA for ; Wed, 19 Jul 2023 08:00:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 15AAA8D004E; Wed, 19 Jul 2023 04:00:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 10BFB8D004B; Wed, 19 Jul 2023 04:00:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F15BB8D004E; Wed, 19 Jul 2023 04:00:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id DDC8A8D004B for ; Wed, 19 Jul 2023 04:00:21 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 9BC4880141 for ; Wed, 19 Jul 2023 08:00:21 +0000 (UTC) X-FDA: 81027613842.07.28DE787 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf19.hostedemail.com (Postfix) with ESMTP id 549731A0020 for ; Wed, 19 Jul 2023 08:00:18 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=JkDmlNul; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf19.hostedemail.com: domain of rppt@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=rppt@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689753618; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=n2g0HT2GVafu4zR3BNeyGHFhQSvs+re8nLMTexJj3dE=; b=ABAWIsygKFOjWuxLI3+MxMiyGWGzR4HlwA+eLNqsP8OWqJKnL3J0elZ4/mcdAn+aJUsCzF PDSoMEsAxYuGGpMEd9gNL4hMrH1oi3qqk2QIk0ykq384MpUC8DqRbARHFECjgCOR6h0mvB wDleH42ZkevuQfZsheXvBBvSytS4ZGc= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=JkDmlNul; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf19.hostedemail.com: domain of rppt@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=rppt@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689753618; a=rsa-sha256; cv=none; b=3Db+nE5khH5O9jEDSgly6Jnj+j9Pj5IABOvnxJ8lDfNG72Ac2PuQt/VwRQGwLSU1u2mioO +N3sJ72iW1VAnEcLFYZEKkQ4Qc9AaRRyix/B+IWnmW73McXjYrRbKkz3P75+ikwJnNVw4O GigwXidHQNqIvdzPSm29/TeHuxSGTps= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 0A24C612FC; Wed, 19 Jul 2023 08:00:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 49482C433C8; Wed, 19 Jul 2023 08:00:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689753616; bh=NTb4KuKKWr4nc9YdxvCXJe6mrt/xu9Zh8G0AeS4XV4w=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=JkDmlNul1ZIiLTrknFa35/tFU8UA4MAK9oBy4J9L/GcA7AU/vugIuFM4V1xVAjCrB suEHTw8Itt8Z6YtWAh03Xc3fuMufsLVMfNp34se3oOcgdObERDMTT4cyY0RThVwvZC 29FT00hI+c0JvnGs/smwONyn3Vq/8tZRmhukMyLb7kRtE03PyW7B5Lb5jBY9nuFGyS pLOCEveAyWuJ/SZ/DK7B0B6BlFLjuaM81AZ2Cs4bB09dErIlmAtyEcgxSpd32+UY1F VPNoiPmlPiKeoIc6TneN0PUiiOczjUge+g9v8x8tzM3GPu85tB4I8MdurTRf0vVebm /Dom2D/x4X9Qw== Date: Wed, 19 Jul 2023 10:59:52 +0300 From: Mike Rapoport To: Michal Hocko Cc: Ross Zwisler , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Matthew Wilcox , Mel Gorman , Vlastimil Babka , David Hildenbrand Subject: Re: collision between ZONE_MOVABLE and memblock allocations Message-ID: <20230719075952.GH1901145@kernel.org> References: <20230718220106.GA3117638@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 549731A0020 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: o1iatbyctxar4zwasjnaqf618bj7z6uh X-HE-Tag: 1689753618-37532 X-HE-Meta: U2FsdGVkX191DvsEhN/OD1ThV1JGE4RKWpzhoiWlz5ukdK98Z9YoCBrZEZHh90xuYcgEh6i3GNskTjbUsI3BlddGhSG5iFIUSYFjhJLQnXp6WSTC5qA48VKSl87YlDgBOq6Ny83+T8fhfetq0CWCo2SoHW2Pwpg3AWyhh0ysdocwMYI6xPG3OmmfNAy+jTKPc2WMirr02YWA9aB4dSyJ4H2kB8L7hnU6/YHPao6qx4FPDEwyh3yzfVak6CMmuYZqpdGV3xFZnUs6IZNwz5dOhkmn/dEUb++nrV8SJxYowj3DMs0XwB1BdKz/9rF0Am1cJhnNvLQD/q1WpVuwDE3tmkye6niu5S3cZN3JOQWtFksNV7PUVzO+JUbEvwb1jvDjZS4W/6UbgonlqBLDLknRJ8V7h36VdnyCoN48E/rFL/JYZScwJIwpbtdZswjZIpMoobjJcGaKBI2pkIxaACmn2bfiWwiTbcu1JEP8NGDIrZLYkMXOqL3qaRJLyr4RIdiIUAuDBpM1egTbSVrxHDzhpyouPKGDRLiwlewYZFB5jj7QP1IqXm2A/t+WLsP/N87PkGwKGthTqjGi8QgqbO6lGzYTd7v8hO7V2xLXsN6VoWY10Di5Ths/+i90fdcKuALAgmNoN/RjVtyg2F5zfzBzYSOn/wfKZ4/ixpGop6qTGiSKD3Hl2HqA6Uh4OZDPog23IXp3rhY3LLjwUY/cnWEwx3itXiR75+7p4KjsXoPJX/fY0IDqmVFIK7ni50BZEI/q27ShDERgi8c6x9I8QlW3fi0OoatmH6Y58SGROsBIhFcusyTqr0wxan4rk+G8CwIWReflDxTmHxWrtIBAS3rM4FWd/az8V/NCxzbajB697UZC2bSSkrfYxVOjRdfDpXUDUnO52lwMgRJKoQf4boFTeng5cpqjJ1aqUNXN9gJVQnQY8lJkyXYQuJrtwA1Z3GjIElPsqHs0QsOBjHBrLFQ jPIbKL44 bMw+w4BBc5CYI/APnNqaCkMb4w4mN3/h5q9lmoBhKAuCFiFepUYAI4OVrnihHH+5FZI1jK4Fu8a4IlLlrWN/50lmQub+JzHw1QY8rqI0FX3EbBS0vjqK1RbDYdu4QUaXcs6A1pD2REKdi94YiT7PUp+SsAEi/uIt0AywAedVnnS37fReopq1GqBb/m3rru6ps7oNOG4OblpXV/heRAv/aSnRpRebcylAOp85210qvazWshYOGOHR6J5WGbwtQfAe2SFNWa4XSEdzUSu8Q7dBQhoDlFqa2teT8TVLah++2TYkxRBApbFX+lJF7uwToIe3SODpr9MxBHKD6fQXc723MBR2UYQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jul 19, 2023 at 08:14:48AM +0200, Michal Hocko wrote: > On Tue 18-07-23 16:01:06, Ross Zwisler wrote: > [...] > > I do think that we need to fix this collision between ZONE_MOVABLE and memmap > > allocations, because this issue essentially makes the movablecore= kernel > > command line parameter useless in many cases, as the ZONE_MOVABLE region it > > creates will often actually be unmovable. > > movablecore is kinda hack and I would be more inclined to get rid of it > rather than build more into it. Could you be more specific about your > use case? > > > Here are the options I currently see for resolution: > > > > 1. Change the way ZONE_MOVABLE memory is allocated so that it is allocated from > > the beginning of the NUMA node instead of the end. This should fix my use case, > > but again is prone to breakage in other configurations (# of NUMA nodes, other > > architectures) where ZONE_MOVABLE and memblock allocations might overlap. I > > think that this should be relatively straightforward and low risk, though. > > > > 2. Make the code which processes the movablecore= command line option aware of > > the memblock allocations, and have it choose a region for ZONE_MOVABLE which > > does not have these allocations. This might be done by checking for > > PageReserved() as we do with offlining memory, though that will take some boot > > time reordering, or we'll have to figure out the overlap in another way. This > > may also result in us having two ZONE_NORMAL zones for a given NUMA node, with > > a ZONE_MOVABLE section in between them. I'm not sure if this is allowed? > > Yes, this is no problem. Zones are allowed to be sparse. The current initialization order is roughly * very early initialization with some memblock allocations * determine zone locations and sizes * initialize memory map - memblock_alloc(lots of memory) * lots of unrelated initializations that may allocate memory * release free pages from memblock to the buddy allocator With 2) we can make sure the memory map and early allocations won't be in the ZONE_MOVABLE, but we'll still may have reserved pages there. > > If > > we can get it working, this seems like the most correct solution to me, but > > also the most difficult and risky because it involves significant changes in > > the code for memory setup at early boot. > > > > Am I missing anything are there other solutions we should consider, or do you > > have an opinion on which solution we should pursue? > > If this really needs to be addressed than 2) is certainly a more robust > approach. > -- > Michal Hocko > SUSE Labs -- Sincerely yours, Mike.