From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8AECC4332F for ; Wed, 4 Jan 2023 19:00:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0D3D98E0002; Wed, 4 Jan 2023 14:00:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0829B8E0001; Wed, 4 Jan 2023 14:00:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6F488E0002; Wed, 4 Jan 2023 14:00:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id D3AC08E0001 for ; Wed, 4 Jan 2023 14:00:19 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A1069AB247 for ; Wed, 4 Jan 2023 19:00:19 +0000 (UTC) X-FDA: 80318032158.21.069F2F4 Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) by imf22.hostedemail.com (Postfix) with ESMTP id D1851C0014 for ; Wed, 4 Jan 2023 19:00:17 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Qw3w27Ym; spf=pass (imf22.hostedemail.com: domain of f.fainelli@gmail.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=f.fainelli@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1672858817; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=d7zBSg+SSbBcixAvOGJQFrPhxW41/S4lHbiSyTmsGHQ=; b=U6q4LYUk0a09n80gHuBrnBEbURaO7JAaEfYG+StC6lyXgVtE5wm8SBht3POmuNHjx9Fr/G z8c/9ZnSXKQq4B/uSoALx4dOCz1OfIWd+KDMmJop2x/gghK/jl5nNnJ0xhIDmn4hDw+EDH BVOrR0xyGTLbJuuBTI/Cn5XKMkgCwZI= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Qw3w27Ym; spf=pass (imf22.hostedemail.com: domain of f.fainelli@gmail.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=f.fainelli@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1672858817; a=rsa-sha256; cv=none; b=iQ92qCeW69ApYhfRfC4W+hdkZJV8VxCYp2KjjWqRqLOTt33WxcHCQpOM/OcHh187p5jzt6 0frgZQxK6yiLE/45RVjOYtcgCBVTwIxXf4K8ZBZqgcDZQpxtb/H/3ojxbspBGWSOcbxJ5t 7Rd6juzR5KeRYFXEEYBmSi6/sd0dL5k= Received: by mail-pg1-f174.google.com with SMTP id 7so22900604pga.1 for ; Wed, 04 Jan 2023 11:00:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=d7zBSg+SSbBcixAvOGJQFrPhxW41/S4lHbiSyTmsGHQ=; b=Qw3w27YmS66E2CaXhintEFgx6Jw+tjjWRYmxfAqJbyNREkX8gVjjR7iP1/nUKeAehb I/MaelP66B+JHVXuOvhRlmqNXilHMX3Wot2DWlnpv18zHeKY4OibS1G4jkOZZQk5yqwT 5uGPTJs3Kd69R6VTja87Kg2xunoMQ635sfFtjQczwaVffNpS1HrlHy9G+wDmuhXvbd01 l9XSGjUjsj2jQo1G2hDoAtgpBslb33lY6UblfiSzrWgaEhbi3USlLTmfqUg42pryjRpy Qr4J5GXa6Aj2RLWS/1iwdfmUpJ9sr3V338svmV2/n/5GPECgdtH/+5RFLlq1xWK4wKpB Cx/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=d7zBSg+SSbBcixAvOGJQFrPhxW41/S4lHbiSyTmsGHQ=; b=0fCwR8iEsmLEq4oERi8b32lcfVswOy1EMn+SF43/2q8MFa6OneBFRRh4AH68/8GIW8 iShtEi4gOfZFH/MqNbTVrfzQSwT+n6W+X9MnaL2tCXtzFsLJeLcDgo0850LH59s7J5T7 16aMKabu83fllFA56yHwfIy0QhAhwDHUQ+azn2TBgcLrQq7hz5j78/l5PwjjQSYQ4aza X2DZ8Gb0CjGiPE0iEXFL0xjPnAD9PXb9iuecppA4A69RsTKGXYB69zZwE9Y5Y8GrV/MM Wwa18hTF49bHX9zD4MLH+bIfPssxeVcj7LDOfhH0yMjvF3FWbzqcgeOG3Hl7SwRmg/Xu vkGw== X-Gm-Message-State: AFqh2koW7dL+pRSEnv/EWkg3VS4ozRM75jpn+8nJSGN/xxh2HxDN20su BvscSge+qWoa6WQ6natCuhc= X-Google-Smtp-Source: AMrXdXu5Ff797FEOTLEuSFJqv9wiZeWBUVb4XgMdEiOoa8K0FKSMQ+KcS2myr2ytEalJEc/a4uP6qA== X-Received: by 2002:a05:6a00:331a:b0:581:8632:b152 with SMTP id cq26-20020a056a00331a00b005818632b152mr28834495pfb.31.1672858816372; Wed, 04 Jan 2023 11:00:16 -0800 (PST) Received: from [10.67.48.245] ([192.19.223.252]) by smtp.googlemail.com with ESMTPSA id h7-20020aa796c7000000b005818d429d98sm14054001pfq.136.2023.01.04.11.00.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 04 Jan 2023 11:00:15 -0800 (PST) Message-ID: Date: Wed, 4 Jan 2023 11:00:11 -0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: [PATCH v3 0/9] mm: introduce Designated Movable Blocks Content-Language: en-US To: David Hildenbrand , Doug Berger , Andrew Morton , Mel Gorman Cc: Jonathan Corbet , Mike Rapoport , Borislav Petkov , "Paul E. McKenney" , Neeraj Upadhyay , Randy Dunlap , Damien Le Moal , Muchun Song , Vlastimil Babka , Johannes Weiner , Michal Hocko , KOSAKI Motohiro , Mike Kravetz , Oscar Salvador , Joonsoo Kim , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20221020215318.4193269-1-opendmb@gmail.com> <0c4e35de-f790-5399-c812-ff90a4ab7531@redhat.com> From: Florian Fainelli In-Reply-To: <0c4e35de-f790-5399-c812-ff90a4ab7531@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Stat-Signature: hsuqr5ombjhbbrsazwo4bissmhijd1u9 X-Rspam-User: X-Rspamd-Queue-Id: D1851C0014 X-Rspamd-Server: rspam06 X-HE-Tag: 1672858817-371706 X-HE-Meta: U2FsdGVkX19X+DHXLz0R22zEDkS9dBHtee79DYvJ7QPeDR/9i4Ci8wjCcIMtdKqr+JAMDTikzF4vf5FbzMUZhNMo83yFLUHoXrD8ZjjN1rN6p6ggfb1VtPq7QGjO/ExW0ycVDbMMihpbl4S45imWjJH9sRaW3ck0rXXrCDMGDASBzBIwM6hcRJw4Br8BZ2mJ9RxSGGrVniI2s8u+S2+ijq//r0XS2hF4KxsOVFBizGR3Tr0aN7h0Rm0Xn7nCyXcU9hsFWypyEFw9QE7GOgpOyPgfsEOPoo/X0O+lI2RWDufeT677a8LRLfxYKiWXC2GRd3fTgnCXlR4sRVPJhHEECkLj4OX8xcaEVQ7fHI7yYXM79f3B07uT2ji9geQu8UIw4uUZ6O7fNTqyuQAt5zlygJ8X/ZxXvkM/rcWIr8OkCpcSrgHkJ+Tt71eZY+uT7v4eK47/HvsRDP/+B5er8WJOYiEoPuoSuKi0SnOtt0FpLvBMEZmIigAzu0asrf3OJYESTbf4TcnHqAunVUGSHpMrDNXD+EovWZ8LFqDMHvmMzXzxzC6KQOlmVUmJpOZuWnaT493xKC6KDpPr9T0qOtCuudI9vtniiQFk0bRI0L9zHqNmnYKh3qb/HftRbGIXCcppyGsEC4yuE6BI2LSkD5i9fS1nLH+zzPh1sqePr5XZ/NEG9o8uvLUJv+Btu4kDXoHb7xajcIJ0bK0D8qwTh/mwSWYW+w/5CKwAGtByEq99vExJ6lP71dWt8XQN85XCBughEf6a7MC9LXpH92FnL2ySN1rD5Z+MfuurMKa307jQ7Hkr6Q7xcxWgNWDiPuEN+/00MyWW4ETEAtUzbZd6l1yGU9Qx0eNxhBFKebiirIccLnjZIC9PS0PRAHIvHDDGR/ZqjPNyAoeXPps9Rr4CDWhDaMT5X+zyd6KhnWopXHZdQPLloeWneq2vc8X3T9csIXkmVQF5kowEN+HH9GVMSEg +Xsh8qpW FMgkOksmlp7FU0p8nhvfDBq39aw2XsNoaICE4bDiWSfhwwbTH3oN3SCElmnNkYEIFJHbfooGk/b9RTi/WVy7XxMptktBx4fcxr9oqsT24Nzg0xSSuUDH/WvD296iG6W79IDeq X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 1/4/23 07:56, David Hildenbrand wrote: > On 04.01.23 00:43, Florian Fainelli wrote: >> On 10/20/22 14:53, Doug Berger wrote: >>> MOTIVATION: >>> Some Broadcom devices (e.g. 7445, 7278) contain multiple memory >>> controllers with each mapped in a different address range within >>> a Uniform Memory Architecture. Some users of these systems have >>> expressed the desire to locate ZONE_MOVABLE memory on each >>> memory controller to allow user space intensive processing to >>> make better use of the additional memory bandwidth. >>> Unfortunately, the historical monotonic layout of zones would >>> mean that if the lowest addressed memory controller contains >>> ZONE_MOVABLE memory then all of the memory available from >>> memory controllers at higher addresses must also be in the >>> ZONE_MOVABLE zone. This would force all kernel memory accesses >>> onto the lowest addressed memory controller and significantly >>> reduce the amount of memory available for non-movable >>> allocations. >>> >>> The main objective of this patch set is therefore to allow a >>> block of memory to be designated as part of the ZONE_MOVABLE >>> zone where it will always only be used by the kernel page >>> allocator to satisfy requests for movable pages. The term >>> Designated Movable Block is introduced here to represent such a >>> block. The favored implementation allows extension of the >>> 'movablecore' kernel parameter to allow specification of a base >>> address and support for multiple blocks. The existing >>> 'movablecore' mechanisms are retained. >>> >>> BACKGROUND: >>> NUMA architectures support distributing movablecore memory >>> across each node, but it is undesirable to introduce the >>> overhead and complexities of NUMA on systems that don't have a >>> Non-Uniform Memory Architecture. >>> >>> Commit 342332e6a925 ("mm/page_alloc.c: introduce kernelcore=mirror >>> option") >>> also depends on zone overlap to support sytems with multiple >>> mirrored ranges. >>> >>> Commit c6f03e2903c9 ("mm, memory_hotplug: remove zone restrictions") >>> embraced overlapped zones for memory hotplug. >>> >>> This commit set follows their lead to allow the ZONE_MOVABLE >>> zone to overlap other zones. Designated Movable Blocks are made >>> absent from overlapping zones and present within the >>> ZONE_MOVABLE zone. >>> >>> I initially investigated an implementation using a Designated >>> Movable migrate type in line with comments[1] made by Mel Gorman >>> regarding a "sticky" MIGRATE_MOVABLE type to avoid using >>> ZONE_MOVABLE. However, this approach was riskier since it was >>> much more instrusive on the allocation paths. Ultimately, the >>> progress made by the memory hotplug folks to expand the >>> ZONE_MOVABLE functionality convinced me to follow this approach. >>> >> >> Mel, David, does the sub-thread discussion with Doug help ensuring that >> all of the context is gathered before getting into a more detailed patch >> review on a patch-by-patch basis? >> >> Eventually we may need a fairly firm answer as to whether the proposed >> approach has any chance of landing upstream in order to either commit to >> in subsequent iterations of this patch set, or find an alternative. > > > As raised, I'd appreciate if less intrusive alternatives could be > evaluated (e.g., fake NUMA nodes and being ablee to just use mbind(), > moving such memory to ZONE_MOVABLE after boot via something like daxctl). This is not an option with the environment we have to ultimately fit in which is Android TV utilizing the GKI kernel which does not enable NUMA and probably never will, and for similar reasons bringing a whole swath of user-space tools like daxctl may not be practical either, from both a logistical perspective (simply getting the tools built with bionic, accepted etc.) as well as system configuration perspective. > > I'm not convinced that these intrusive changes are worth it at this > point. Further, some of the assumptions (ZONE_MOVABLE == user space) are > not really future proof as I raised. I find this patch set reasonably small in contrast to a lot of other mm/ changes, what did you find intrusive specifically? AFAICT, there only assumption that is being made is that ZONE_MOVABLE contains memory that can be moved, but even if it did not in the future, there should hopefully be enough opportunities, given a large enough DMB region to service the allocation requests of its users. I will go back and read your comment to make sure I don't misunderstand it. Thanks -- Florian