From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 692ACC4332F for ; Wed, 12 Oct 2022 23:58:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 79BBE6B0071; Wed, 12 Oct 2022 19:58:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 74C2D6B0073; Wed, 12 Oct 2022 19:58:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5ECBA6B0074; Wed, 12 Oct 2022 19:58:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 4A2BB6B0071 for ; Wed, 12 Oct 2022 19:58:00 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 177A140308 for ; Wed, 12 Oct 2022 23:58:00 +0000 (UTC) X-FDA: 80013963120.21.89A88D2 Received: from mail-qk1-f170.google.com (mail-qk1-f170.google.com [209.85.222.170]) by imf01.hostedemail.com (Postfix) with ESMTP id BF17140031 for ; Wed, 12 Oct 2022 23:57:58 +0000 (UTC) Received: by mail-qk1-f170.google.com with SMTP id z30so148747qkz.13 for ; Wed, 12 Oct 2022 16:57:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=FmGTt81upNKJCIsUibynyofUcfo56u99EQM7Huk1XAY=; b=CgtNenL/z9CyDUzMXKS8FMIVEYPcOxh4jHQx/8pN6R0+6F7+VW5sS7NklPE7bmXIeQ XlqeopIhYfCMzOFNlev/XbSVskzyfM+wIFWhpM5HxUwVDqsV4n8YGPSvO72n0aXlTjZb pQx8vjEUO+T/W4vYbflHq2m43HAjcE/ipFf0Jwc0UxGVnMEnWJo/hPqEyZ4SQwX3dVXw TTq7+BTvX7xIGQBR5X+0pDLXCcUCgwQ9VglUwTAN+xsGuy9x7mgu0MrPicOHcdqczPe0 UYenH3Qnbe+aQXpO2QU13S23tsaJVRGA3Je281PDBV4lb4AmOvyZa+SJrINuVIXyiv/q 0wIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FmGTt81upNKJCIsUibynyofUcfo56u99EQM7Huk1XAY=; b=JieTzojzeS6mqAVm/Qw/Kef89to4rbprh9w9CZJorhDb3yd5M2vDbpqz3gd63QThUB Htd4nedNwPwyqXIFbWCHc0FGXQPLr9N68XpeRO4fN5rV8JsxNJGX37dxD5X9AZZFvdvO X/Re6ioddmXtyrIaxRXLiPfI8elaFKgnq3Zm7ZKdy1DVA6p1uGboxhEmZfTdoJIcdO3b AQet6vin4FYJYEJWeyQIoc1C+BClOIbh5oY/W576wpAt1JEexZXUhFijpyRmHgbLcx3J hCjW5fIwSqHsSPucvFTLEd45TLclp0o5LzCuxJNwWdla6fDgtZ7+rm7UhMDqJZF9JxfV u+6Q== X-Gm-Message-State: ACrzQf3dsnwrkWtdycdqoigjHsJKdJ30jAlXYgYrQCPUu4loutei/NcM ldHbBBpwx2mU0KoTT422XsQ= X-Google-Smtp-Source: AMsMyM6sAy/Ash+9v4opTMFFF5CqbSfOxYuzqswsETtOQTHE46P24Et4A6MkmJdYKowAg6hvDglWmw== X-Received: by 2002:a05:620a:43a8:b0:6ee:9b14:779e with SMTP id a40-20020a05620a43a800b006ee9b14779emr3519974qkp.343.1665619078010; Wed, 12 Oct 2022 16:57:58 -0700 (PDT) Received: from [10.69.53.73] ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id h19-20020a05620a401300b006eeb185c209sm1256588qko.50.2022.10.12.16.57.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 12 Oct 2022 16:57:57 -0700 (PDT) Message-ID: <0f038010-ed83-55bb-70a5-24f5c6d68666@gmail.com> Date: Wed, 12 Oct 2022 16:57:53 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.1.2 Subject: Re: [PATCH v2 2/9] mm/vmstat: show start_pfn when zone spans pages Content-Language: en-US To: David Hildenbrand , Andrew Morton Cc: Jonathan Corbet , Mike Rapoport , Borislav Petkov , "Paul E. McKenney" , Neeraj Upadhyay , Randy Dunlap , Damien Le Moal , Muchun Song , KOSAKI Motohiro , Mel Gorman , Mike Kravetz , Florian Fainelli , Oscar Salvador , Michal Hocko , Joonsoo Kim , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20220928223301.375229-1-opendmb@gmail.com> <20220928223301.375229-3-opendmb@gmail.com> <8e61d0f4-0c40-6c2d-da60-fa97e2ee7530@redhat.com> <84ee3d9e-9579-d3f2-fe5a-ec6ec4a2710a@redhat.com> From: Doug Berger In-Reply-To: <84ee3d9e-9579-d3f2-fe5a-ec6ec4a2710a@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665619078; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FmGTt81upNKJCIsUibynyofUcfo56u99EQM7Huk1XAY=; b=JXHHcs9wHLQTYdH5JpLKknKyEMIMGjUg597X49ruO6BvKs+CSc88B0DPaQauy36oENBmYB rjMS1PQnJ/M1ihGH7LaWnYcPdnPOAVN1EKkQYJidBQQQDWX2bet+pJFe3kXVEoBK1aQp+m j60vCRnP40hGXRCSRwN6HzFZQXmJ0Ak= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="CgtNenL/"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of opendmb@gmail.com designates 209.85.222.170 as permitted sender) smtp.mailfrom=opendmb@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665619078; a=rsa-sha256; cv=none; b=ywP29WrTTtkq6tRmHDxSJqleYlkYp7wcIgKo238DpuMWx/I41ZBbm4hDXR2LjFPSYoZO04 k2FmQmhiasi0lDOlDYjOhYucVOu5G/ZgCdh7HYTygzrzka87aO2Wwn1WSfO7xuNy82eSJb LFH05jzvLcqF3Ne/CAGRgnOJcam2ns0= X-Stat-Signature: q57qakm6doc1gegsy9qeqhpawwhwaynb X-Rspamd-Queue-Id: BF17140031 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="CgtNenL/"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of opendmb@gmail.com designates 209.85.222.170 as permitted sender) smtp.mailfrom=opendmb@gmail.com X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1665619078-91238 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 10/5/2022 11:09 AM, David Hildenbrand wrote: > On 01.10.22 03:28, Doug Berger wrote: >> On 9/29/2022 1:15 AM, David Hildenbrand wrote: >>> On 29.09.22 00:32, Doug Berger wrote: >>>> A zone that overlaps with another zone may span a range of pages >>>> that are not present. In this case, displaying the start_pfn of >>>> the zone allows the zone page range to be identified. >>>> >>> >>> I don't understand the intention here. >>> >>> "/* If unpopulated, no other information is useful */" >>> >>> Why would the start pfn be of any use here? >>> >>> What is the user visible impact without that change? >> Yes, this is very subtle. I only caught it while testing some >> pathological cases. >> >> If you take the example system: >> The 7278 device has four ARMv8 CPU cores in an SMP cluster and two >> memory controllers (MEMCs). Each MEMC is capable of controlling up to >> 8GB of DRAM. An example 7278 system might have 1GB on each controller, >> so an arm64 kernel might see 1GB on MEMC0 at 0x40000000-0x7FFFFFFF and >> 1GB on MEMC1 at 0x300000000-0x33FFFFFFF. >> > > Okay, thanks. You should make it clearer in the patch description -- > especially how this relates to DMB. Having that said, I still have to > digest your examples: > >> Placing a DMB on MEMC0 with 'movablecore=256M@0x70000000' will lead to >> the ZONE_MOVABLE zone spanning from 0x70000000-0x33fffffff and the >> ZONE_NORMAL zone spanning from 0x300000000-0x33fffffff. > > Why is ZONE_MOVABLE spanning more than 256M? It should span > > 0x70000000-0x80000000 > > Or what am I missing? I was working from the notion that the classic 'movablecore' implementation keeps the ZONE_MOVABLE zone the last zone on System RAM so it always spans the last page on the node (i.e. 0x33ffff000). My implementation moves the start of ZONE_MOVABLE up to the lowest page of any defined DMBs on the node. I see that memory hotplug does not behave this way, which is probably more intuitive (though less consistent with the classic zone layout). I could attempt to change this in a v3 if desired. > >> >> If instead you specified 'movablecore=256M@0x70000000,512M' you would >> get the same ZONE_MOVABLE span, but the ZONE_NORMAL would now span >> 0x300000000-0x32fffffff. The requested 512M of movablecore would be >> divided into a 256MB DMB at 0x70000000 and a 256MB "classic" movable >> zone start would be displayed in the bootlog as: >> [    0.000000] Movable zone start for each node >> [    0.000000]   Node 0: 0x000000330000000 > > > Okay, so that's the movable zone range excluding DMB. > >> >> Finally, if you specified the pathological >> 'movablecore=256M@0x70000000,1G@12G' you would still have the same >> ZONE_MOVABLE span, and the ZONE_NORMAL span would go back to >> 0x300000000-0x33fffffff. However, because the second DMB (1G@12G) >> completely overlaps the ZONE_NORMAL there would be no pages present in >> ZONE_NORMAL and /proc/zoneinfo would report ZONE_NORMAL 'spanned >> 262144', but not where those pages are. This commit adds the 'start_pfn' >> back to the /proc/zoneinfo for ZONE_NORMAL so the span has context. > > ... but why? If there are no pages present, there is no ZONE_NORMAL we > care about. The zone span should be 0. Does this maybe rather indicate > that there is a zone span processing issue in your DMB implementation? My implementation uses the zones created by the classic 'movablecore' behavior and relocates the pages within DMBs. In this case the ZONE_NORMAL still has a span which gets output but no present pages so the output didn't show where the zone was without this patch. This is a convenience to avoid adding zone resizing and destruction logic outside of memory hotplug support, but I could attempt to add that code in a v3 if desired. > > Special-casing zones based on DMBs feels wrong. But most probably I am > missing something important :) > Thanks for making me aware of your confusion so I can attempt to make it clearer. -Doug