From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AAF0C432C0 for ; Wed, 27 Nov 2019 17:15:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E00AD206E0 for ; Wed, 27 Nov 2019 17:15:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="jR0SezjA" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E00AD206E0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 77B356B04C9; Wed, 27 Nov 2019 12:15:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 72AE56B04CA; Wed, 27 Nov 2019 12:15:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F2D56B04CB; Wed, 27 Nov 2019 12:15:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0048.hostedemail.com [216.40.44.48]) by kanga.kvack.org (Postfix) with ESMTP id 455256B04C9 for ; Wed, 27 Nov 2019 12:15:25 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id F0136283D for ; Wed, 27 Nov 2019 17:15:24 +0000 (UTC) X-FDA: 76202708568.25.beam40_86862f4006c55 X-HE-Tag: beam40_86862f4006c55 X-Filterd-Recvd-Size: 8871 Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Wed, 27 Nov 2019 17:15:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1574874923; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:autocrypt:autocrypt; bh=7ZcBy28hizPMfZeEDgKCEYlU/bLmg8GB/X04DeSxh3k=; b=jR0SezjADZO3Z0AveIYSN40cstN8+rktYDWZL371rsdqKL6QRXJ9CvkqbiQoyF86oIsRXk B6CGJVaxc/urXCl+yM8LJOKLO16NVflQG0kdwqSBKaPa6tJfK9iQz8+ygzStBQOd8r87Ca KKR4VJc7T+cJW2FFay/AInAmkkBhHPY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-157-_9rLdKelNoumnUVa3acQ_g-1; Wed, 27 Nov 2019 12:15:20 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A480C18AAFA7; Wed, 27 Nov 2019 17:15:18 +0000 (UTC) Received: from [10.36.116.69] (ovpn-116-69.ams2.redhat.com [10.36.116.69]) by smtp.corp.redhat.com (Postfix) with ESMTP id 624B919C69; Wed, 27 Nov 2019 17:15:17 +0000 (UTC) Subject: Re: [RFC PATCH] mm, page_alloc: avoid page_to_pfn() in move_freepages() To: Michal Hocko , Kefeng Wang Cc: linux-mm@kvack.org, Andrew Morton , Vlastimil Babka References: <20191127102800.51526-1-wangkefeng.wang@huawei.com> <20191127114750.GP20912@dhcp22.suse.cz> <5d11a679-d822-1c41-8798-1dbb285d3bf6@huawei.com> <20191127141340.GA26807@dhcp22.suse.cz> From: David Hildenbrand Autocrypt: addr=david@redhat.com; prefer-encrypt=mutual; keydata= mQINBFXLn5EBEAC+zYvAFJxCBY9Tr1xZgcESmxVNI/0ffzE/ZQOiHJl6mGkmA1R7/uUpiCjJ dBrn+lhhOYjjNefFQou6478faXE6o2AhmebqT4KiQoUQFV4R7y1KMEKoSyy8hQaK1umALTdL QZLQMzNE74ap+GDK0wnacPQFpcG1AE9RMq3aeErY5tujekBS32jfC/7AnH7I0v1v1TbbK3Gp XNeiN4QroO+5qaSr0ID2sz5jtBLRb15RMre27E1ImpaIv2Jw8NJgW0k/D1RyKCwaTsgRdwuK Kx/Y91XuSBdz0uOyU/S8kM1+ag0wvsGlpBVxRR/xw/E8M7TEwuCZQArqqTCmkG6HGcXFT0V9 PXFNNgV5jXMQRwU0O/ztJIQqsE5LsUomE//bLwzj9IVsaQpKDqW6TAPjcdBDPLHvriq7kGjt WhVhdl0qEYB8lkBEU7V2Yb+SYhmhpDrti9Fq1EsmhiHSkxJcGREoMK/63r9WLZYI3+4W2rAc UucZa4OT27U5ZISjNg3Ev0rxU5UH2/pT4wJCfxwocmqaRr6UYmrtZmND89X0KigoFD/XSeVv jwBRNjPAubK9/k5NoRrYqztM9W6sJqrH8+UWZ1Idd/DdmogJh0gNC0+N42Za9yBRURfIdKSb B3JfpUqcWwE7vUaYrHG1nw54pLUoPG6sAA7Mehl3nd4pZUALHwARAQABtCREYXZpZCBIaWxk ZW5icmFuZCA8ZGF2aWRAcmVkaGF0LmNvbT6JAj4EEwECACgFAljj9eoCGwMFCQlmAYAGCwkI BwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEE3eEPcA/4Na5IIP/3T/FIQMxIfNzZshIq687qgG 8UbspuE/YSUDdv7r5szYTK6KPTlqN8NAcSfheywbuYD9A4ZeSBWD3/NAVUdrCaRP2IvFyELj xoMvfJccbq45BxzgEspg/bVahNbyuBpLBVjVWwRtFCUEXkyazksSv8pdTMAs9IucChvFmmq3 jJ2vlaz9lYt/lxN246fIVceckPMiUveimngvXZw21VOAhfQ+/sofXF8JCFv2mFcBDoa7eYob s0FLpmqFaeNRHAlzMWgSsP80qx5nWWEvRLdKWi533N2vC/EyunN3HcBwVrXH4hxRBMco3jvM m8VKLKao9wKj82qSivUnkPIwsAGNPdFoPbgghCQiBjBe6A75Z2xHFrzo7t1jg7nQfIyNC7ez MZBJ59sqA9EDMEJPlLNIeJmqslXPjmMFnE7Mby/+335WJYDulsRybN+W5rLT5aMvhC6x6POK z55fMNKrMASCzBJum2Fwjf/VnuGRYkhKCqqZ8gJ3OvmR50tInDV2jZ1DQgc3i550T5JDpToh dPBxZocIhzg+MBSRDXcJmHOx/7nQm3iQ6iLuwmXsRC6f5FbFefk9EjuTKcLMvBsEx+2DEx0E UnmJ4hVg7u1PQ+2Oy+Lh/opK/BDiqlQ8Pz2jiXv5xkECvr/3Sv59hlOCZMOaiLTTjtOIU7Tq 7ut6OL64oAq+uQINBFXLn5EBEADn1959INH2cwYJv0tsxf5MUCghCj/CA/lc/LMthqQ773ga uB9mN+F1rE9cyyXb6jyOGn+GUjMbnq1o121Vm0+neKHUCBtHyseBfDXHA6m4B3mUTWo13nid 0e4AM71r0DS8+KYh6zvweLX/LL5kQS9GQeT+QNroXcC1NzWbitts6TZ+IrPOwT1hfB4WNC+X 2n4AzDqp3+ILiVST2DT4VBc11Gz6jijpC/KI5Al8ZDhRwG47LUiuQmt3yqrmN63V9wzaPhC+ xbwIsNZlLUvuRnmBPkTJwwrFRZvwu5GPHNndBjVpAfaSTOfppyKBTccu2AXJXWAE1Xjh6GOC 8mlFjZwLxWFqdPHR1n2aPVgoiTLk34LR/bXO+e0GpzFXT7enwyvFFFyAS0Nk1q/7EChPcbRb hJqEBpRNZemxmg55zC3GLvgLKd5A09MOM2BrMea+l0FUR+PuTenh2YmnmLRTro6eZ/qYwWkC u8FFIw4pT0OUDMyLgi+GI1aMpVogTZJ70FgV0pUAlpmrzk/bLbRkF3TwgucpyPtcpmQtTkWS gDS50QG9DR/1As3LLLcNkwJBZzBG6PWbvcOyrwMQUF1nl4SSPV0LLH63+BrrHasfJzxKXzqg rW28CTAE2x8qi7e/6M/+XXhrsMYG+uaViM7n2je3qKe7ofum3s4vq7oFCPsOgwARAQABiQIl BBgBAgAPBQJVy5+RAhsMBQkJZgGAAAoJEE3eEPcA/4NagOsP/jPoIBb/iXVbM+fmSHOjEshl KMwEl/m5iLj3iHnHPVLBUWrXPdS7iQijJA/VLxjnFknhaS60hkUNWexDMxVVP/6lbOrs4bDZ NEWDMktAeqJaFtxackPszlcpRVkAs6Msn9tu8hlvB517pyUgvuD7ZS9gGOMmYwFQDyytpepo YApVV00P0u3AaE0Cj/o71STqGJKZxcVhPaZ+LR+UCBZOyKfEyq+ZN311VpOJZ1IvTExf+S/5 lqnciDtbO3I4Wq0ArLX1gs1q1XlXLaVaA3yVqeC8E7kOchDNinD3hJS4OX0e1gdsx/e6COvy qNg5aL5n0Kl4fcVqM0LdIhsubVs4eiNCa5XMSYpXmVi3HAuFyg9dN+x8thSwI836FoMASwOl C7tHsTjnSGufB+D7F7ZBT61BffNBBIm1KdMxcxqLUVXpBQHHlGkbwI+3Ye+nE6HmZH7IwLwV W+Ajl7oYF+jeKaH4DZFtgLYGLtZ1LDwKPjX7VAsa4Yx7S5+EBAaZGxK510MjIx6SGrZWBrrV TEvdV00F2MnQoeXKzD7O4WFbL55hhyGgfWTHwZ457iN9SgYi1JLPqWkZB0JRXIEtjd4JEQcx +8Umfre0Xt4713VxMygW0PnQt5aSQdMD58jHFxTk092mU+yIHj5LeYgvwSgZN4airXk5yRXl SE+xAvmumFBY Organization: Red Hat GmbH Message-ID: <23594bc4-195a-9dac-cbd9-78698e655ebe@redhat.com> Date: Wed, 27 Nov 2019 18:15:16 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.1.1 MIME-Version: 1.0 In-Reply-To: <20191127141340.GA26807@dhcp22.suse.cz> Content-Language: en-US X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: _9rLdKelNoumnUVa3acQ_g-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 27.11.19 15:13, Michal Hocko wrote: > On Wed 27-11-19 21:13:00, Kefeng Wang wrote: >> >> >> On 2019/11/27 19:47, Michal Hocko wrote: >>> On Wed 27-11-19 18:28:00, Kefeng Wang wrote: >>>> The start_pfn and end_pfn are already available in move_freepages_bloc= k(), >>>> pfn_valid_within() should validate pfn first before touching the page, >>>> or we might access an unitialized page with CONFIG_HOLES_IN_ZONE confi= gs. >>>> >>>> Cc: Andrew Morton >>>> Cc: Michal Hocko >>>> Cc: Vlastimil Babka >>>> Signed-off-by: Kefeng Wang >>>> --- >>>> >>>> Here is an oops in 4.4(arm64 enabled CONFIG_HOLES_IN_ZONE), >>> >>> Is this reproducible with the current upstream kernel? There were large >>> changes in this aread since 4.4 >> >> Our inner tester found this oops twice, but couldn't be reproduced for n= ow, >> even in 4.4 kernel, still trying... >> >> But the page_to_pfn() shouldn't be used in move_freepages(), right? ; ) >=20 > Well, I do agree that going back and forth between page and pfn is ugly. > So this as a cleanup makes sense to me. But you are trying to fix a bug > and that bug should be explained. NULL ptr dereference sounds like a > memmap is not allocated for the particular pfn and this is a bit > unexpected even with holes, at least on x86, maybe arm64 allows that. AFAIK ARM allows that. (and arm64) It's basically CONFIG_HAVE_ARCH_PFN_VALID (and CONFIG_HOLES_IN_ZONE if I am not wrong) commit eb33575cf67d3f35fa2510210ef92631266e2465 Author: Mel Gorman Date: Wed May 13 17:34:48 2009 +0100 [ARM] Double check memmap is actually valid with a memmap has unexpecte= d holes V2 =20 pfn_valid() is meant to be able to tell if a given PFN has valid memmap associated with it or not. In FLATMEM, it is expected that holes always have valid memmap as long as there is valid PFNs either side of the hol= e. In SPARSEMEM, it is assumed that a valid section has a memmap for the entire section. =20 However, ARM and maybe other embedded architectures in the future free memmap backing holes to save memory on the assumption the memmap is nev= er used. The page_zone linkages are then broken even though pfn_valid() returns true. A walker of the full memmap must then do this additional check to ensure the memmap they are looking at is sane by making sure t= he zone and PFN linkages are still valid. This is expensive, but walkers o= f the full memmap are extremely rare. [...] And commit 7b7bf499f79de3f6c85a340c8453a78789523f85 Author: Will Deacon Date: Thu May 19 13:21:14 2011 +0100 ARM: 6913/1: sparsemem: allow pfn_valid to be overridden when using SPA= RSEMEM Side note: I find overriding pfn_valid() extremely ugly ... ... and CONFIG_HOLES_IN_ZONE as well. > But the changelog should be clear about all this rather than paper over > a deeper problem potentially. Please also make sure to involve arm64 > people. >=20 --=20 Thanks, David / dhildenb