From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17653EB64DD for ; Thu, 15 Jun 2023 07:47:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3D7826B0072; Thu, 15 Jun 2023 03:47:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 35F896B0074; Thu, 15 Jun 2023 03:47:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1D9BF8E0001; Thu, 15 Jun 2023 03:47:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 082BB6B0072 for ; Thu, 15 Jun 2023 03:47:05 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C630940A9E for ; Thu, 15 Jun 2023 07:47:04 +0000 (UTC) X-FDA: 80904201168.08.29A9E05 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf07.hostedemail.com (Postfix) with ESMTP id 626C24001F for ; Thu, 15 Jun 2023 07:47:02 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="UGOPw/Aa"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf07.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686815222; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oAC3VUgTK58AFtVKiltpsFbsCaEbRXjpjSi3HmTg4+U=; b=ageKvAPPHkZ7/+ejzDqPOuwqI9i30Rp4DjQKuT5+bZYan200HweWJdpF3n1jNQUVLlPyci n7ntE21XNHC214+AzRwnelTQY2NMAC4ArGkp6TTzSjL3qG2aAcGSj+H1we6We0zcAWV99K zWbQOn20TqiE/Z1gb/vHMOz+au8FoYs= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="UGOPw/Aa"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf07.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686815222; a=rsa-sha256; cv=none; b=1cauh4/wvVJ3OHGvGSL8X0nGg+et3+97HkP2FU824H0nMNToMMotqgIQ07GxP4L5rPXPQN TpaZx3SqnTnDcIwg0A6Zrwhyb5khJVbTK4zHrDpYOYP8fkCPj6KJOs7E5JHh7BiAoMZV5v L04Rti3WPH8Tt8GMPb0/OlZD1pD6lqM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686815220; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oAC3VUgTK58AFtVKiltpsFbsCaEbRXjpjSi3HmTg4+U=; b=UGOPw/AazkXOvJ5gbIz6X6gdHsW+UezXo3tQgNr69jd7MkQNz7I7on/rHX36RjDKzv3d5V nwS/XkI66IOh50WesUROVxO/7dVyrVBaPSgBAuOnKJ/ZZMpT2I0CRnf4ls1TU7z4fwsAKt LgfyF9rLljC4npEBKXy1YVs+DEj4rPM= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-84-_G03RfShNDuBmxeCFVkwRA-1; Thu, 15 Jun 2023 03:46:57 -0400 X-MC-Unique: _G03RfShNDuBmxeCFVkwRA-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-3f80e7e15b0so9660555e9.2 for ; Thu, 15 Jun 2023 00:46:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686815216; x=1689407216; h=content-transfer-encoding:in-reply-to:subject:organization:from :references:cc:to:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=oAC3VUgTK58AFtVKiltpsFbsCaEbRXjpjSi3HmTg4+U=; b=Oez4UdeSjxYdXlI3bTRP9ZVjXHHd9N5ihj1TpaOGtJ6/Lp34SjNo/Nl6lEDLci93XE nkt/mM3jn4dbDW1vfhzGp8vD2sCQQsKtJRxct03i0popRGJYXaFfkMf1nRmdwfbXIOv2 0S3UKlPWt3AcQ8DaASQwW4ucwajtmA1yUd1fR1+EHEL/IyvC2mnAhQzNzOK9lmRMo57f 7EZ6Ge4oETXXFQBLaPIZOfPGV+2LJLlMEupelbTWtFJRPiPy132tsFpHPWw2SZtGairN duqU4dupjqUvfOhLNmMP13t0wRjumWAcqPQ1bFjIwCkWIaXVo8uPdUw4peo2d9h2fTP9 RDIg== X-Gm-Message-State: AC+VfDyjVhGiudyC3nU64XVXiq9f1k0QBIiEcgpaGnX58fWMOS4c/22c nmBqo5TEr/SiiizzOTJzuQ5kecpv8tYrSpy0DNoQVs5C8tGCTAg1Pd3hOU3YNTC78NuUvOllyk8 h+MzEWq1ctbw= X-Received: by 2002:adf:f592:0:b0:30f:c679:793a with SMTP id f18-20020adff592000000b0030fc679793amr7440795wro.3.1686815216294; Thu, 15 Jun 2023 00:46:56 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6ByPKOjUuV6aqVpU6Kxdu/rKoMFQNGQoBKGhCydv3UJZ41+p4N/AV9s1LjZ5f/ZpfOQzYKbA== X-Received: by 2002:adf:f592:0:b0:30f:c679:793a with SMTP id f18-20020adff592000000b0030fc679793amr7440781wro.3.1686815215804; Thu, 15 Jun 2023 00:46:55 -0700 (PDT) Received: from ?IPV6:2a09:80c0:192:0:5dac:bf3d:c41:c3e7? ([2a09:80c0:192:0:5dac:bf3d:c41:c3e7]) by smtp.gmail.com with ESMTPSA id k1-20020a5d6e81000000b0030e5ccaec84sm20400430wrz.32.2023.06.15.00.46.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 15 Jun 2023 00:46:55 -0700 (PDT) Message-ID: <5b5e7dd0-d60b-ca46-215c-f59947b805fe@redhat.com> Date: Thu, 15 Jun 2023 09:46:53 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 To: "Huang, Ying" , Baolin Wang Cc: Mel Gorman , akpm@linux-foundation.org, vbabka@suse.cz, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <770f9f61472b24b6bc89adbd71a77d9cf62bb54f.1686646361.git.baolin.wang@linux.alibaba.com> <20230614095501.m4porztaibchrgwx@techsingularity.net> <87ilbpo1d9.fsf@yhuang6-desk2.ccr.corp.intel.com> <5f340d98-4ee0-35a9-58ed-943834d68042@linux.alibaba.com> <87fs6tfaw5.fsf@yhuang6-desk2.ccr.corp.intel.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [PATCH v2] mm: compaction: skip memory hole rapidly when isolating migratable pages In-Reply-To: <87fs6tfaw5.fsf@yhuang6-desk2.ccr.corp.intel.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 626C24001F X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: kpfa1spon1se5pob7tkatmjp9yutjfex X-HE-Tag: 1686815222-408304 X-HE-Meta: U2FsdGVkX19ii/VR3pWVS88vPYrpYcu2clbZpct/jEbnAlWfIBfXtPBe+tTdwlk/LMpTp8CuAh4/gyMnoOc33O+nVnZJQFpijANPoIPUPFs+HHtzxNPxITA1JZxalQes+n0w07XbwTPLct3Zrpw/4AoVonOi5eGgUeY4RNiHzO+jlKLJBoK5zP844EPFRf9n7i4diEXNgD1vTGVAK98c24EwzQhXS3rB2IhERD8zMhPW3xQyi1c8zSmiosS2Jv2RdDMI5uTkiCOOz4/oWNouqqUWpFaDPy9Q4Dp6Crq34KhgePl9mENMZMh/At/qnfWKfLOsgsu4ABFUwj6Q5c7cO1gT8B+OSP/ZdOZFmXT2cZ32UzvJNp30rUZjRy0P1VOENHguRR250nGD1Pd/6VHM0hLiBjuosyEM5rlekDcIbfJ3bQ2nQ/Nor9aRycPLO3Z/bZG1mtPgtI0h2vf5HWANAFbUPf9RO99Hx7TRW+2NYx1ViuXi4zAT68iG06HLDJjujzptvJf4E9CelFCW3/0RQNtgINPBViVlc1xkzW4O3uYG+E9dTCqry4PgEfytvaM7rUeuEzKB/ZeIOP4MBRkSpXAJdVJL27hJPrGwLKVN1xQgxYsDDGXoNqDy/7MC06NSHAtL+vlsy1PBzjwuHKQ7JJp0XZ1ARZNEvEl+D7KBLfdaNnRE8wY2RyGIDwBAh2npezq0j7LDSWM6jMThONf5926192OmJyAx8GAuwy2V+ihGJAPW12/mZfY1sHoendTYfJhCetsgp6+S1/trVkunMIAYhEXdpexUFUONgRJ2n4honr5pDAqSPK+ix/rmkG2LpTx4dQpXHW34qyeRaplIFIx2xABEO24XIMHejXRVZGqWdHYCDE26EcEFUE6DteLLRKMEZQ7J7XCT6eNLub14TUVX1bodXk2ZFjlX5PRQ0rUNTRCYwX7rXk+8DODdjbbwLuJsfVQ/70QKyWcH/db hlmHalob ZKB3c3AipzTpc0ruCAP4mj2SAqX45NyZr135o1WRMU28RlaNzcdu1lQyeyTHqRytZPOmch5n7DWEA2Yz/yywQHdVk9E8JmaqZ4V86qGD9oXCHVA28OSMJAqT5j8uHaAU0IRL3fA1x8405ncAioLhAOyx01+JbpwVO/tSLtEiLraRQ97Xutb6rJuwh2XTYuJHVM5jXHoLAAO2RKS8s4yC8WiW1GKM3q6+dNWULBasCr+gBkNsMkSTP1Zfkc71qvYUaY223AQWfC8aqEuhn8Dn3yVpEA3wjx5zIDAFn4VQQ0kNSj7q9VbwDVlxGDAwqBcWgepQbppHiLkIq7Ld1XDKs9GDp7cooviDTY2ge7FwkSIb2Y/C1pM/cIMHTWlqg81FurBHKFTFuY/8BjOotgEij7QQRm6C56CXjUnXOmAgufvHZsL9cu8/N+sM8yNzJutgz0tB3+KXpcDaUeDhYACMWnx0MOHuHgUYmLn9BZixEDCDBYz6zQmoKhVSBHDE0k0wkr6/Ukka++S90TtlYvqe/PaPaoUOKdnuj562+HyltweLCbXTFcBmz2/MBLzL2tmnNKB0c X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 15.06.23 09:22, Huang, Ying wrote: > Baolin Wang writes: > >> On 6/15/2023 11:22 AM, Huang, Ying wrote: >>> Hi, Mel, >>> Mel Gorman writes: >>> >>>> On Tue, Jun 13, 2023 at 04:55:04PM +0800, Baolin Wang wrote: >>>>> On some machines, the normal zone can have a large memory hole like >>>>> below memory layout, and we can see the range from 0x100000000 to >>>>> 0x1800000000 is a hole. So when isolating some migratable pages, the >>>>> scanner can meet the hole and it will take more time to skip the large >>>>> hole. From my measurement, I can see the isolation scanner will take >>>>> 80us ~ 100us to skip the large hole [0x100000000 - 0x1800000000]. >>>>> >>>>> So adding a new helper to fast search next online memory section >>>>> to skip the large hole can help to find next suitable pageblock >>>>> efficiently. With this patch, I can see the large hole scanning only >>>>> takes < 1us. >>>>> >>>>> [ 0.000000] Zone ranges: >>>>> [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff] >>>>> [ 0.000000] DMA32 empty >>>>> [ 0.000000] Normal [mem 0x0000000100000000-0x0000001fa7ffffff] >>>>> [ 0.000000] Movable zone start for each node >>>>> [ 0.000000] Early memory node ranges >>>>> [ 0.000000] node 0: [mem 0x0000000040000000-0x0000000fffffffff] >>>>> [ 0.000000] node 0: [mem 0x0000001800000000-0x0000001fa3c7ffff] >>>>> [ 0.000000] node 0: [mem 0x0000001fa3c80000-0x0000001fa3ffffff] >>>>> [ 0.000000] node 0: [mem 0x0000001fa4000000-0x0000001fa402ffff] >>>>> [ 0.000000] node 0: [mem 0x0000001fa4030000-0x0000001fa40effff] >>>>> [ 0.000000] node 0: [mem 0x0000001fa40f0000-0x0000001fa73cffff] >>>>> [ 0.000000] node 0: [mem 0x0000001fa73d0000-0x0000001fa745ffff] >>>>> [ 0.000000] node 0: [mem 0x0000001fa7460000-0x0000001fa746ffff] >>>>> [ 0.000000] node 0: [mem 0x0000001fa7470000-0x0000001fa758ffff] >>>>> [ 0.000000] node 0: [mem 0x0000001fa7590000-0x0000001fa7ffffff] >>>>> >>>>> Signed-off-by: Baolin Wang >>>> >>>> This may only be necessary for non-contiguous zones so a check for >>>> zone_contiguous could be made but I suspect the saving, if any, would be >>>> marginal. >>>> >>>> However, it's subtle that block_end_pfn can end up in an arbirary location >>>> past the end of the zone or past cc->free_pfn. As the "continue" will update >>>> cc->migrate_pfn, that might lead to errors in the future. It would be a >>>> lot safer to pass in cc->free_pfn and do two things with the value. First, >>>> there is no point scanning for a valid online section past cc->free_pfn so >>>> terminating after cc->free_pfn may save some cycles. Second, cc->migrate_pfn >>>> does not end up with an arbitrary value which is a more defensive approach >>>> to any future programming errors. >>> I have thought about this before. Originally, I had thought that we >>> were safe because cc->free_pfn should be in a online section and >>> block_end_pfn should reach cc->free_pfn before the end of zone. But >>> after checking more code and thinking about it again, I found that the >>> underlying sections may go offline under us during compaction. So that, >>> cc->free_pfn may be in a offline section or after the end of zone. So, >>> you are right, we need to consider the range of block_end_pfn. >>> But, if we thought in this way (memory online/offline at any time), >>> it >>> appears that we need to check whether the underlying section was >>> offlined. For example, is it safe to use "pfn_to_page()" in >>> "isolate_migratepages_block()"? Is it possible for the underlying >>> section to be offlined under us? >> >> It is possible. There is a previous discussion[1] about the race >> between pfn_to_online_page() and memory offline. >> >> [1] >> https://lore.kernel.org/lkml/87zgc6buoq.fsf@nvidia.com/T/#m642d91bcc726437e1848b295bc57ce249c7ca399 > > Thank you very much for sharing! That answers my questions directly! I remember another discussion (but can't find it) regarding why memory compaction can get away without pfn_to_online_page() all over the place. The use is limited to __reset_isolation_pfn(). But yes, in theory pfn_to_online_page() can race with memory offlining. -- Cheers, David / dhildenb