From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17496C433B4 for ; Wed, 14 Apr 2021 19:05:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A1B56610FC for ; Wed, 14 Apr 2021 19:05:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A1B56610FC Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2EAB46B0071; Wed, 14 Apr 2021 15:05:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 29A896B0072; Wed, 14 Apr 2021 15:05:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 115556B0073; Wed, 14 Apr 2021 15:05:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0058.hostedemail.com [216.40.44.58]) by kanga.kvack.org (Postfix) with ESMTP id ED1306B0071 for ; Wed, 14 Apr 2021 15:05:03 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A79FC82499B9 for ; Wed, 14 Apr 2021 19:05:03 +0000 (UTC) X-FDA: 78031900086.26.D30D2C0 Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by imf22.hostedemail.com (Postfix) with ESMTP id 7E795C0007CB for ; Wed, 14 Apr 2021 19:04:59 +0000 (UTC) Received: by mail-wm1-f53.google.com with SMTP id t14-20020a05600c198eb029012eeb3edfaeso898820wmq.2 for ; Wed, 14 Apr 2021 12:05:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=jYrJlRv7hMOslZniAZd/Q50Y9bWCr5RlPObcMlcuTkQ=; b=hzHoiVkWIZEHVJk9H2nbZSqd2yL7vB4H89ftPNLX5CWSYzwiF/RYeI69rFC5JdnwLA UHsS7ckUkdL2XPeDeclyaivwKYvS9FvTlvIlhp5J1LT3v/iBhwEzc/ISgcNvQ9OjW16l lPWSF2KT9PTYioS8NKV/OqP6BSd6PmwZxxQLwaFr2v+g0kmybzjX9DC2McZ5dzV/2r4E zdtLhlIiPAOJV9G01jDnkciAXGfgoEpSJT07eBaNecUCr+f8UV2BY9zGFBeStRIA79UO RUp8cQfGP2+bJ3l3uIUenDrHs7o4+D7D3bBUP9MUly2cOMOD+EIZS/GBCBmmr+kEAiup UbKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=jYrJlRv7hMOslZniAZd/Q50Y9bWCr5RlPObcMlcuTkQ=; b=ph94hgspDq5/h+EXGLt6T9W39ekMiFVWuJnUEZ4y0QBkzrGEb3OKgIc955ELLhN/ra odL2APyu59qi31Zk+s7fdbT4gcKMCE17mLXaHVBsHZ0SjSJxwyQo7NtUjBPS7qBeZUhv xjtarcl0Vdu5lcI7Fn9RZDSn6TWiMVpLlw6S3cIvAc6EE7RANxRdkJv9HTG+zCbqr9lU 5uee6nFYwlSy+TTQi65W5J7MYx96MHlU2/aPEuYO7E3u0n/QX7VCHgyuU3jFbO0Unq7w SW+wk1BFnITdr5fTai84XrKELZqObrYYpufwSVN/hwR3wL/yuL9No7L8Zpjaz4F5bNNH RcPw== X-Gm-Message-State: AOAM532mgOeGXB5pUtnnmhMdcrpBQBlSR49QTDztS0AgoQ1lakGKyZHL 4vW0MCE+c8XJiDAvHkBgwkW/RJKwUzwEU2r80uVTLQ== X-Google-Smtp-Source: ABdhPJx8wYHD6o/5YkOM5SOkidUeX3KOuOvNsPwfC3FA9lg5HylEC513k8hW6k28TsAef/FayqryFtlQqZcBkHp8JWY= X-Received: by 2002:a7b:ce8a:: with SMTP id q10mr4414305wmj.101.1618427101999; Wed, 14 Apr 2021 12:05:01 -0700 (PDT) MIME-Version: 1.0 References: <20210413075155.32652-1-sjpark@amazon.de> <3ddd4f8a-8e51-662b-df11-a63a0e75b2bc@kernel.dk> <20210413231436.GF63242@dread.disaster.area> <20210414155130.GU3762101@tassilo.jf.intel.com> In-Reply-To: <20210414155130.GU3762101@tassilo.jf.intel.com> From: Yu Zhao Date: Wed, 14 Apr 2021 13:04:50 -0600 Message-ID: Subject: Re: [PATCH v2 00/16] Multigenerational LRU Framework To: Andi Kleen Cc: Rik van Riel , Dave Chinner , Jens Axboe , SeongJae Park , Linux-MM , Andrew Morton , Benjamin Manes , Dave Hansen , Hillf Danton , Johannes Weiner , Jonathan Corbet , Joonsoo Kim , Matthew Wilcox , Mel Gorman , Miaohe Lin , Michael Larabel , Michal Hocko , Michel Lespinasse , Roman Gushchin , Rong Chen , SeongJae Park , Tim Chen , Vlastimil Babka , Yang Shi , Ying Huang , Zi Yan , linux-kernel , lkp@lists.01.org, Kernel Page Reclaim v2 Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: w4nzktcnk1d6cmjfj1daqipb7qqir3s5 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7E795C0007CB Received-SPF: none (google.com>: No applicable sender policy available) receiver=imf22; identity=mailfrom; envelope-from=""; helo=mail-wm1-f53.google.com; client-ip=209.85.128.53 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618427099-892939 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Apr 14, 2021 at 9:51 AM Andi Kleen wrote: > > > 2) It will not scan PTE tables under non-leaf PMD entries that do not > > have the accessed bit set, when > > CONFIG_HAVE_ARCH_PARENT_PMD_YOUNG=y. > > This assumes that workloads have reasonable locality. Could there > be a worst case where only one or two pages in each PTE are used, > so this PTE skipping trick doesn't work? Hi Andi, Yes, it does make that assumption. And yes, there could. AFAIK, only x86 supports this. I wrote a crude test to verify this, and it maps exactly one page within each PTE table. And I found page table scanning didn't underperform the rmap: https://lore.kernel.org/linux-mm/YHFuL%2FDdtiml4biw@google.com/#t The reason (sorry for repeating this) is page table scanning is conditional: bool should_skip_mm() { ... /* leave the legwork to the rmap if mapped pages are too sparse */ if (RSS < mm_pgtables_bytes(mm) / PAGE_SIZE) return true; .... } We fall back to the rmap when it's obviously not smart to do so. There is still a lot of room for improvement in this function though, i.e., it should be per VMA and NUMA aware. Note that page table scanning doesn't replace the existing rmap scan. It's complementary, and it happens when there is a good chance that most of the pages on a system under pressure have been referenced. IOW, scanning them one by one with the rmap would cost more than scanning them all at once via page tables. Sounds reasonable? Thanks.