From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7C721C433EF for ; Wed, 12 Jan 2022 02:17:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=OFk2bWWDiRlQuLSciD4rz/ie482Y6jjhCBrO3A2+R9M=; b=u7PKgWa6WmDEgg Sspwk3FcEMIB6Re9jEanXy9cQHsGuqCcJPQTHF5qgoZo5L55OpLg54FOsdLtmEtuiKNI+NaguGeLe L/YhPaJEa++Nvsi4Y7LNrA/TRTi8gT0y6jO05DhaHVAgmchhrc4sal0RtmWLzImC1JDN+4z0GN11l j2uwgcU3p3+iVGbtxAeFIJwe89SdguegXbosWi0PrJYj6k2dofufViHdl3ouaOSaoqsZDkZtYUV0F +iTC//ntcbqAhrYKhVj9TQrhv/2/wZ7zRRKAWKQyJPWfYVmNAD0AK0NwEULHoYu5O3lhXv+UvA5PM WJZg9EtfaYK4ckfUe5yg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1n7TBb-000oty-5u; Wed, 12 Jan 2022 02:16:27 +0000 Received: from mail-il1-x12d.google.com ([2607:f8b0:4864:20::12d]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1n7TBV-000oru-8l for linux-arm-kernel@lists.infradead.org; Wed, 12 Jan 2022 02:16:24 +0000 Received: by mail-il1-x12d.google.com with SMTP id y18so962229iln.3 for ; Tue, 11 Jan 2022 18:16:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=gY3L/ohoJT+LGnrwf3WbcW3Sm7Rhragj/gK43AHxeKI=; b=R20ghVrn5UJ8eOBbViYvB+y95FXvSM8YsY4jkyuom2uoij7S7LrAgIeSEOGDRPA+oS iWVwWaRMk3A9koqyfHaYY9Y2G9eMU5QZyqz03157b2D11Afz7+CmJuQCQLYTiGBq4YkZ 8bPsYi1DASShARCETRBXl8tVBmtAFMefDw0YuY6ENC+5x4YAaMpx3hUfuwvkEe9rYIN/ P+l6g0Dk4LnR7DfjXPgHR2NNHq+1bFxFzcKyg4jxeZXEJi89+6bF/NNeVg/MKE2JY7W2 7PdCn+yfRYn9hVjBeZoOQyiQs7xq7ZZiUTE6gkwtk5+674JvmqMN+QdjMmExYWGZAQu+ 5u7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=gY3L/ohoJT+LGnrwf3WbcW3Sm7Rhragj/gK43AHxeKI=; b=WJMc7atTlYC+3gcFWFOHKhRRjyvSP3gl83JTsjA1zaUBguhCrdqy9rdLQzZSGIgUiG 1NswYzZcg3cnbkEqHcVqUiJoG5PzH6CnM/DV539SstJHqRylK8F7QMYpUMaB7/E/uItl Ny98Q2GqSM5iMa8/aYjimsfGSe+mE5hz6W3V9cCkalODFU1q8c2SStEFwsf7+UVmc6Oj KqoEnnMjYWTjj/coJToIa57dU3y7AVTNh1HquNUQHu4CuBsRieYcFeBwuwPwMnEFGuTA X9/wiI4Lto32wvVxtW1yjpX1rhvoWFS7XB3dbfaex8d8ihWIh7o7clyyuAlq5xF7tZ+0 B08w== X-Gm-Message-State: AOAM531g9ii+67JkaW7VGX1yCBbSr3IoYpUeXiaM6HX/UfqobUIA5WLE BDQq1k6TrDuPCtKqIuTE/FoJhw== X-Google-Smtp-Source: ABdhPJy1/vlLDtXylu7HegiyqkVdh1jTeQZXKMC3AuUfrN9c+AdF/siL+FtwpT5W+sz9AGxGn8ptaQ== X-Received: by 2002:a05:6e02:12e8:: with SMTP id l8mr3848743iln.111.1641953780261; Tue, 11 Jan 2022 18:16:20 -0800 (PST) Received: from google.com ([2620:15c:183:200:b6b6:70f4:b540:6383]) by smtp.gmail.com with ESMTPSA id o16sm972436ioo.36.2022.01.11.18.16.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jan 2022 18:16:19 -0800 (PST) Date: Tue, 11 Jan 2022 19:16:15 -0700 From: Yu Zhao To: "Aneesh Kumar K.V" Cc: Andrew Morton , Linus Torvalds , Andi Kleen , Catalin Marinas , Dave Hansen , Hillf Danton , Jens Axboe , Jesse Barnes , Johannes Weiner , Jonathan Corbet , Matthew Wilcox , Mel Gorman , Michael Larabel , Michal Hocko , Rik van Riel , Vlastimil Babka , Will Deacon , Ying Huang , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, page-reclaim@google.com, x86@kernel.org, Konstantin Kharlamov Subject: Re: [PATCH v6 4/9] mm: multigenerational lru: groundwork Message-ID: References: <20220104202227.2903605-1-yuzhao@google.com> <20220104202227.2903605-5-yuzhao@google.com> <878rvm661z.fsf@linux.ibm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <878rvm661z.fsf@linux.ibm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220111_181621_328244_311C001C X-CRM114-Status: GOOD ( 36.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Jan 11, 2022 at 01:46:24PM +0530, Aneesh Kumar K.V wrote: > Yu Zhao writes: > > ..... > > + > > +/* > > + * Evictable pages are divided into multiple generations. The youngest and the > > + * oldest generation numbers, max_seq and min_seq, are monotonically increasing. > > + * They form a sliding window of a variable size [MIN_NR_GENS, MAX_NR_GENS]. An > > + * offset within MAX_NR_GENS, gen, indexes the lru list of the corresponding > > + * generation. The gen counter in folio->flags stores gen+1 while a page is on > > + * lrugen->lists[]. Otherwise, it stores 0. > > + * > > + * A page is added to the youngest generation on faulting. The aging needs to > > + * check the accessed bit at least twice before handing this page over to the > > + * eviction. The first check takes care of the accessed bit set on the initial > > + * fault; the second check makes sure this page hasn't been used since then. > > + * This process, AKA second chance, requires a minimum of two generations, > > + * hence MIN_NR_GENS. And to be compatible with the active/inactive lru, these > > + * two generations are mapped to the active; the rest of generations, if they > > + * exist, are mapped to the inactive. PG_active is always cleared while a page > > + * is on lrugen->lists[] so that demotion, which happens consequently when the > > + * aging creates a new generation, needs not to worry about it. > > + */ > > Where do we clear PG_active in the code? Is this the reason we endup > with We clear PG_active when we add a page (folio) to MGLRU lists: include/linux/mm_inline.h lru_gen_add_folio() do { new_flags = old_flags = READ_ONCE(folio->flags); ... new_flags &= ~(LRU_GEN_MASK | BIT(PG_active)); ^^^^^^^^^ ... } while (cmpxchg(&folio->flags, old_flags, new_flags) != old_flags); We also set it when we isolate a page (for page migration): include/linux/mm_inline.h lru_gen_del_folio() do { new_flags = old_flags = READ_ONCE(folio->flags); ... else if (lru_gen_is_active(lruvec, gen)) new_flags |= BIT(PG_active); ^^^^^^^^^ } while (cmpxchg(&folio->flags, old_flags, new_flags) != old_flags); > > void deactivate_page(struct page *page) > { > - if (PageLRU(page) && PageActive(page) && !PageUnevictable(page)) { > + if (PageLRU(page) && !PageUnevictable(page) && (PageActive(page) || lru_gen_enabled())) { That's correct. > > +#define MIN_NR_GENS 2U > > +#define MAX_NR_GENS ((unsigned int)CONFIG_NR_LRU_GENS) > > + > > +struct lru_gen_struct { > > + /* the aging increments the youngest generation number */ > > + unsigned long max_seq; > > + /* the eviction increments the oldest generation numbers */ > > + unsigned long min_seq[ANON_AND_FILE]; > > + /* the birth time of each generation in jiffies */ > > + unsigned long timestamps[MAX_NR_GENS]; > > + /* the multigenerational lru lists */ > > + struct list_head lists[MAX_NR_GENS][ANON_AND_FILE][MAX_NR_ZONES]; > > + /* the sizes of the above lists */ > > + unsigned long nr_pages[MAX_NR_GENS][ANON_AND_FILE][MAX_NR_ZONES]; > > + /* whether the multigenerational lru is enabled */ > > + bool enabled; > > +}; > > + > > .... > > > static void __meminit zone_init_internals(struct zone *zone, enum zone_type idx, int nid, > > diff --git a/mm/swap.c b/mm/swap.c > > index e8c9dc6d0377..d7dde3b7d4b5 100644 > > --- a/mm/swap.c > > +++ b/mm/swap.c > > @@ -462,6 +462,11 @@ void folio_add_lru(struct folio *folio) > > VM_BUG_ON_FOLIO(folio_test_active(folio) && folio_test_unevictable(folio), folio); > > VM_BUG_ON_FOLIO(folio_test_lru(folio), folio); > > > > + /* see the comment in lru_gen_add_folio() */ > > + if (lru_gen_enabled() && !folio_test_unevictable(folio) && > > + task_in_lru_fault() && !(current->flags & PF_MEMALLOC)) > > + folio_set_active(folio); > > + > > > Can you explain this better? What is the significance of marking the > folio active here. Do we need to differentiate parallel page faults (across > different vmas) w.r.t task_in_lru_fault()? All pages faulted in need to be added to the youngest generation. But without PG_active, lru_gen_add_folio() doesn't know whether a page was faulted in, or something else, e.g., page cache readahead. This is because pages aren't immediately sent to lru_gen_add_folio(). They are batched by lru_pvecs: /** * folio_add_lru - Add a folio to an LRU list. * @folio: The folio to be added to the LRU. * * Queue the folio for addition to the LRU. The decision on whether * to add the page to the [in]active [file|anon] list is deferred until the * pagevec is drained. This gives a chance for the caller of folio_add_lru() * have the folio added to the active list using folio_mark_accessed(). */ void folio_add_lru(struct folio *folio) { struct pagevec *pvec; VM_BUG_ON_FOLIO(folio_test_active(folio) && folio_test_unevictable(folio), folio); VM_BUG_ON_FOLIO(folio_test_lru(folio), folio); /* see the comment in lru_gen_add_folio() */ if (lru_gen_enabled() && !folio_test_unevictable(folio) && lru_gen_in_pgfault() && !(current->flags & PF_MEMALLOC)) folio_set_active(folio); folio_get(folio); local_lock(&lru_pvecs.lock); pvec = this_cpu_ptr(&lru_pvecs.lru_add); if (pagevec_add_and_need_flush(pvec, &folio->page)) __pagevec_lru_add(pvec); local_unlock(&lru_pvecs.lock); } _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel