From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 184EAC433FE for ; Fri, 18 Nov 2022 21:25:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 494686B0071; Fri, 18 Nov 2022 16:25:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 443F36B0072; Fri, 18 Nov 2022 16:25:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 30BCB8E0001; Fri, 18 Nov 2022 16:25:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 2330A6B0071 for ; Fri, 18 Nov 2022 16:25:33 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id F324740735 for ; Fri, 18 Nov 2022 21:25:32 +0000 (UTC) X-FDA: 80147844504.17.59D3954 Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) by imf27.hostedemail.com (Postfix) with ESMTP id 9C36E4000B for ; Fri, 18 Nov 2022 21:25:32 +0000 (UTC) Received: by mail-pg1-f176.google.com with SMTP id r18so6012696pgr.12 for ; Fri, 18 Nov 2022 13:25:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:from:to:cc:subject:date:message-id :reply-to; bh=DLh5doROug0AR98N2OygZkQE4UplxgMWleU0EUmybvQ=; b=mbh0qHvp3pARDEPRMkX/KZGLJXzXuJMiVjDfKzzz6U5dgE9ruPvTYRW+IfH0yZkbTs Kq4BYwUN/qhDP0oeBEnaVV9D67Y3mT0CSHHRucW7slzv18IXDg5bgZzBP7SAHdjmf9G7 MEteXl+DBL8EwvGU3SDnzRhFBTl/cnVfTEgZH+ZCFTUDsu2zbHHPV71ls0n3kjB7xGE6 6hWQIZHT9/uiFptNsjXShoY3GCrkPaeKW3Wi4kEkpsY14vp5ZOKGKwbSXh2fBEEBBFmj Oom6hRlDEoRI9LBCnOxZN5+Nmsju8/pAy7OOugpc6AroAEu2aSmq0BjfPfFn7oVKLdyO yaCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:sender:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DLh5doROug0AR98N2OygZkQE4UplxgMWleU0EUmybvQ=; b=xLC3bcEsBNk+h7lgnHQsdu4DWrRh/3pajLHUB46PJpptb2N9RFeLMic5cshI+sFlDj 9flP33/SPyLax0SjgiVTNdcjtq8GPKnEcp/bcl4bk9W1DkcwdowPtJBH1IZal74cDliJ jM2djkva26h8XXHbAJr0HL2+pSbZmg5GHdBiqjXdm+SpZU5jaMP4FmvD4D9YD3kVne92 /Cy3nRbS63/8C1VUCN6/d3AAUNW2yuALKSdHmMYA5biA4psT656brTgsYy+qic/3LgAV f8eXsCrOzXJ7U8aJImoEhizwRMMGrVwYEfEQ0iH5nzOlMoswybADpOlXyO3X6LdlexmU CmGw== X-Gm-Message-State: ANoB5pnMfVnS3jKlrt1hhPr/nN0SAzpLaBSdlFHDvARXTi4KYFh6TfO3 avCpEY7jlwlH/Llx33xOfaVgZ/TNrYE= X-Google-Smtp-Source: AA0mqf4MpnPepbOhHq13lJhtQCbHblz01zD8zZhrMhTKZ528TsDzeu7mWuqUR9zHJ56J24U7ldFatg== X-Received: by 2002:a63:d909:0:b0:440:5517:5d8a with SMTP id r9-20020a63d909000000b0044055175d8amr8234240pgg.200.1668806731535; Fri, 18 Nov 2022 13:25:31 -0800 (PST) Received: from google.com ([2620:15c:211:201:bba9:9f92:b2cc:16a4]) by smtp.gmail.com with ESMTPSA id z12-20020a170903018c00b00186616b8fbasm4296178plg.10.2022.11.18.13.25.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 13:25:31 -0800 (PST) Date: Fri, 18 Nov 2022 13:25:29 -0800 From: Minchan Kim To: Yu Zhao Cc: Andrew Morton , linux-mm@kvack.org Subject: Re: [PATCH 1/2] mm: multi-gen LRU: retry folios written back while isolated Message-ID: References: <20221116013808.3995280-1-yuzhao@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1668806732; a=rsa-sha256; cv=none; b=E/Z2Tioa8CSlQnACUAeo/VvP4FtYilXSw/xx0ilOzMBXsa9z5EOzsioMfq35Ab1bvDd0bC mZMYfNk2yCXwXp8i+bslXop43IgJKfrdxAMHgn8j/NroTsuPMGW9HhBhHKMzZt25z7aLGU ++FxQq8bWusYjBmxSAwHc1IiEBlj+jM= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=mbh0qHvp; spf=pass (imf27.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.215.176 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1668806732; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DLh5doROug0AR98N2OygZkQE4UplxgMWleU0EUmybvQ=; b=CC6B7pr7l5e16pRHYT86A+Hpq0SpjKID2wYTR24cTINp8Um2QEAofYG4BeG9b/shP/Gq5O eAuy3INtUoRIVz/pkbNuSlTGE4HzKftv0mbHVeB+aW1re/VdHLLOEdsUegr+/l9WRF3UNt oLC+Czi0GpWRb25lCd3F+841jIYLsNw= X-Stat-Signature: oaqwzfnra1o5d5sm3y88mxa913tr7xr6 X-Rspamd-Queue-Id: 9C36E4000B X-Rspamd-Server: rspam01 X-Rspam-User: Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=mbh0qHvp; spf=pass (imf27.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.215.176 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) X-HE-Tag: 1668806732-855897 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Nov 17, 2022 at 06:40:05PM -0700, Yu Zhao wrote: > On Thu, Nov 17, 2022 at 6:26 PM Minchan Kim wrote: > > > > On Thu, Nov 17, 2022 at 03:22:42PM -0700, Yu Zhao wrote: > > > On Thu, Nov 17, 2022 at 12:47 AM Minchan Kim wrote: > > > > > > > > On Tue, Nov 15, 2022 at 06:38:07PM -0700, Yu Zhao wrote: > > > > > The page reclaim isolates a batch of folios from the tail of one of > > > > > the LRU lists and works on those folios one by one. For a suitable > > > > > swap-backed folio, if the swap device is async, it queues that folio > > > > > for writeback. After the page reclaim finishes an entire batch, it > > > > > puts back the folios it queued for writeback to the head of the > > > > > original LRU list. > > > > > > > > > > In the meantime, the page writeback flushes the queued folios also by > > > > > batches. Its batching logic is independent from that of the page > > > > > reclaim. For each of the folios it writes back, the page writeback > > > > > calls folio_rotate_reclaimable() which tries to rotate a folio to the > > > > > tail. > > > > > > > > > > folio_rotate_reclaimable() only works for a folio after the page > > > > > reclaim has put it back. If an async swap device is fast enough, the > > > > > page writeback can finish with that folio while the page reclaim is > > > > > still working on the rest of the batch containing it. In this case, > > > > > that folio will remain at the head and the page reclaim will not retry > > > > > it before reaching there. > > > > > > > > > > This patch adds a retry to evict_folios(). After evict_folios() has > > > > > finished an entire batch and before it puts back folios it cannot free > > > > > immediately, it retries those that may have missed the rotation. > > > > > > > > Can we make something like this? > > > > > > This works for both the active/inactive LRU and MGLRU. > > > > I hope we fix both altogether. > > > > > > > > But it's not my prefered way because of these two subtle differences: > > > 1. Folios eligible for retry take an unnecessary round trip below -- > > > they are first added to the LRU list and then removed from there for > > > retry. For high speed swap devices, the LRU lock contention is already > > > quite high (>10% in CPU profile under heavy memory pressure). So I'm > > > hoping we can avoid this round trip. > > > 2. The number of retries of a folio on folio_wb_list is unlimited, > > > whereas this patch limits the retry to one. So in theory, we can spin > > > on a bunch of folios that keep failing. > > > > > > The most ideal solution would be to have the one-off retry logic in > > > shrink_folio_list(). But right now, that function is very cluttered. I > > > plan to refactor it (low priority at the moment), and probably after > > > that, we can add a generic retry for both the active/inactive LRU and > > > MGLRU. I'll raise its priority if you strongly prefer this. Please > > > feel free to let me know. > > > > Well, my preference for *ideal solution* is writeback completion drops > > page immediately without LRU rotating. IIRC, concern was softirq latency > > and locking relevant in the context at that time when I tried it. > > Are we good for now or are there other ideas we want to try while we are at it? > good for now with what solution you are thinking? The retry logic you suggested? I personally don't like the solution relies on the timing. If you are concerning about unnecessary round trip, it shouldn't happen frequency since your assumption is swap device is so fast so second loop would see their wb done? Anyway, I am strongly push my preference. Feel free to go with way you want if the solution can fix both LRU schemes.