From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71115C4332F for ; Wed, 16 Nov 2022 22:59:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B9BE36B0080; Wed, 16 Nov 2022 17:59:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B4C0C6B0081; Wed, 16 Nov 2022 17:59:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3B146B0082; Wed, 16 Nov 2022 17:59:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 95F826B0080 for ; Wed, 16 Nov 2022 17:59:58 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 6CCDD8035A for ; Wed, 16 Nov 2022 22:59:58 +0000 (UTC) X-FDA: 80140824876.01.904B98A Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf28.hostedemail.com (Postfix) with ESMTP id EF9F2C0009 for ; Wed, 16 Nov 2022 22:59:56 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id E9FE8B81EE5; Wed, 16 Nov 2022 22:59:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 839E0C433C1; Wed, 16 Nov 2022 22:59:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1668639593; bh=smVJkag9FFEN64aZOZ4+12+IwhkB1uvxIRN7tAlEaJk=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=MzZ/Ad5mGZUeExNMinKu/cghmM7xFN7eQmJ8ZXBJQF0ozg8WlQjzemFfaa7kHxnSE NrV4GNXH+R6S1Ap3kFJuQo7VLfa1bDYT6Xpwm6BcOSmi4dnAm6uYzMMCEx3YHHJoyP QaL6ous9AN3uUACePfx0fwOp4Y34UVyKRLhKP6h8= Date: Wed, 16 Nov 2022 14:59:52 -0800 From: Andrew Morton To: Yu Zhao Cc: linux-mm@kvack.org Subject: Re: [PATCH 1/2] mm: multi-gen LRU: retry folios written back while isolated Message-Id: <20221116145952.3a88ac84ea0b6c5dba1056df@linux-foundation.org> In-Reply-To: <20221116013808.3995280-1-yuzhao@google.com> References: <20221116013808.3995280-1-yuzhao@google.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1668639597; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aHB87DBq9V5BLFc5b5x7JB1a/DI/OFyzUIdjiFiwVWY=; b=SRDbfuJensVzG7Sw6zcOYMZkgovAfJkrbh05AvhFDe4woigMfDJP7070uN8zCmyEV7WkT4 djBGgpOEMBuN1sUXwHH3IX2RhIZ9p09Oot3mm8eRT4ZbS/RoOdvEnPJnLw1bHF8sruK0wE raxJhHcNxTLxhcw+QeiWTECVAKfj73w= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="MzZ/Ad5m"; spf=pass (imf28.hostedemail.com: domain of akpm@linux-foundation.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1668639597; a=rsa-sha256; cv=none; b=ofGUtfiQFW8lMM+eBOAS/S9kjwcrTRzOJ1nLKuBTDcSNiZkZDNIVohzqV4J9Hg68jXeteg y8w5+Fnw3dtBNrkK3xjf7WsPuds6/Cjh0tXX2dnVJj9cumQd2ffmkN3ilC2IP8mGpXckHE t9JkDg5Tt2uvHFEDf6hCm57SdJv7dZk= X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: EF9F2C0009 Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="MzZ/Ad5m"; spf=pass (imf28.hostedemail.com: domain of akpm@linux-foundation.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: nk4i1f8qy1o88no4e8iigyhf1m5mkijs X-HE-Tag: 1668639596-741101 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, 15 Nov 2022 18:38:07 -0700 Yu Zhao wrote: > The page reclaim isolates a batch of folios from the tail of one of > the LRU lists and works on those folios one by one. For a suitable > swap-backed folio, if the swap device is async, it queues that folio > for writeback. After the page reclaim finishes an entire batch, it > puts back the folios it queued for writeback to the head of the > original LRU list. > > In the meantime, the page writeback flushes the queued folios also by > batches. Its batching logic is independent from that of the page > reclaim. For each of the folios it writes back, the page writeback > calls folio_rotate_reclaimable() which tries to rotate a folio to the > tail. > > folio_rotate_reclaimable() only works for a folio after the page > reclaim has put it back. If an async swap device is fast enough, the > page writeback can finish with that folio while the page reclaim is > still working on the rest of the batch containing it. In this case, > that folio will remain at the head and the page reclaim will not retry > it before reaching there. > > This patch adds a retry to evict_folios(). After evict_folios() has > finished an entire batch and before it puts back folios it cannot free > immediately, it retries those that may have missed the rotation. > > Before this patch, ~60% of folios swapped to an Intel Optane missed > folio_rotate_reclaimable(). After this patch, ~99% of missed folios > were reclaimed upon retry. > > This problem affects relatively slow async swap devices like Samsung > 980 Pro much less and does not affect sync swap devices like zram or > zswap at all. As I understand it, this approach has an implicit assumption that by the time evict_folios() has completed its first pass, write IOs will have completed and the resulting folios are available for processing on evict_folios()'s second pass, yes? If so, it all kinda works by luck of timing. If the swap device is even slower, the number of folios which are unavailable on the second pass will increase? Can we make this more deterministic? For example change evict_folios() to recognize this situation and to then do folio_rotate_reclaimable()'s work for it? Or if that isn't practical, do something else? (Is folio_rotate_reclaimable() actually useful? That concept must be 20 years old. What breaks if we just delete it and leave the pages wherever they are?)