From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26110C48BC3 for ; Sun, 18 Feb 2024 02:52:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 94ED66B007D; Sat, 17 Feb 2024 21:52:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8D9646B0085; Sat, 17 Feb 2024 21:52:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7788A6B0088; Sat, 17 Feb 2024 21:52:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 602456B007D for ; Sat, 17 Feb 2024 21:52:25 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 2A1A1160358 for ; Sun, 18 Feb 2024 02:52:25 +0000 (UTC) X-FDA: 81803401050.10.1DDFC90 Received: from out-170.mta1.migadu.com (out-170.mta1.migadu.com [95.215.58.170]) by imf24.hostedemail.com (Postfix) with ESMTP id 6CA3F180006 for ; Sun, 18 Feb 2024 02:52:22 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=WQOq4yJ4; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf24.hostedemail.com: domain of chengming.zhou@linux.dev designates 95.215.58.170 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708224742; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wEo9y40FrQbW3MYg4fpLWpV9+DPnE1npkypG6mMinwo=; b=k/vyWonWAFV8VVcgzi2cmbs0dF/eZ5FWSn7Q7tOq+RVQwLIQ7NL1oIeytm9bevWvWHLPDi c7wOePJhJeTpaNGVW4Tu5yRQ70TYnMAYWgkYIKF/9hpZ3nfPQwyK5NFgPtD2s6lZ79h30V 7mLec+wc/nd589acDXaCp+UwB/Al6Lk= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=WQOq4yJ4; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf24.hostedemail.com: domain of chengming.zhou@linux.dev designates 95.215.58.170 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708224742; a=rsa-sha256; cv=none; b=Ar++FX+/7FljayBmiKjSwsi8J5Unvlf26klLRyP+b+clCJoOD0+95g7F5W2+lcm0yc9gnk U30VG4oihjusMKST9D0FmjsMAwjPqHgvIFEm20TnpWr+6HVSV/0nIku7Xpuipj+LA7JFEb Q4niYaV/ewwUoA12x2LNi5jfh0IKwTo= Message-ID: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1708224740; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wEo9y40FrQbW3MYg4fpLWpV9+DPnE1npkypG6mMinwo=; b=WQOq4yJ4wLqqR4VvI1glPgtT8lO4FrpVJUhR3gAAHaPUSMb/qy4Vc1/laptw90Lb1zRsu8 ANWM/0a+iN4zT8R1k8ZBFyAjCthGc5dM5Foy3hvalbAXe11N6+V2tK4EMiM5Trgf+yzicP EiC/tT3vZmtqjIRgnS5n/G8Ixz5dp9g= Date: Sun, 18 Feb 2024 10:52:01 +0800 MIME-Version: 1.0 Subject: Re: [PATCH RFC 1/1] mm/swap: queue reclaimable folio to local rotate batch when !folio_test_lru() Content-Language: en-US To: Yu Zhao Cc: willy@infradead.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Chengming Zhou References: <20240209115950.3885183-1-chengming.zhou@linux.dev> <20240209115950.3885183-2-chengming.zhou@linux.dev> <8123c4be-d696-4e9e-884f-aa12f6099ddb@linux.dev> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Chengming Zhou In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: 6CA3F180006 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: zag9o4hbd46ebo4pkjofjmkzt68um8hi X-HE-Tag: 1708224742-332875 X-HE-Meta: U2FsdGVkX1/VsIG8A0akDdsT1EjJg3linxlAQSBWk63IIlB4IPAbL2PR9o5Gi5LSz//pk+bDRfrj7+O0TvRUz9Ag6+L4152Smd81HhvVA/rVmQEVliX47sK25wZTgyXLz4pdoV/OBJIJzpPthtSpwiMAsgVZFOIaSnVmyeE2HMXxfhGYM9kXHrPsdrEqpt/4PwKsjwigblzFt29G/bt3T0NEQbkA3dMiZJK1lF6mmeBrMCZgRExGWMxJXpt+XNRayIv8Se2GC7KYzcjzMyEBZYuFhqulB8uQnm2mX/JP6L2stl3lKzMsGEJ0E0XSWQQGqlqHQ6IwVwKpDW/Gwm6Y/6DXYjVv2Xp46bDC8GAEib2W05yH+vxWA30x/j0waXW8K9csGwR2z/4CdsFT1KKWQhAz1eft9jpx6hjYwbgMBaV3MH6uN7I9UsG3MIoLUSMTd4jVDDkxNSVTiB0pFQaiNDp18tgVcmtoa1UkndzBt7jiRMF7NvdWJpQkGHAIc79Jxh6lR9XhLjrSUDq1SoAN0wX+t4KKyUDi+VU5Vi2TI4SQEb0L+jDgA4xFnZmnqY8jBOeoxCCpCRc+VsT8ePbTlh+lmJsZXNb+oeMxM3vCmSff2SXyLFHvDOemiKErmfgz9/j67wCzHuskis+VIGooxs1cyuehr6MRKIByVaWP0hSu9h3sqNsDCZhoNO58fAnG6rbhc2dt5TeV7RGLmA+JKwEUBYM26BRK0dN0pPJ5BFB/scrrBcHF+6rZjXJvNQMmeEBrHT/233A17VBX6gMMqkylYAcHe6z6Lp8NgK89v3+aQYO4hvk34AVt6+Tfp3g7UXCNmhsSnaEXJvz4ij/DRQUPxCg3ct/+PVOt7G0ofkGCxOWEmF/jPOynPf/9ZoeCJNuf2gV375Q= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/2/15 15:06, Yu Zhao wrote: > On Wed, Feb 14, 2024 at 4:18 AM Chengming Zhou wrote: >> >> On 2024/2/14 15:13, Yu Zhao wrote: >>> On Fri, Feb 9, 2024 at 6:00 AM wrote: >>>> >>>> From: Chengming Zhou >>>> >>>> All LRU move interfaces have a problem that it has no effect if the >>>> folio is isolated from LRU (in cpu batch or isolated by shrinker). >>>> Since it can't move/change folio LRU status when it's isolated, mostly >>>> just clear the folio flag and do nothing in this case. >>>> >>>> In our case, a written back and reclaimable folio won't be rotated to >>>> the tail of inactive list, since it's still in cpu lru_add batch. It >>>> may cause the delayed reclaim of this folio and evict other folios. >>>> >>>> This patch changes to queue the reclaimable folio to cpu rotate batch >>>> even when !folio_test_lru(), hoping it will likely be handled after >>>> the lru_add batch which will put folio on the LRU list first, so >>>> will be rotated to the tail successfully when handle rotate batch. >>>> >>>> Signed-off-by: Chengming Zhou >>> >>> I don't think the analysis is correct. IIRC, writeback from non >>> reclaim paths doesn't require isolation and the reclaim path doesn't >>> use struct folio_batch lru_add. >> >> Ah, my bad, I forgot to mention the important context in the message: >> >> This is not from the normal reclaim context, it's from zswap writeback >> reclaim context, which will first set PG_reclaim flag, then submit the >> async writeback io. >> >> If the writeback io complete fast enough, folio_rotate_reclaimable() >> will be called before that folio put on LRU list (it still in the local >> lru_add batch, so it's somewhat like isolated too) >> >>> >>> Did you see any performance improvements with this patch? In general, >>> this kind of patches should have performance numbers to show it really >>> helps (not just in theory). >> >> Right, there are some improvements, the numbers are put in cover letter. >> But this solution is not good enough, just RFC for discussion. :) >> >> mm-unstable-hot zswap-lru-reclaim >> real 63.34 62.72 >> user 1063.20 1060.30 >> sys 272.04 256.14 >> workingset_refault_anon 2103297.00 1788155.80 >> workingset_refault_file 28638.20 39249.40 >> workingset_activate_anon 746134.00 695435.40 >> workingset_activate_file 4344.60 4255.80 >> workingset_restore_anon 653163.80 605315.60 >> workingset_restore_file 1079.00 883.00 >> workingset_nodereclaim 0.00 0.00 >> pgscan 12971305.60 12730331.20 >> pgscan_kswapd 0.00 0.00 >> pgscan_direct 12971305.60 12730331.20 >> pgscan_khugepaged 0.00 0.00 >> >>> >>> My guess is that you are hitting this problem [1]. >>> >>> [1] https://lore.kernel.org/linux-mm/20221116013808.3995280-1-yuzhao@google.com/ >> >> Right, I just see it, it's the same problem. The only difference is that >> in your case the folio is isolated by shrinker, in my case, the folio is >> in cpu lru_add batch. Anyway, the result is the same, that folio can't be >> rotated successfully when writeback complete. > > In that case, a better solution would be to make lru_add add > (_reclaim() && !_dirty() && !_writeback()) folios at the tail. > (_rotate() needs to leave _reclaim() set if it fails to rotate.) Right, this is a solution. But PG_readahead is alias of PG_reclaim, I'm afraid this would rotate readahead folio to the inactive tail.