From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46130C5472F for ; Tue, 27 Aug 2024 17:56:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9CBC16B007B; Tue, 27 Aug 2024 13:56:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 97B4B6B0083; Tue, 27 Aug 2024 13:56:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81D206B0085; Tue, 27 Aug 2024 13:56:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 6507D6B007B for ; Tue, 27 Aug 2024 13:56:31 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id ED023A16CB for ; Tue, 27 Aug 2024 17:56:30 +0000 (UTC) X-FDA: 82498780140.06.8C4E3A0 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf02.hostedemail.com (Postfix) with ESMTP id D881E80011 for ; Tue, 27 Aug 2024 17:56:27 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=WdzUt9R2; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=quarantine); spf=pass (imf02.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724781323; a=rsa-sha256; cv=none; b=KTpvWI/jhEl/u99g+Qik6m47I/A38hWHqXe5okB2x7pkW6NjhPgFvQrlpCaFhfD8WaTAOd D3maHWdYfz0b/78WsYeQokUU8KqCSCQEw25eenBU7hNVW4oe4pEsLfyQubgDyYqXApFcSA EQF3WEiuO/4RO7WlhWUxK4KxzoRTh/w= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=WdzUt9R2; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=kernel.org (policy=quarantine); spf=pass (imf02.hostedemail.com: domain of minchan.kim@gmail.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=minchan.kim@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724781323; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=N7aMAk3bc0KCr5nUGMiMqeFQkv5fTbr62wKgmksEh4M=; b=jdmzHnfMfF8RLTmz212++409SXS4fymLQxDX7Syw0KmhrEbWWev9k1DHedZse5xrlcUifq AwUsjNQKwM+kmbNdBtnO55wHuREpQJUxauPq34lfxJjz9V3Mg0fMK1pVHth6Wd+Nv2cR9F ZX6OsA9bKzL2BEgS238vykhwXj8zXa0= Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-204f52fe74dso438885ad.1 for ; Tue, 27 Aug 2024 10:56:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724781387; x=1725386187; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :from:to:cc:subject:date:message-id:reply-to; bh=N7aMAk3bc0KCr5nUGMiMqeFQkv5fTbr62wKgmksEh4M=; b=WdzUt9R2rsSd0LxpzaQISPE2xWVyYQyFjbzOGNOcmtY4rc7r5X6MWX8VXAA0PDpJXU vG5Y/aUBT0ItmhVYvDaM6l1uR6UiE8B3/epCgzEUhR+JLBktCudTblzfOS/fh4AvXrHP 9oEUXSCPPtRurw3BHxbIOHAFlCsdXQ+JtFimqQ2lbSH8WQ7EKVtU82j2axBPYUSMBBkg sgRqoya/zVS+ibGYT4bF+/pdJQ+J6xlIzBzixwrdFoUNRUVzPtZl8EMb7vwCeJBRHP9t qQHBJs8/ip+f4roV/J97Cre/UbbVS6cJTdJrTH8M9GTO/cN0uzJgepoLL90KC2qodFof Rqpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724781387; x=1725386187; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=N7aMAk3bc0KCr5nUGMiMqeFQkv5fTbr62wKgmksEh4M=; b=asuYHUh9DDJhvwKARZY4/rCxl/KG7/O7GfMR3XMrM3fitSThcUjBptU2r7DNl+VXW9 6nFz9Cey/CVJPTFcwX8YymPn4potYDPN3KfTmoS8EOl5T/Lc88pyKna5ybfaFYkvEVJc sCbsS9/QWdbZ3MS0r8qHsOIraGjR6efuqznV5FRIyOOGp1yMg1VJHnZML6LbQxX4Gf0L 3xtvvaJaEpjG+43wRvNC3CaaK93MmarvHcf9wjuMLpZkqvu2j8xoyIu7VIpuIQ1t+OIC A1OjLkTRWHXSTZobV8ZqBNnVov9DIyTvBPnEzle1RvZxUQTf4T7aFQhQumxCUKUZ4W6c BROQ== X-Forwarded-Encrypted: i=1; AJvYcCXpXCH++tnvLhh64pm6gu5u7N0nbHqgrfow2s0h68stzPfUmL2E2yKVMoF3FC5Foy3o026wvAbdSw==@kvack.org X-Gm-Message-State: AOJu0Yzm/Fqp8QD8z390pAbqfMpwGX5gdhW5GSr4WWVIjwAGIoO1zpY1 RQFlpFuW/nwkaakCANIXv8Yp2IVuvchvt2vY68UQZyohUGWJn2rn X-Google-Smtp-Source: AGHT+IH28/cO0g/kV8NLGXvd0MhT2wq0VthOuOw/elA6M8+G3tRDAS1523ccKn1pzYOdx5Om5I5hvQ== X-Received: by 2002:a17:902:e552:b0:202:1b1e:c1d1 with SMTP id d9443c01a7336-204df4e034amr39741765ad.48.1724781386528; Tue, 27 Aug 2024 10:56:26 -0700 (PDT) Received: from google.com (51.193.125.34.bc.googleusercontent.com. [34.125.193.51]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-203855df501sm85892445ad.132.2024.08.27.10.56.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 27 Aug 2024 10:56:24 -0700 (PDT) Date: Tue, 27 Aug 2024 17:56:22 +0000 From: Minchan Kim To: gaoxu Cc: Lokesh Gidra , Barry Song <21cnbao@gmail.com>, Suren Baghdasaryan , Nicolas Geoffray , Michal Hocko , Andrew Morton , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Shaohua Li , yipengxiang , fengbaopeng , Kalesh Singh Subject: Re: =?utf-8?B?5Zue5aSN?= =?utf-8?Q?=3A?= [PATCH v2] mm: add lazyfree folio to lru tail Message-ID: References: <1757d01334ee4391beba1ea3dcdfed7c@honor.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1757d01334ee4391beba1ea3dcdfed7c@honor.com> X-Rspamd-Pre-Result: action=add header; module=dmarc; Action set by DMARC X-Rspamd-Queue-Id: D881E80011 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 3z89ynxnr37myc56e7uapshswsftfjxk X-Rspam: Yes X-HE-Tag: 1724781387-210974 X-HE-Meta: U2FsdGVkX18Bq7jqfRoDflfeLr7+a3KE42P4GWVuYRPwhfZjazKx05x5XM6+4C4VEKRxcKKsF/1R60x4wbCZptOMz/79A0AjS2P1fa5atYkfUdXfKeO+baS/VgJN2O6ReEz2YIC5/QXG4zFQs/zpQOtQpnSHf6h8XAV/yxHr0TQg8Z4YJEd36i3E5oreq1/7hNoWsHQ9cjVMeGnZaj0f/qd0JLxa3Qz2z1t4P8YLACCcfS6wFO0iN17ov1xTostxHMN5jPwqFTM/9En+DGdtsPwH2TjdhYs6IztXV7LRCOuhRukIlDHlfkMN/AA34pEXLpI7uLs6uY9o/5ohjtCbRYqRnjRlHzppEqN5aZp2RxiBXmomlCuENumXMd3JKSyNF3y6414B+wFYIyg/QJCWOom6qYlWonvv3Wqs2POS3w6X6rOhGrdUrhIxvIlz4c7ocbgJxZAIgoH2znI8zM7ifXrAxG0BCQ4VpNirGCkNRJTkcF70h8CKJ0mkabxtK7S+xNlTw7uG6/zzKvidFfZqkYsxVAfUULiyKyYtOPDz/MQSCrBTgbGqKov5NlmoXwLw7XOibfdStB/Aq0GLnGEhI+YSDCq027HvXASKffG1wVknsVq4n9At3Jzh6FnlpEU/88X2NQPHzxbMt450TE8yZLt3caQYfswKv3F9oYPDPC2lBGv1Uy5dupI+xnh1aI/X+H0OCrk0JQ1rsEP8urWr9Z2yibZh8o5hmkI7BfB6tWxdgjODVx+9GJHxdaryjSLNzP5GjbIuPqE9xKfah3D6HAmiZUMTJtQqv2o+inyeo0vL65pr6RyEzjWjrKCsgTbrai5PQg8Zz4UhxoyVLCAfBgZrH7YSEOsL9h/al7xvUEE9mqfnu1TH4oXSo/HrPi74hG+PFwyxOJIhb36CVtuSfI6TBvRkW6e3lWzGp0ug+h9R361QzDW95WlamhrPpYgtrlebmNl+MNgaJL7epB2 usxeR+p5 aBuaKV/2Hn0wmDOmt+YsKP06vAbJwxTrXAsGnPr+knerkBAB0+J0tXFPlbmUgO1G0yBGo7fz5XcWkdqtuaF6kHUOm55pumLdPfkFT1ik9PkYRdfoPct9IMEsxgJQHz08zx5T0evKsjHYKmPuTdD2jkls03S/VZL8A0dd/87NCKPG9DqlXn2S23RfmAiGf1MO4m6cqYwRCm0J5hQUozf8K+HZ2Fpq8igFA2o+NdO5+T7hA+7mcBz40pgIQ9qnXw+pl9csdvE8LqwkMTWYxHQmDldY5jotLavM3OxhN+lgqGjc9qIZpU3QgfOM3PvKH8iv6kIFn42XqOhNbqKuxqCaO284Jau0AAEWkGfF8QMjhbH46A8yzu16lEaQVmgLqDtShytxpyxl9VL0g2hQldqlFt+5GiCznfotIi39Efy+co0x82p18SfMvsIneudva7JhBVaA810hixclYUY1BGnYE2U1bFgGThxRZjRQ8ABJk1c8fqsD2M3upk14zhK2aQEfVGqpRdYDatUcQNkj0LAKi3kGfbt0/NdRTFNTbsjZj2aER9Szr5cE1mrlL+faSCZq0QvNjnPGwelzq06oBB5OxZYukJfCRR0sbTBZHhbzPQqiztZVOVEvqQYLl7X0LPKP5rMoYuGeHB9AwUifiPd8IGduxSZ+AxyKC73Dr9Tk466GPaUS45nC3+4xVSJZRVfKEXfDNQLgniki94WOVWDLVIYhhZ/Mh+WK8tXOE8Vw7zkA0Xlbt2rVizxMJmlgj3/Dkk7I+9RMY/+uhdgkSjqcZj8d6F1OGlsWL/lDEmORexI5dnlSpQ7SMduvFlJXbP4SxykH0hfHFa/jn8PZeSWgbztUcj+RKdK4PCIFJ1ibLY/kJgm8bTNe3HPSacFVj54LI2AWpYdavfxYDV2G7adXDwhLD9M0MRO0usiP/54i7GopskZxEc51GzYbZGKPQx7JcBjt+mGtbypE8ursMAg3FSMHjSWQU eIFDsNnV fwk+b1BskKfrMc+/xjoeby22VpyIJCDhiimBIKw/ZIk1dMODG3HQkGYt9XnYk9CXfPA2nXqXDajRTMpR8oC9toVJTjxJXXu5/LXQ6WvNay4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.010788, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Aug 27, 2024 at 04:07:57AM +0000, gaoxu wrote: > > > > -----邮件原件----- > > 发件人: Lokesh Gidra > > 发送时间: 2024年8月27日 8:12 > > 收件人: Barry Song <21cnbao@gmail.com> > > 抄送: Suren Baghdasaryan ; Nicolas Geoffray > > ; Michal Hocko ; gaoxu > > ; Andrew Morton ; > > linux-mm@kvack.org; linux-kernel@vger.kernel.org; Shaohua Li ; > > yipengxiang ; fengbaopeng > > ; Kalesh Singh > > 主题: Re: [PATCH v2] mm: add lazyfree folio to lru tail > > > > On Mon, Aug 26, 2024 at 12:55 PM Barry Song <21cnbao@gmail.com> wrote: > > > > > > On Tue, Aug 27, 2024 at 4:37 AM Lokesh Gidra > > wrote: > > > > > > > > Thanks Suren for looping in > > > > > > > > On Fri, Aug 23, 2024 at 4:39 PM Suren Baghdasaryan > > wrote: > > > > > > > > > > On Wed, Aug 21, 2024 at 2:47 PM Barry Song <21cnbao@gmail.com> > > wrote: > > > > > > > > > > > > On Wed, Aug 21, 2024 at 8:46 PM Michal Hocko > > wrote: > > > > > > > > > > > > > > On Fri 16-08-24 07:48:01, gaoxu wrote: > > > > > > > > Replace lruvec_add_folio with lruvec_add_folio_tail in the > > lru_lazyfree_fn: > > > > > > > > 1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's > > > > > > > > moved to the LRU tail, it allows for faster release lazy-free folio > > and > > > > > > > > reduces the impact on file refault. > > > > > > > > > > > > > > This has been discussed when MADV_FREE was introduced. The > > question was > > > > > > > whether this memory has a lower priority than other inactive memory > > that > > > > > > > has been marked that way longer ago. Also consider several > > MADV_FREE > > > > > > > users should they be LIFO from the reclaim POV? > > > > > > > > Thinking from the user's perspective, it seems to me that FIFO within > > > > MADV_FREE'ed pages makes more sense. As a user I expect the longer a > > > > MADV_FREE'ed page hasn't been touched, the chances are higher that it > > > > may not be around anymore. > > > > > > > > > > > > Hi Lokesh, > > > Thanks! > > > > > > > > > The priority of this memory compared to other inactive memory that has > > been > > > > > > marked for a longer time likely depends on the user's expectations - How > > soon > > > > > > do users expect MADV_FREE to be reclaimed compared with old file > > folios. > > > > > > > > > > > > art guys moved to MADV_FREE from MADV_DONTNEED without any > > > > > > useful performance data and reason in the changelog: > > > > > > https://android-review.googlesource.com/c/platform/art/+/2633132 > > > > > > > > > > > > Since art is the Android Java heap, it can be quite large. This increases the > > > > > > likelihood of packing the file LRU and reduces the chances of reclaiming > > > > > > anonymous memory, which could result in more file re-faults while > > helping > > > > > > anonymous folio persist longer in memory. > > > > > > > > Individual heaps of android apps are not big, and even in there we > > > > don't call MADV_FREE on the entire heap. > > > > > > How do you define "Individual heaps of android apps", do you know the usual > > > total_size for a phone with memory pressure by running multiple apps and > > how > > > much for each app? > > > > > Every app is a separate process and therefore has its own private ART > > heap. Those numbers that you are asking vary drastically. But here's > > what I can tell you: > > > > Max heap size for an app is 512MB typically. But it is rarely entirely > > used. Typical heap usage is 50MB to 250MB. But as I said, not all of > > it is MADV_FREE'ed. Only those pages which are freed after GC > > compaction are. > > > > > > > > > > > > I am really curious why art guys have moved to MADV_FREE if we have > > > > > > an approach to reach them. > > > > > > > > Honestly, it makes little sense as a user that calling MADV_FREE on an > > > > anonymous mapping will impact file LRU. That was never the intention > > > > with our ART change. > > > > > > > > > > This is just how MADV_FREE is implemented in the kernel, this kind of lazyfree > > > anon folios are moved to file but *NOT* anon LRU. > > > > > > > From our perspective, once a set of pages are MADV_FREE'ed, they are > > > > like a page-cache. It gives an opportunity, without hurting memory > > > > use, to avoid overhead of page-faults, which happen frequently after > > > > GC is done on running apps. > > > > > > > > IMHO, within LRU_INACTIVE_FILE, MADV_FREE'ed pages should be > > > > prioritized for reclamation over file ones. > > > > > > This is exactly what this patch is doing, putting lazyfree anon folios > > > to the tail of file LRU so that they can be reclaimed earlier than file > > > folios. But the question is: is the requirement "MADV_FREE'ed pages > > > should be prioritized for reclamation over file ones" universally true for > > > all other non-Android users? > > > > > That's definitely an important question to get answered. But putting > > my users hat on again, by explicitly MADV_FREE'ing we ask for that > > behavior. IMHO, MADV_FREE'ed pages should be the first ones to be > > reclaimed on memory pressure. > For non-Android systems, perhaps the author of MADV_FREE can provide a more > reasonable opinion; > > Add Minchan Kim. > Please forgive me for forgetting to add you when sending the patch. AFAIR, there were two concerns: 1. The file LRU would contain pages used only once. While MADV_FREE allows discarding pages under memory pressure, the system would still have non-working set pages within the file LRU (e.g., those used only once). 2. LRU inversion among MADV_FREE users. Consider this time order: 1. A process: MADV_FREE 2. B process: MADV_FREE 3. C process: MADV_FREE The moving tail approach would discard the most recent pages from Process C first, instead of those from Process A. Of course, this isn't universally true for all workloads, but it's the reality. At the time, I proposed introducing an additional "ez_reclaimable" LRU list to store MADV_FREE pages (and potentially other hinted pages in the future). This would allow differentiating priority among LRU lists based on knobs or heuristics. However, this idea wasn't well-received.