From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1042EC27C53 for ; Wed, 12 Jun 2024 17:23:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9E5C36B0099; Wed, 12 Jun 2024 13:23:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 995A36B009A; Wed, 12 Jun 2024 13:23:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 836AF6B009B; Wed, 12 Jun 2024 13:23:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 60D586B0099 for ; Wed, 12 Jun 2024 13:23:44 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id DAF0F121836 for ; Wed, 12 Jun 2024 17:23:43 +0000 (UTC) X-FDA: 82222908726.27.1EB9999 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf25.hostedemail.com (Postfix) with ESMTP id 1EFA8A000D for ; Wed, 12 Jun 2024 17:23:40 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=kUfhJ70K; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of 3nNlpZgYKCDEfRNaWPTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--seanjc.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3nNlpZgYKCDEfRNaWPTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--seanjc.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718213021; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=v0uZvClGSP7b0LMzzAPVM0AhM29ZDe2y2D54OZuyJ4o=; b=cR4rklt3jrVpGm3JO/c4RqWXZD8t8F7ttC8swyoWH4+Qwd9srg/0Ievyg0S2zr69jhkVAn aQJyWa7OC+GWe3UboKOmecc/E/JeEeEbdAja5hCm+fmUhXM125cbz0JfzWRHDY1davZJRa KVhACt0ECT4Od7MuWLG7nNDubZl8AKk= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=kUfhJ70K; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf25.hostedemail.com: domain of 3nNlpZgYKCDEfRNaWPTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--seanjc.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3nNlpZgYKCDEfRNaWPTbbTYR.PbZYVahk-ZZXiNPX.beT@flex--seanjc.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718213021; a=rsa-sha256; cv=none; b=S6odUD8Qe7MMb9bmD9wn7OyOhpOo05ovlgrOma79+TmFc4LCtrBoTA/ihB8QT+hjphRGB6 3A/uoaPjkLZ09hGza6KNdc2CR2QKilUTXCOyxD+Wk0DcKLnUpiiRSvvNwntvdtpjJE4SbU pA93X6llYl8zgxjPSy0E5/JIAMt6Vpo= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dfb0e59ac7cso181589276.0 for ; Wed, 12 Jun 2024 10:23:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1718213020; x=1718817820; darn=kvack.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=v0uZvClGSP7b0LMzzAPVM0AhM29ZDe2y2D54OZuyJ4o=; b=kUfhJ70KJPpiLdKcrLSGWpTUUUnoxYcvx8elfELeholFqnPfeGePfPt+icA4GGm1Kg afoc0ZgiRc1NXSmSTfnNqe1tzZb5uVwQD55ooliUlvADcKX4gFcJgVp7/q5/hF+Ck4RB 6eEZtyMOxKydsmxf//XG8VU5Gt6Cx/IBsryyUXMc28ZKu5vkVNNOghchA85njS6JWoXt klRnKayD2GFnT871FWUsXBbP77Mk76BChXMi9IKCEoczfcwxEh/WWO3rdnUMawseH2sW StBdf/ryXmmgX1YO0ZrPROO+otTBv1jh/MFxSctDK89C5My3KKfHizTPPDffR+NmshJW b0TQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718213020; x=1718817820; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=v0uZvClGSP7b0LMzzAPVM0AhM29ZDe2y2D54OZuyJ4o=; b=jtq3woula7p4dkl7L/pIuIkd/UONmEKPdo8qWN5drpaA/DTYWShY2GoUpvRF6FBRAP vCgyHmp8sCSpReFkQ+clyZcmsSw2tTJ1TSjjDQ3k3D2xpUu8hPOt3HXQXdhj26rKg/5R hPrmNHZnYHUQ5h9GR4xIyLyXfNfa2w7CfdQNGRnF72MfrfKhI6ZG4Il8oYcOKNK0uGE2 zHiEv11UrSDH236r+vr5IzygWYd5E5GLvDYoOXT+2WD7UFOBQzNgRHq4lXajeFB/cOMB ujnmLoZ+9Jc+sjzSEDbsjBRIGFGoh58tjrKpDy3ypqJgO3aQL1YSQ5vINoTwTknh6/kL EP8A== X-Forwarded-Encrypted: i=1; AJvYcCWXtzK6NGagx5+S/pfnVcvYSq0h5T6+XGsV3fMi7X5c/UFbSnYaZXy7ws6kVzNB1T3E/hkn3bEQ/TEJ7xlVrAtKMgI= X-Gm-Message-State: AOJu0Yx9zKX1MU3HhNK074Hy67ag3ldCDlakzch5ZTVEniWby4nSpS5r xL6O5gvL1fnqsxNHHp2hfnAo+hDrTt5iIO8D2dE45eT19LKurnfawHcG63XG847G1oynamtaIvO yIw== X-Google-Smtp-Source: AGHT+IHsuziTjwu8H3LsGOZG3nq2ECdUkM7fJ/s+9lw1k304/SuAsQ64WUp2lf97WzRuhz9PeWwlSE0Bo58= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:729:b0:dfb:b4e:407a with SMTP id 3f1490d57ef6-dfe68035fbemr647880276.9.1718213020117; Wed, 12 Jun 2024 10:23:40 -0700 (PDT) Date: Wed, 12 Jun 2024 10:23:38 -0700 In-Reply-To: Mime-Version: 1.0 References: <20240611002145.2078921-1-jthoughton@google.com> <20240611002145.2078921-9-jthoughton@google.com> Message-ID: Subject: Re: [PATCH v5 8/9] mm: multi-gen LRU: Have secondary MMUs participate in aging From: Sean Christopherson To: Yu Zhao Cc: James Houghton , Andrew Morton , Paolo Bonzini , Ankit Agrawal , Axel Rasmussen , Catalin Marinas , David Matlack , David Rientjes , James Morse , Jonathan Corbet , Marc Zyngier , Oliver Upton , Raghavendra Rao Ananta , Ryan Roberts , Shaoqin Huang , Suzuki K Poulose , Wei Xu , Will Deacon , Zenghui Yu , kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 1EFA8A000D X-Rspamd-Server: rspam10 X-Rspam-User: X-Stat-Signature: rmnpod38qyno3mrg5hd6yfm7ufbdf3bn X-HE-Tag: 1718213020-743233 X-HE-Meta: U2FsdGVkX19SbPv2P8MUA3ei81BvI0o23o4auDxXKfbRZrSPxKaznVpmrB23vtqw7eE1BMRlnIX8022pVnDtmglooYvpl/3dQGfOVH6lv+WCO6M4j/CNDudxP9qytp661DYNUUujpRZlya+A2Ogl5DTOs/y42PDBbKh/9B2areaMyCHJujtyB6fHZSWvflE1kOks5g+HyFJkyKabQ/0EdgY/grDebqPBCQ+5zPtANFRbgftZvA2xSM1tpbk54wxQ3ZP71j6FMu/5SmAW+RxVubTX10izDa2xiLK+Cwrb6pA0HTv6FAq+yGNHR5ojxq1hVCneYaroTl5s6TB3vP8cbD26ZkEvfGquTb1y/KJMVOYOEBNd9QesJ+172he2DDxLTgiukkVtNy5aNq9zyOq0J2kAT276u1JbfqSpHhss/Ame7d7TS7RyaLrSWBmIGpbE42fOB9oSbtb5ZgJa5+oYBZeWG1pxK5htRi5CP37G4nKaDzLi/rXxgxka2InIAqkOxCCs0DOoIX+erOYib/FQbT2k41nBbNOH3rakhWDm4GqUZbD0ccC2kMgQaUXniqd8ZOc/6tWjgfWO3AbZ4DCa2kEGlI+KPphOis7X5cCHFQdnNiXMcwb9kPcXau2uAtml2cwH/xJzmmOJYi/VMoQDpWXbTIvEICkf1oPfFSnfya9BUWN5j61HP9F/qBO/k2vYs3BzsUhzaTVTgBm1/FHSuTeqJE0ZGFrHZcbfOPp5MSUnt/51xaGUjG2h7uzKLjhQSdZ2pheaHasosdd+hVpJh6oUNoqEQVb/1vyhIDb+sKBfXcRO2ArUT5V7l8eu4nbQxs1ZGbEojcD3QksbzdtnP+Bka+Al16fGVS8qQ196PRJdUZt8OdaCUZRwKF3O5eEYWxF1HvseqIlsyYFOb2f8OzSwyOdNDPSqgblk3glZdKjvxVnuL4iZdnBabuCDtKoUfZD6wsSCph0yd4bzFrm AvpPAJmg 4kru+au2VJhpuCtdg8e2lx/75XxCSDeZNpTkFD9gEvxrlk37AQsu7MkbdrPRhjKceEICMJuEK5er1rKNzsjXl6zsJ4FPHuRQYHRPetnkhrH/62bBDSIcC+3RO06V6o9pPrL2hDJKb368WpjPN3pF9mjsgBDVBUtylIDOYhr0JePcOtd/lQ8mbNPZzmUV1P/jPei+lYLGOp9XoQEepzDvDSoCGnVf5Mw7cLr7gq3FMgVnscLQnwmi77T3Icbd++IKSA/aJzAVj+tKr+633egdmke0GYycxWibuVLNp8tBol6WQte9NrBOQ6V4hpQafo++EWVnWS+gcYmYwgbtVjbsr5T+/uwpF0QtwumItGu0F9IJwg8MhPTsMpiYlc4bM0Yxbf7L4KvWGw4e44gaExE8MTABAHcyNnjPAHzcxf/VLAJ+h2z6mzPXUlRanDKItWbkB/Hrex8P5EtnlyimJMqHVEMn6pS43KUQUmPWACBfmIO/6oKvFqTEPzs47INEYGZKvHpG/+rCq3AMeF96MZ0dM5otwMgqpKCo5bW/aZgrQQ6iR/MNSrrL9+Jmr8MGZuHFH8kKBCvGwEBxmZPU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000035, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jun 12, 2024, Yu Zhao wrote: > On Wed, Jun 12, 2024 at 10:02=E2=80=AFAM Sean Christopherson wrote: > > > > On Tue, Jun 11, 2024, James Houghton wrote: > > > diff --git a/mm/rmap.c b/mm/rmap.c > > > index e8fc5ecb59b2..24a3ff639919 100644 > > > --- a/mm/rmap.c > > > +++ b/mm/rmap.c > > > @@ -870,13 +870,10 @@ static bool folio_referenced_one(struct folio *= folio, > > > continue; > > > } > > > > > > - if (pvmw.pte) { > > > - if (lru_gen_enabled() && > > > - pte_young(ptep_get(pvmw.pte))) { > > > - lru_gen_look_around(&pvmw); > > > + if (lru_gen_enabled() && pvmw.pte) { > > > + if (lru_gen_look_around(&pvmw)) > > > referenced++; > > > - } > > > - > > > + } else if (pvmw.pte) { > > > if (ptep_clear_flush_young_notify(vma, address, > > > pvmw.pte)) > > > referenced++; > > > > Random question not really related to KVM/secondary MMU participation. = AFAICT, > > the MGLRU approach doesn't flush TLBs after aging pages. How does MGLR= U mitigate > > false negatives on pxx_young() due to the CPU not setting Accessed bits= because > > of stale TLB entries? >=20 > I do think there can be false negatives but we have not been able to > measure their practical impacts since we disabled the flush on some > host MMUs long ago (NOT by MGLRU), e.g., on x86 and ppc, > ptep_clear_flush_young() is just ptep_test_andclear_young(). Aha! That's what I was missing, I somehow didn't see x86's ptep_clear_flus= h_young(). That begs the question, why does KVM flush TLBs on architectures that don't= need to? And since kvm_mmu_notifier_clear_young() explicitly doesn't flush, are= there even any KVM-supported architectures for which the flush is mandatory? Skipping the flush on KVM x86 seems like a complete no-brainer. Will, Marc and/or Oliver, what are arm64's requirements in this area? E.g.= I see that arm64's version of __ptep_clear_flush_young() does TLBI but not DSB. = Should KVM be doing something similar? Can KVM safely skip even the TBLI? > theoretical basis is that, given the TLB coverage trend (Figure 1 in > [1]), when a system is running out of memory, it's unlikely to have > many long-lived entries in its TLB. IOW, if that system had a stable > working set (hot memory) that can fit into its TLB, it wouldn't hit > page reclaim. Again, this is based on the theory (proposition) that > for most systems, their TLB coverages are much smaller than their > memory sizes. >=20 > If/when the above proposition doesn't hold, the next step in the page > reclaim path, which is to unmap the PTE, will cause a page fault. The > fault can be minor or major (requires IO), depending on the race > between the reclaiming and accessing threads. In this case, the > tradeoff, in a steady state, is between the PF cost of pages we > shouldn't reclaim and the flush cost of pages we scan. The PF cost is > higher than the flush cost per page. But we scan many pages and only > reclaim a few of them; pages we shouldn't reclaim are a (small) > portion of the latter. >=20 > [1] https://www.usenix.org/legacy/events/osdi02/tech/full_papers/navarro/= navarro.pdf