From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sean Christopherson Date: Mon, 3 Jun 2024 16:03:05 -0700 Subject: [PATCH v4 2/7] mm: multi-gen LRU: Have secondary MMUs participate in aging In-Reply-To: References: <20240529180510.2295118-1-jthoughton@google.com> <20240529180510.2295118-3-jthoughton@google.com> Message-ID: List-Id: To: kvm-riscv@lists.infradead.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Mon, Jun 03, 2024, James Houghton wrote: > On Thu, May 30, 2024 at 11:06?PM Yu Zhao wrote: > > What I don't think is acceptable is simplifying those optimizations > > out without documenting your justifications (I would even call it a > > design change, rather than simplification, from v3 to v4). > > I'll put back something similar to what you had before (like a > test_clear_young() with a "fast" parameter instead of "bitmap"). I > like the idea of having a new mmu notifier, like > fast_test_clear_young(), while leaving test_young() and clear_young() > unchanged (where "fast" means "prioritize speed over accuracy"). Those two statements are contradicting each other, aren't they? Anyways, I vote for a "fast only" variant, e.g. test_clear_young_fast_only() or so. gup() has already established that terminology in mm/, so hopefully it would be familiar to readers. We could pass a param, but then the MGLRU code would likely end up doing a bunch of useless indirect calls into secondary MMUs, whereas a dedicated hook allows implementations to nullify the pointer if the API isn't supported for whatever reason. And pulling in Oliver's comments about locking, I think it's important that the mmu_notifier API express it's requirement that the operation be "fast", not that it be lockless. E.g. if a secondary MMU can guarantee that a lock will be contented only in rare, slow cases, then taking a lock is a-ok. Or a secondary MMU could do try-lock and bail if the lock is contended. That way KVM can honor the intent of the API with an implementation that works best for KVM _and_ for MGRLU. I'm sure there will be future adjustments and fixes, but that's just more motivation for using something like "fast only" instead of "lockless". > > > I made this logic change as part of removing batching. > > > > > > I'd really appreciate guidance on what the correct thing to do is. > > > > > > In my mind, what would work great is: by default, do aging exactly > > > when KVM can do it locklessly, and then have a Kconfig to always have > > > MGLRU to do aging with KVM if a user really cares about proactive > > > reclaim (when the feature bit is set). The selftest can check the > > > Kconfig + feature bit to know for sure if aging will be done. > > > > I still don't see how that Kconfig helps. Or why the new static branch > > isn't enough? > > Without a special Kconfig, the feature bit just tells us that aging > with KVM is possible, not that it will necessarily be done. For the > self-test, it'd be good to know exactly when aging is being done or > not, so having a Kconfig like LRU_GEN_ALWAYS_WALK_SECONDARY_MMU would > help make the self-test set the right expectations for aging. > > The Kconfig would also allow a user to know that, no matter what, > we're going to get correct age data for VMs, even if, say, we're using > the shadow MMU. Heh, unless KVM flushes, you won't get "correct" age data. > This is somewhat important for me/Google Cloud. Is that reasonable? Maybe > there's a better solution. Hmm, no? There's no reason to use a Kconfig, e.g. if we _really_ want to prioritize accuracy over speed, then a KVM (x86?) module param to have KVM walk nested TDP page tables would give us what we want. But before we do that, I think we need to perform due dilegence (or provide data) showing that having KVM take mmu_lock for write in the "fast only" API provides better total behavior. I.e. that the additional accuracy is indeed worth the cost. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B7EB5028C for ; Mon, 3 Jun 2024 23:03:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717455789; cv=none; b=Cxlsn0fX2NOcocqHn7YFc/RUbub8Lj0B13gqD1lHk/3MsQI6OJ49fCrML7/DHDmdOQRHnnkQFkGieI8ofcU/X5355nr6fs69SpkrzcuDB4X3Zd6OVuIFrwvZYQFxsfdAZrsLukI8kaMCKOyUa5unkCVoenCQP+JCNAmYTCnEub4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717455789; c=relaxed/simple; bh=5+qBTnQ2jcGHCKlfoNDyIJ+P5xpxSFVMeOuIzIxDers=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jLpEQ7HcAJm/FqKr/vy9DP+a+zbXkcqauhIdjTsJ2vYQ+UjLs1V2sGfcBY1GT0o7BY9UXQ6/P0s6KPF7YwU5HbJMRVxKCtdmwmXcFJMDwR19/voRJv5cwRgrnzcsRdC57uTWFYGVwtCKvAZicmrZEjoWiXAyjPth2OV79zWpmiw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=awioB+gF; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="awioB+gF" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1f6174d0421so19615455ad.2 for ; Mon, 03 Jun 2024 16:03:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1717455787; x=1718060587; darn=lists.linux.dev; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=w0SJTxFtK/1G56YdHzcPYdHV7zjy1ZVa0L4hU78kSrI=; b=awioB+gF4tGuz6anvbp+WsNTYZg3V/L7mi0+OstEegnjtIYXdllt85bQQlHF0RC/Lj tXfPRC1cJuzrUlvDt9axhuNS5kYEygju81zjK8gpT6MNhMrtQZwe+MkdMvPdO8nyxeK8 JUDeeadMKYhqBLBIXUBLQYOek7LEv2Rbax4/qFiDYfryJ5cVXJZo4HCHXtXlNbogkx6o el18J5CiPSbeQsMp6JZiQ8X0RM5P/ukCFMDaIuiv7UU5lzR6nR8EcjPaQNeoG+PYHQ2B /b/IR6Pt4y9Q9Iu/E0FAO4lIV9HqjI2irhxFB3ZmfsdYTREonqZsZwckv7mH5hdELU4a BoFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717455787; x=1718060587; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=w0SJTxFtK/1G56YdHzcPYdHV7zjy1ZVa0L4hU78kSrI=; b=mFu3jXveUP77ngTTttOafCdCSoSWOKUGbtq47RtRmQfT4CECg57A52CcULsR+k1qWW BDNsN/UayJGEY6A1frqkEqAw6oTN6va19KrNf3SkHfydu8JQl4dEs9K1oTMKTIkiZ20D oB5Y74bPWwB3Qtbm7I6NDEdqHVeD9oC86VQuPWkL9oRsmaZTC3CbFe8GK9OwVTHHy7ob TPPNy6321cgIh5XzeYmQMYFZ/tDqtjIsMJ80CfAV/twCKczY3LsaXkXzY4T+WCBZhhZM l+KxDsEDIjE+hpq6vyLjK5ti0dJxE7bBZSaSAYzBye3I6Lvsgby6WHi20YV9x17Lk65m yQaw== X-Forwarded-Encrypted: i=1; AJvYcCUH4Hke/cvIJAUbgbrZE9BhBcGSSCvAU4mPwpuWs20a2Eb2uzPHlLKGHLD3Uo0HObvmeD/QlGW3o7xeBEX7AgA5px8TvoA2 X-Gm-Message-State: AOJu0YxjaNHfHchDAUIDRL3DYGh0rugckL56VMG4rMu8AnQMSV/PrD69 J7HfgN57UTLc0yp5bfSSGTCDlvSS0IJCFKd8wVV7elDGx5TxpUVuB6QaPrSdK6TyJPz78A8kzvy S5g== X-Google-Smtp-Source: AGHT+IH9puzU5V8aMTwgzoabUylaHv5IhJEL6Kgk0jFfHrRaPIZ6XCL4YZWlIMRWpWZxlsRoR1ZSqKiiwm0= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:ea05:b0:1f6:3891:794a with SMTP id d9443c01a7336-1f638917b67mr7110545ad.10.1717455787406; Mon, 03 Jun 2024 16:03:07 -0700 (PDT) Date: Mon, 3 Jun 2024 16:03:05 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240529180510.2295118-1-jthoughton@google.com> <20240529180510.2295118-3-jthoughton@google.com> Message-ID: Subject: Re: [PATCH v4 2/7] mm: multi-gen LRU: Have secondary MMUs participate in aging From: Sean Christopherson To: James Houghton Cc: Yu Zhao , Andrew Morton , Paolo Bonzini , Albert Ou , Ankit Agrawal , Anup Patel , Atish Patra , Axel Rasmussen , Bibo Mao , Catalin Marinas , David Matlack , David Rientjes , Huacai Chen , James Morse , Jonathan Corbet , Marc Zyngier , Michael Ellerman , Nicholas Piggin , Oliver Upton , Palmer Dabbelt , Paul Walmsley , Raghavendra Rao Ananta , Ryan Roberts , Shaoqin Huang , Shuah Khan , Suzuki K Poulose , Tianrui Zhao , Will Deacon , Zenghui Yu , kvm-riscv@lists.infradead.org, kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On Mon, Jun 03, 2024, James Houghton wrote: > On Thu, May 30, 2024 at 11:06=E2=80=AFPM Yu Zhao wrot= e: > > What I don't think is acceptable is simplifying those optimizations > > out without documenting your justifications (I would even call it a > > design change, rather than simplification, from v3 to v4). >=20 > I'll put back something similar to what you had before (like a > test_clear_young() with a "fast" parameter instead of "bitmap"). I > like the idea of having a new mmu notifier, like > fast_test_clear_young(), while leaving test_young() and clear_young() > unchanged (where "fast" means "prioritize speed over accuracy"). Those two statements are contradicting each other, aren't they? Anyways, I= vote for a "fast only" variant, e.g. test_clear_young_fast_only() or so. gup() = has already established that terminology in mm/, so hopefully it would be famil= iar to readers. We could pass a param, but then the MGLRU code would likely en= d up doing a bunch of useless indirect calls into secondary MMUs, whereas a dedi= cated hook allows implementations to nullify the pointer if the API isn't support= ed for whatever reason. And pulling in Oliver's comments about locking, I think it's important that= the mmu_notifier API express it's requirement that the operation be "fast", not= that it be lockless. E.g. if a secondary MMU can guarantee that a lock will be contented only in rare, slow cases, then taking a lock is a-ok. Or a secon= dary MMU could do try-lock and bail if the lock is contended. That way KVM can honor the intent of the API with an implementation that wo= rks best for KVM _and_ for MGRLU. I'm sure there will be future adjustments an= d fixes, but that's just more motivation for using something like "fast only" instea= d of "lockless". > > > I made this logic change as part of removing batching. > > > > > > I'd really appreciate guidance on what the correct thing to do is. > > > > > > In my mind, what would work great is: by default, do aging exactly > > > when KVM can do it locklessly, and then have a Kconfig to always have > > > MGLRU to do aging with KVM if a user really cares about proactive > > > reclaim (when the feature bit is set). The selftest can check the > > > Kconfig + feature bit to know for sure if aging will be done. > > > > I still don't see how that Kconfig helps. Or why the new static branch > > isn't enough? >=20 > Without a special Kconfig, the feature bit just tells us that aging > with KVM is possible, not that it will necessarily be done. For the > self-test, it'd be good to know exactly when aging is being done or > not, so having a Kconfig like LRU_GEN_ALWAYS_WALK_SECONDARY_MMU would > help make the self-test set the right expectations for aging. >=20 > The Kconfig would also allow a user to know that, no matter what, > we're going to get correct age data for VMs, even if, say, we're using > the shadow MMU. Heh, unless KVM flushes, you won't get "correct" age data. > This is somewhat important for me/Google Cloud. Is that reasonable? Maybe > there's a better solution. Hmm, no? There's no reason to use a Kconfig, e.g. if we _really_ want to p= rioritize accuracy over speed, then a KVM (x86?) module param to have KVM walk nested= TDP page tables would give us what we want. But before we do that, I think we need to perform due dilegence (or provide= data) showing that having KVM take mmu_lock for write in the "fast only" API prov= ides better total behavior. I.e. that the additional accuracy is indeed worth t= he cost. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5406EC25B78 for ; Mon, 3 Jun 2024 23:03:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=dg6Q26slwZY5XwUEaLUQJPBWavEk997J4aiCm8Mbg3w=; b=uoEd0t5DFGYABe8vyGKD/Jsq8n CR1mJuRaC3mU01PM4Hkdgf3ry+1i2e8CjrO3ZPI5TPGCERwfvrdVKS4pdJZyS7xFP64Tt8NGBXwhC omDWRnbsPlQ24GN/uD5HYLV1rY0kLtlRYjS8Sa6XJcry7QCASvvq5yIeEOFoG88KV8eETpLigYLDy P2nCXmIaDZSmLiuoSBNai8zh8CqHiYgUFY8LDOWy2y2EsYdG4jFTk6XKk/Q+QIV+196yV+7Gy0F2w V5z8fAciVU+7liBK0mMNjdUNEwrU8httRNm+ppp6i3LV/wKleldljQxa3D3vlyl6qtGk+ZUCK/Q0j L+P2uivw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEGi1-00000000ZiD-2fYE; Mon, 03 Jun 2024 23:03:21 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEGhw-00000000ZbY-07NP for linux-riscv@lists.infradead.org; Mon, 03 Jun 2024 23:03:19 +0000 Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-1f6174d0421so19615475ad.2 for ; Mon, 03 Jun 2024 16:03:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1717455787; x=1718060587; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=w0SJTxFtK/1G56YdHzcPYdHV7zjy1ZVa0L4hU78kSrI=; b=IVSkrx2SVLgc6N/ZyTV8E9DWpjYxSsSk0dfu+0sxPTtR3B+Y9Myk1iB/OI1cNb6PjJ xPubOkfyVVR+qbicp2q/PDhTKXUa0cgSxpBAhvhe1Q/hctwm5vCXjkTsBGnrMzD1/ddp pHlsBEyXR331MV34eFEx5r/xZzhITwDmBUnQeiW3qUtdomJRx8PVLCH7upG6z7JvMkM6 NxJfl/C1wti4At/owVm3LPP80JZL3F3pHbvbGK4N/JOkn7rulphO0IwyF585V0SKdiIb Q/JidIvlkv320B+DdaLAhT+Scm/ECUzBDJnxhXQRZKMJrTvahAsiZvowCVA7uAJPEXvd RNuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717455788; x=1718060588; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=w0SJTxFtK/1G56YdHzcPYdHV7zjy1ZVa0L4hU78kSrI=; b=QyZx7alHjDNK38fT5IbyVu3jM0QJ/PuGYv06QzWPZstQ8kRNT3lrqtC++mKTb/2HE0 UrWCZXIdcbGgeygKMMhDG7E8LREClWMhQVGdf9KolXJBjLkhFHRULKhXMlCd8+MnMXsH o5Ad3nm1eWXdfnnOw0n/1h/lUaCTM+Gkkv4Pp3m0btZoYrx7BL6Cf9w1ZB+wt9/MC5eN 7PZxIUru41gdJKMzgdetkfekQfxwtbIwTIxZ3ckqzrw0MYACnjOM2RBO7mDc3pih2DiB kXVNmAxo7JX2CAS5KaNwEIiBzqwUkddn9Pxk/eeLSEYXtVK6+ilHuBM/xatMdu9+MLP9 KOfQ== X-Forwarded-Encrypted: i=1; AJvYcCU0oKy2z6LZmrDNi0pp8/hk+y3jPnqS57Oz+VRO7FpWf8bzrW9BOBsaGCdwtFT2Y4Yq13beHFjpYsbUrdSbISg9SPXVVqfAJ+Cvl9Va5kms X-Gm-Message-State: AOJu0YyZawVyHfPU2MD+IrA1rrJp8L1v7++iIKVj16P4310pUxXMC2Ne UKFtOCEaRe6S5XTWZrti631tAtoRusZQ8QhfolxgBuu9/VF7iaFXsFZoKCUa87Ky2eYLTfCW4Ik LQg== X-Google-Smtp-Source: AGHT+IH9puzU5V8aMTwgzoabUylaHv5IhJEL6Kgk0jFfHrRaPIZ6XCL4YZWlIMRWpWZxlsRoR1ZSqKiiwm0= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:ea05:b0:1f6:3891:794a with SMTP id d9443c01a7336-1f638917b67mr7110545ad.10.1717455787406; Mon, 03 Jun 2024 16:03:07 -0700 (PDT) Date: Mon, 3 Jun 2024 16:03:05 -0700 In-Reply-To: Mime-Version: 1.0 References: <20240529180510.2295118-1-jthoughton@google.com> <20240529180510.2295118-3-jthoughton@google.com> Message-ID: Subject: Re: [PATCH v4 2/7] mm: multi-gen LRU: Have secondary MMUs participate in aging From: Sean Christopherson To: James Houghton Cc: Yu Zhao , Andrew Morton , Paolo Bonzini , Albert Ou , Ankit Agrawal , Anup Patel , Atish Patra , Axel Rasmussen , Bibo Mao , Catalin Marinas , David Matlack , David Rientjes , Huacai Chen , James Morse , Jonathan Corbet , Marc Zyngier , Michael Ellerman , Nicholas Piggin , Oliver Upton , Palmer Dabbelt , Paul Walmsley , Raghavendra Rao Ananta , Ryan Roberts , Shaoqin Huang , Shuah Khan , Suzuki K Poulose , Tianrui Zhao , Will Deacon , Zenghui Yu , kvm-riscv@lists.infradead.org, kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240603_160316_096565_00DBD78B X-CRM114-Status: GOOD ( 32.02 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org T24gTW9uLCBKdW4gMDMsIDIwMjQsIEphbWVzIEhvdWdodG9uIHdyb3RlOgo+IE9uIFRodSwgTWF5 IDMwLCAyMDI0IGF0IDExOjA24oCvUE0gWXUgWmhhbyA8eXV6aGFvQGdvb2dsZS5jb20+IHdyb3Rl Ogo+ID4gV2hhdCBJIGRvbid0IHRoaW5rIGlzIGFjY2VwdGFibGUgaXMgc2ltcGxpZnlpbmcgdGhv c2Ugb3B0aW1pemF0aW9ucwo+ID4gb3V0IHdpdGhvdXQgZG9jdW1lbnRpbmcgeW91ciBqdXN0aWZp Y2F0aW9ucyAoSSB3b3VsZCBldmVuIGNhbGwgaXQgYQo+ID4gZGVzaWduIGNoYW5nZSwgcmF0aGVy IHRoYW4gc2ltcGxpZmljYXRpb24sIGZyb20gdjMgdG8gdjQpLgo+IAo+IEknbGwgcHV0IGJhY2sg c29tZXRoaW5nIHNpbWlsYXIgdG8gd2hhdCB5b3UgaGFkIGJlZm9yZSAobGlrZSBhCj4gdGVzdF9j bGVhcl95b3VuZygpIHdpdGggYSAiZmFzdCIgcGFyYW1ldGVyIGluc3RlYWQgb2YgImJpdG1hcCIp LiBJCj4gbGlrZSB0aGUgaWRlYSBvZiBoYXZpbmcgYSBuZXcgbW11IG5vdGlmaWVyLCBsaWtlCj4g ZmFzdF90ZXN0X2NsZWFyX3lvdW5nKCksIHdoaWxlIGxlYXZpbmcgdGVzdF95b3VuZygpIGFuZCBj bGVhcl95b3VuZygpCj4gdW5jaGFuZ2VkICh3aGVyZSAiZmFzdCIgbWVhbnMgInByaW9yaXRpemUg c3BlZWQgb3ZlciBhY2N1cmFjeSIpLgoKVGhvc2UgdHdvIHN0YXRlbWVudHMgYXJlIGNvbnRyYWRp Y3RpbmcgZWFjaCBvdGhlciwgYXJlbid0IHRoZXk/ICBBbnl3YXlzLCBJIHZvdGUKZm9yIGEgImZh c3Qgb25seSIgdmFyaWFudCwgZS5nLiB0ZXN0X2NsZWFyX3lvdW5nX2Zhc3Rfb25seSgpIG9yIHNv LiAgZ3VwKCkgaGFzCmFscmVhZHkgZXN0YWJsaXNoZWQgdGhhdCB0ZXJtaW5vbG9neSBpbiBtbS8s IHNvIGhvcGVmdWxseSBpdCB3b3VsZCBiZSBmYW1pbGlhcgp0byByZWFkZXJzLiAgV2UgY291bGQg cGFzcyBhIHBhcmFtLCBidXQgdGhlbiB0aGUgTUdMUlUgY29kZSB3b3VsZCBsaWtlbHkgZW5kIHVw CmRvaW5nIGEgYnVuY2ggb2YgdXNlbGVzcyBpbmRpcmVjdCBjYWxscyBpbnRvIHNlY29uZGFyeSBN TVVzLCB3aGVyZWFzIGEgZGVkaWNhdGVkCmhvb2sgYWxsb3dzIGltcGxlbWVudGF0aW9ucyB0byBu dWxsaWZ5IHRoZSBwb2ludGVyIGlmIHRoZSBBUEkgaXNuJ3Qgc3VwcG9ydGVkCmZvciB3aGF0ZXZl ciByZWFzb24uCgpBbmQgcHVsbGluZyBpbiBPbGl2ZXIncyBjb21tZW50cyBhYm91dCBsb2NraW5n LCBJIHRoaW5rIGl0J3MgaW1wb3J0YW50IHRoYXQgdGhlCm1tdV9ub3RpZmllciBBUEkgZXhwcmVz cyBpdCdzIHJlcXVpcmVtZW50IHRoYXQgdGhlIG9wZXJhdGlvbiBiZSAiZmFzdCIsIG5vdCB0aGF0 Cml0IGJlIGxvY2tsZXNzLiAgRS5nLiBpZiBhIHNlY29uZGFyeSBNTVUgY2FuIGd1YXJhbnRlZSB0 aGF0IGEgbG9jayB3aWxsIGJlCmNvbnRlbnRlZCBvbmx5IGluIHJhcmUsIHNsb3cgY2FzZXMsIHRo ZW4gdGFraW5nIGEgbG9jayBpcyBhLW9rLiAgT3IgYSBzZWNvbmRhcnkKTU1VIGNvdWxkIGRvIHRy eS1sb2NrIGFuZCBiYWlsIGlmIHRoZSBsb2NrIGlzIGNvbnRlbmRlZC4KClRoYXQgd2F5IEtWTSBj YW4gaG9ub3IgdGhlIGludGVudCBvZiB0aGUgQVBJIHdpdGggYW4gaW1wbGVtZW50YXRpb24gdGhh dCB3b3JrcwpiZXN0IGZvciBLVk0gX2FuZF8gZm9yIE1HUkxVLiAgSSdtIHN1cmUgdGhlcmUgd2ls bCBiZSBmdXR1cmUgYWRqdXN0bWVudHMgYW5kIGZpeGVzLApidXQgdGhhdCdzIGp1c3QgbW9yZSBt b3RpdmF0aW9uIGZvciB1c2luZyBzb21ldGhpbmcgbGlrZSAiZmFzdCBvbmx5IiBpbnN0ZWFkIG9m CiJsb2NrbGVzcyIuCgo+ID4gPiBJIG1hZGUgdGhpcyBsb2dpYyBjaGFuZ2UgYXMgcGFydCBvZiBy ZW1vdmluZyBiYXRjaGluZy4KPiA+ID4KPiA+ID4gSSdkIHJlYWxseSBhcHByZWNpYXRlIGd1aWRh bmNlIG9uIHdoYXQgdGhlIGNvcnJlY3QgdGhpbmcgdG8gZG8gaXMuCj4gPiA+Cj4gPiA+IEluIG15 IG1pbmQsIHdoYXQgd291bGQgd29yayBncmVhdCBpczogYnkgZGVmYXVsdCwgZG8gYWdpbmcgZXhh Y3RseQo+ID4gPiB3aGVuIEtWTSBjYW4gZG8gaXQgbG9ja2xlc3NseSwgYW5kIHRoZW4gaGF2ZSBh IEtjb25maWcgdG8gYWx3YXlzIGhhdmUKPiA+ID4gTUdMUlUgdG8gZG8gYWdpbmcgd2l0aCBLVk0g aWYgYSB1c2VyIHJlYWxseSBjYXJlcyBhYm91dCBwcm9hY3RpdmUKPiA+ID4gcmVjbGFpbSAod2hl biB0aGUgZmVhdHVyZSBiaXQgaXMgc2V0KS4gVGhlIHNlbGZ0ZXN0IGNhbiBjaGVjayB0aGUKPiA+ ID4gS2NvbmZpZyArIGZlYXR1cmUgYml0IHRvIGtub3cgZm9yIHN1cmUgaWYgYWdpbmcgd2lsbCBi ZSBkb25lLgo+ID4KPiA+IEkgc3RpbGwgZG9uJ3Qgc2VlIGhvdyB0aGF0IEtjb25maWcgaGVscHMu IE9yIHdoeSB0aGUgbmV3IHN0YXRpYyBicmFuY2gKPiA+IGlzbid0IGVub3VnaD8KPiAKPiBXaXRo b3V0IGEgc3BlY2lhbCBLY29uZmlnLCB0aGUgZmVhdHVyZSBiaXQganVzdCB0ZWxscyB1cyB0aGF0 IGFnaW5nCj4gd2l0aCBLVk0gaXMgcG9zc2libGUsIG5vdCB0aGF0IGl0IHdpbGwgbmVjZXNzYXJp bHkgYmUgZG9uZS4gRm9yIHRoZQo+IHNlbGYtdGVzdCwgaXQnZCBiZSBnb29kIHRvIGtub3cgZXhh Y3RseSB3aGVuIGFnaW5nIGlzIGJlaW5nIGRvbmUgb3IKPiBub3QsIHNvIGhhdmluZyBhIEtjb25m aWcgbGlrZSBMUlVfR0VOX0FMV0FZU19XQUxLX1NFQ09OREFSWV9NTVUgd291bGQKPiBoZWxwIG1h a2UgdGhlIHNlbGYtdGVzdCBzZXQgdGhlIHJpZ2h0IGV4cGVjdGF0aW9ucyBmb3IgYWdpbmcuCj4g Cj4gVGhlIEtjb25maWcgd291bGQgYWxzbyBhbGxvdyBhIHVzZXIgdG8ga25vdyB0aGF0LCBubyBt YXR0ZXIgd2hhdCwKPiB3ZSdyZSBnb2luZyB0byBnZXQgY29ycmVjdCBhZ2UgZGF0YSBmb3IgVk1z LCBldmVuIGlmLCBzYXksIHdlJ3JlIHVzaW5nCj4gdGhlIHNoYWRvdyBNTVUuCgpIZWgsIHVubGVz cyBLVk0gZmx1c2hlcywgeW91IHdvbid0IGdldCAiY29ycmVjdCIgYWdlIGRhdGEuCgo+IFRoaXMg aXMgc29tZXdoYXQgaW1wb3J0YW50IGZvciBtZS9Hb29nbGUgQ2xvdWQuIElzIHRoYXQgcmVhc29u YWJsZT8gTWF5YmUKPiB0aGVyZSdzIGEgYmV0dGVyIHNvbHV0aW9uLgoKSG1tLCBubz8gIFRoZXJl J3Mgbm8gcmVhc29uIHRvIHVzZSBhIEtjb25maWcsIGUuZy4gaWYgd2UgX3JlYWxseV8gd2FudCB0 byBwcmlvcml0aXplCmFjY3VyYWN5IG92ZXIgc3BlZWQsIHRoZW4gYSBLVk0gKHg4Nj8pIG1vZHVs ZSBwYXJhbSB0byBoYXZlIEtWTSB3YWxrIG5lc3RlZCBURFAKcGFnZSB0YWJsZXMgd291bGQgZ2l2 ZSB1cyB3aGF0IHdlIHdhbnQuCgpCdXQgYmVmb3JlIHdlIGRvIHRoYXQsIEkgdGhpbmsgd2UgbmVl ZCB0byBwZXJmb3JtIGR1ZSBkaWxlZ2VuY2UgKG9yIHByb3ZpZGUgZGF0YSkKc2hvd2luZyB0aGF0 IGhhdmluZyBLVk0gdGFrZSBtbXVfbG9jayBmb3Igd3JpdGUgaW4gdGhlICJmYXN0IG9ubHkiIEFQ SSBwcm92aWRlcwpiZXR0ZXIgdG90YWwgYmVoYXZpb3IuICBJLmUuIHRoYXQgdGhlIGFkZGl0aW9u YWwgYWNjdXJhY3kgaXMgaW5kZWVkIHdvcnRoIHRoZSBjb3N0LgoKX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtcmlzY3YgbWFpbGluZyBsaXN0Cmxp bnV4LXJpc2N2QGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcv bWFpbG1hbi9saXN0aW5mby9saW51eC1yaXNjdgo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7F870C25B78 for ; Mon, 3 Jun 2024 23:03:59 +0000 (UTC) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20230601 header.b=VLQYa8B5; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4VtTms0nZFz3dBD for ; Tue, 4 Jun 2024 09:03:57 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20230601 header.b=VLQYa8B5; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=flex--seanjc.bounces.google.com (client-ip=2607:f8b0:4864:20::649; helo=mail-pl1-x649.google.com; envelope-from=3q0tezgykdpqoawjfyckkcha.ykihejqtlly-zarheopo.kvhwxo.knc@flex--seanjc.bounces.google.com; receiver=lists.ozlabs.org) Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4VtTlz6LXLz3cWv for ; Tue, 4 Jun 2024 09:03:10 +1000 (AEST) Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-1f6582eca2bso20013785ad.1 for ; Mon, 03 Jun 2024 16:03:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1717455788; x=1718060588; darn=lists.ozlabs.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=w0SJTxFtK/1G56YdHzcPYdHV7zjy1ZVa0L4hU78kSrI=; b=VLQYa8B5EOdp9sBbxFFkktIPgnOLpsJmTKOmyIipOauInO0xuQPLlqLlaciQHg4qba 6qGbRYMRVtV8N4kBKO1RFqC37a/E1rSnTH2TLwHiLtYFOPjuaItBLZpP0rLWamuWNaL+ CnKKDG2t89l7ifK3nAP/gJa1YeVanyRigjAlTBVjmy7ZFdOTcaSF6Oxg8p+LAkJBpXlH hZjwkwPQo509Ja9sbweq9BcR/MoIMvd8y27uR8Petz0V3hFDwC/FsDoBskB6KW935wTN cxHX1sfSu9mWP9rFxl3QCOVuOpdEs6y3VcxmuC7U3VsKlLOD3eyjtgN5oD85c2/siksK jitg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717455788; x=1718060588; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=w0SJTxFtK/1G56YdHzcPYdHV7zjy1ZVa0L4hU78kSrI=; b=ld3Lzpr+NqATNxNYOBX6qsSODKqde//U00Biyt+ZjFolYI6LsVQvG2mgdmmonVHhow KQ6FhHI3He0BrXRosu9poumFKzQ03beCWbDAi4tNvcHavkZYBcgqpF/O1x0uzVzCRefX OAaWav7c3KouXzstQLGMCT1/XE8p2euQevQgRro9AJDTfraIYjDtP9Ci+ed1oe9d5wIK n2CyT9Rkh1OyHxQM+/+iGok1Oicn375TKPieJapvbNeUF8QsY/cCp1DlFCJp2YRxU53K s8JDHrhkfl50sc4n60mbCZZeBNr/WpwmUCWPSxCcu47KU/HFdx60Dr6O6Mst39PqnYe7 6t9A== X-Forwarded-Encrypted: i=1; AJvYcCW4+Z3irfuZVKJXgY4RnyUlIZJHTFU879/0dYdqKBtgy30cosmVbHRUI7ovJfNg2h7GbqXoDo+KfKbAO1BDbJn+w1YlNNUMHBvO6dSNMg== X-Gm-Message-State: AOJu0Yy0INyeV29Ba2L8pEOOQXMfjUVo5EhDKy3+0EyVpiCwPL4q0mY4 Hb32yyLb6IrusbC/vNjbH7cep4n2oMilOrDy44DCiNPBDkfoQUuxhrzUEqssR81ovwVxr4Reoyw RGg== X-Google-Smtp-Source: AGHT+IH9puzU5V8aMTwgzoabUylaHv5IhJEL6Kgk0jFfHrRaPIZ6XCL4YZWlIMRWpWZxlsRoR1ZSqKiiwm0= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:ea05:b0:1f6:3891:794a with SMTP id d9443c01a7336-1f638917b67mr7110545ad.10.1717455787406; Mon, 03 Jun 2024 16:03:07 -0700 (PDT) Date: Mon, 3 Jun 2024 16:03:05 -0700 In-Reply-To: Mime-Version: 1.0 References: <20240529180510.2295118-1-jthoughton@google.com> <20240529180510.2295118-3-jthoughton@google.com> Message-ID: Subject: Re: [PATCH v4 2/7] mm: multi-gen LRU: Have secondary MMUs participate in aging From: Sean Christopherson To: James Houghton Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, linux-doc@vger.kernel.org, Catalin Marinas , Atish Patra , linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, linux-kselftest@vger.kernel.org, Raghavendra Rao Ananta , linux-riscv@lists.infradead.org, Shuah Khan , Yu Zhao , Jonathan Corbet , Anup Patel , Huacai Chen , David Rientjes , Zenghui Yu , Axel Rasmussen , linux-mips@vger.kernel.org, Albert Ou , Ryan Roberts , Will Deacon , Suzuki K Poulose , Shaoqin Huang , Nicholas Piggin , Bibo Mao , loongarch@lists.linux.dev, Paul Walmsley , David Matlack , Palmer Dabbelt , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, Ankit Agrawal , Oliver Upton , James Morse , kvm-riscv@lists.infradead.org, Marc Zyngier , Paolo Bonzini , Andrew Morton , Tianrui Zhao , linuxppc-dev@lists.ozlabs.org Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Mon, Jun 03, 2024, James Houghton wrote: > On Thu, May 30, 2024 at 11:06=E2=80=AFPM Yu Zhao wrot= e: > > What I don't think is acceptable is simplifying those optimizations > > out without documenting your justifications (I would even call it a > > design change, rather than simplification, from v3 to v4). >=20 > I'll put back something similar to what you had before (like a > test_clear_young() with a "fast" parameter instead of "bitmap"). I > like the idea of having a new mmu notifier, like > fast_test_clear_young(), while leaving test_young() and clear_young() > unchanged (where "fast" means "prioritize speed over accuracy"). Those two statements are contradicting each other, aren't they? Anyways, I= vote for a "fast only" variant, e.g. test_clear_young_fast_only() or so. gup() = has already established that terminology in mm/, so hopefully it would be famil= iar to readers. We could pass a param, but then the MGLRU code would likely en= d up doing a bunch of useless indirect calls into secondary MMUs, whereas a dedi= cated hook allows implementations to nullify the pointer if the API isn't support= ed for whatever reason. And pulling in Oliver's comments about locking, I think it's important that= the mmu_notifier API express it's requirement that the operation be "fast", not= that it be lockless. E.g. if a secondary MMU can guarantee that a lock will be contented only in rare, slow cases, then taking a lock is a-ok. Or a secon= dary MMU could do try-lock and bail if the lock is contended. That way KVM can honor the intent of the API with an implementation that wo= rks best for KVM _and_ for MGRLU. I'm sure there will be future adjustments an= d fixes, but that's just more motivation for using something like "fast only" instea= d of "lockless". > > > I made this logic change as part of removing batching. > > > > > > I'd really appreciate guidance on what the correct thing to do is. > > > > > > In my mind, what would work great is: by default, do aging exactly > > > when KVM can do it locklessly, and then have a Kconfig to always have > > > MGLRU to do aging with KVM if a user really cares about proactive > > > reclaim (when the feature bit is set). The selftest can check the > > > Kconfig + feature bit to know for sure if aging will be done. > > > > I still don't see how that Kconfig helps. Or why the new static branch > > isn't enough? >=20 > Without a special Kconfig, the feature bit just tells us that aging > with KVM is possible, not that it will necessarily be done. For the > self-test, it'd be good to know exactly when aging is being done or > not, so having a Kconfig like LRU_GEN_ALWAYS_WALK_SECONDARY_MMU would > help make the self-test set the right expectations for aging. >=20 > The Kconfig would also allow a user to know that, no matter what, > we're going to get correct age data for VMs, even if, say, we're using > the shadow MMU. Heh, unless KVM flushes, you won't get "correct" age data. > This is somewhat important for me/Google Cloud. Is that reasonable? Maybe > there's a better solution. Hmm, no? There's no reason to use a Kconfig, e.g. if we _really_ want to p= rioritize accuracy over speed, then a KVM (x86?) module param to have KVM walk nested= TDP page tables would give us what we want. But before we do that, I think we need to perform due dilegence (or provide= data) showing that having KVM take mmu_lock for write in the "fast only" API prov= ides better total behavior. I.e. that the additional accuracy is indeed worth t= he cost. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0EDB7C25B75 for ; Mon, 3 Jun 2024 23:03:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=7/Zm3Z5F9q8k3+pSsmhwxtfKSHu4Ctk4w7ioHytY4ig=; b=DlNo1yvB2Dxl/0YqZLmKoFtYJ3 5qYpAzybKbOz+nalBGjo2eoJlRQ7ZCI4DFEsoUZUbL8PsHw0DOSB0R7ZlcVY1zAjC+qcFYXHJG7Ah nH8ATWjv1ErGkRAlDkcs+7JwLyqFg90UzbjveEr/6UBZ1zrC/juix74CoUqJgkN6vAR+kV0gQ2agf Gm4FkbEa+rblNPz88IbmGL5a2AAToi65hJBBaI5gdIbNyoSsGSeKm2lywMgZszelePV9ECASupG1A nvKwfxUfqA7jZnk4/IaHJ/H2AfGhdgIEztKM1gCq3hAA0IZChcTJ4VGalxQRffN2JLf1y8pEUZiE3 n5fjz9ig==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEGi0-00000000Zi1-41S5; Mon, 03 Jun 2024 23:03:20 +0000 Received: from mail-pl1-x64a.google.com ([2607:f8b0:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sEGhw-00000000ZbX-0AQW for linux-arm-kernel@lists.infradead.org; Mon, 03 Jun 2024 23:03:19 +0000 Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-1f4f00cff60so25296625ad.0 for ; Mon, 03 Jun 2024 16:03:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1717455787; x=1718060587; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=w0SJTxFtK/1G56YdHzcPYdHV7zjy1ZVa0L4hU78kSrI=; b=IVSkrx2SVLgc6N/ZyTV8E9DWpjYxSsSk0dfu+0sxPTtR3B+Y9Myk1iB/OI1cNb6PjJ xPubOkfyVVR+qbicp2q/PDhTKXUa0cgSxpBAhvhe1Q/hctwm5vCXjkTsBGnrMzD1/ddp pHlsBEyXR331MV34eFEx5r/xZzhITwDmBUnQeiW3qUtdomJRx8PVLCH7upG6z7JvMkM6 NxJfl/C1wti4At/owVm3LPP80JZL3F3pHbvbGK4N/JOkn7rulphO0IwyF585V0SKdiIb Q/JidIvlkv320B+DdaLAhT+Scm/ECUzBDJnxhXQRZKMJrTvahAsiZvowCVA7uAJPEXvd RNuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717455788; x=1718060588; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=w0SJTxFtK/1G56YdHzcPYdHV7zjy1ZVa0L4hU78kSrI=; b=gC4T5U9Q8aHvSCCC+7f910RUTYWqF+zHg4NlB+E3+oL5h7SAkvUozcaFPfu8GR9Vfm XlmbCXkgva6hYGWYU+k3Qx0a7/gCk5Nq3+NMTJWqhUqCZdZ78bWzSYbFkFhgn7RtLjtv AOfMaFclWADeUka5o/+D5x8AyeY1wX3RStrRRyJ1d0MlY5zO3wH4NKFdIoh7jalHeGJV G9Kd29xV7K+nWx3n1lz+1bXxbU7BlEkixSrAGBLk7RLHi/5r0r8t11rhgQzOLw56E0tT fNHnO16Lx+6ZnnUCXoy9fttPI1tmWN6YPuhN1aj1ylflef4Ls7ubmAa3/7yISNYUOOA3 w0pw== X-Forwarded-Encrypted: i=1; AJvYcCVP2YRH8g3UTYBCpoc7mbv/4lHtnD787FapsbVMp/Jh5r06zwqZi6GxotYhCaURkMcn6cLc629p7mYIA9Uak8WejRSoaUw9ji28wSBiviDE5Iczqcs= X-Gm-Message-State: AOJu0Yy2phlvlOEu/qrr02RATsRJBnPvQ9S6o9cDtkf1tU7IpexoHG6v ZvgHPVSm9Vf+zKdmpWeETqHUQk6jeQoummjCOulDCQ9kofpIDXB0gdOk627VKe4DRM9og0DyoSy Wkw== X-Google-Smtp-Source: AGHT+IH9puzU5V8aMTwgzoabUylaHv5IhJEL6Kgk0jFfHrRaPIZ6XCL4YZWlIMRWpWZxlsRoR1ZSqKiiwm0= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:902:ea05:b0:1f6:3891:794a with SMTP id d9443c01a7336-1f638917b67mr7110545ad.10.1717455787406; Mon, 03 Jun 2024 16:03:07 -0700 (PDT) Date: Mon, 3 Jun 2024 16:03:05 -0700 In-Reply-To: Mime-Version: 1.0 References: <20240529180510.2295118-1-jthoughton@google.com> <20240529180510.2295118-3-jthoughton@google.com> Message-ID: Subject: Re: [PATCH v4 2/7] mm: multi-gen LRU: Have secondary MMUs participate in aging From: Sean Christopherson To: James Houghton Cc: Yu Zhao , Andrew Morton , Paolo Bonzini , Albert Ou , Ankit Agrawal , Anup Patel , Atish Patra , Axel Rasmussen , Bibo Mao , Catalin Marinas , David Matlack , David Rientjes , Huacai Chen , James Morse , Jonathan Corbet , Marc Zyngier , Michael Ellerman , Nicholas Piggin , Oliver Upton , Palmer Dabbelt , Paul Walmsley , Raghavendra Rao Ananta , Ryan Roberts , Shaoqin Huang , Shuah Khan , Suzuki K Poulose , Tianrui Zhao , Will Deacon , Zenghui Yu , kvm-riscv@lists.infradead.org, kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240603_160316_107499_751A177C X-CRM114-Status: GOOD ( 33.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gTW9uLCBKdW4gMDMsIDIwMjQsIEphbWVzIEhvdWdodG9uIHdyb3RlOgo+IE9uIFRodSwgTWF5 IDMwLCAyMDI0IGF0IDExOjA24oCvUE0gWXUgWmhhbyA8eXV6aGFvQGdvb2dsZS5jb20+IHdyb3Rl Ogo+ID4gV2hhdCBJIGRvbid0IHRoaW5rIGlzIGFjY2VwdGFibGUgaXMgc2ltcGxpZnlpbmcgdGhv c2Ugb3B0aW1pemF0aW9ucwo+ID4gb3V0IHdpdGhvdXQgZG9jdW1lbnRpbmcgeW91ciBqdXN0aWZp Y2F0aW9ucyAoSSB3b3VsZCBldmVuIGNhbGwgaXQgYQo+ID4gZGVzaWduIGNoYW5nZSwgcmF0aGVy IHRoYW4gc2ltcGxpZmljYXRpb24sIGZyb20gdjMgdG8gdjQpLgo+IAo+IEknbGwgcHV0IGJhY2sg c29tZXRoaW5nIHNpbWlsYXIgdG8gd2hhdCB5b3UgaGFkIGJlZm9yZSAobGlrZSBhCj4gdGVzdF9j bGVhcl95b3VuZygpIHdpdGggYSAiZmFzdCIgcGFyYW1ldGVyIGluc3RlYWQgb2YgImJpdG1hcCIp LiBJCj4gbGlrZSB0aGUgaWRlYSBvZiBoYXZpbmcgYSBuZXcgbW11IG5vdGlmaWVyLCBsaWtlCj4g ZmFzdF90ZXN0X2NsZWFyX3lvdW5nKCksIHdoaWxlIGxlYXZpbmcgdGVzdF95b3VuZygpIGFuZCBj bGVhcl95b3VuZygpCj4gdW5jaGFuZ2VkICh3aGVyZSAiZmFzdCIgbWVhbnMgInByaW9yaXRpemUg c3BlZWQgb3ZlciBhY2N1cmFjeSIpLgoKVGhvc2UgdHdvIHN0YXRlbWVudHMgYXJlIGNvbnRyYWRp Y3RpbmcgZWFjaCBvdGhlciwgYXJlbid0IHRoZXk/ICBBbnl3YXlzLCBJIHZvdGUKZm9yIGEgImZh c3Qgb25seSIgdmFyaWFudCwgZS5nLiB0ZXN0X2NsZWFyX3lvdW5nX2Zhc3Rfb25seSgpIG9yIHNv LiAgZ3VwKCkgaGFzCmFscmVhZHkgZXN0YWJsaXNoZWQgdGhhdCB0ZXJtaW5vbG9neSBpbiBtbS8s IHNvIGhvcGVmdWxseSBpdCB3b3VsZCBiZSBmYW1pbGlhcgp0byByZWFkZXJzLiAgV2UgY291bGQg cGFzcyBhIHBhcmFtLCBidXQgdGhlbiB0aGUgTUdMUlUgY29kZSB3b3VsZCBsaWtlbHkgZW5kIHVw CmRvaW5nIGEgYnVuY2ggb2YgdXNlbGVzcyBpbmRpcmVjdCBjYWxscyBpbnRvIHNlY29uZGFyeSBN TVVzLCB3aGVyZWFzIGEgZGVkaWNhdGVkCmhvb2sgYWxsb3dzIGltcGxlbWVudGF0aW9ucyB0byBu dWxsaWZ5IHRoZSBwb2ludGVyIGlmIHRoZSBBUEkgaXNuJ3Qgc3VwcG9ydGVkCmZvciB3aGF0ZXZl ciByZWFzb24uCgpBbmQgcHVsbGluZyBpbiBPbGl2ZXIncyBjb21tZW50cyBhYm91dCBsb2NraW5n LCBJIHRoaW5rIGl0J3MgaW1wb3J0YW50IHRoYXQgdGhlCm1tdV9ub3RpZmllciBBUEkgZXhwcmVz cyBpdCdzIHJlcXVpcmVtZW50IHRoYXQgdGhlIG9wZXJhdGlvbiBiZSAiZmFzdCIsIG5vdCB0aGF0 Cml0IGJlIGxvY2tsZXNzLiAgRS5nLiBpZiBhIHNlY29uZGFyeSBNTVUgY2FuIGd1YXJhbnRlZSB0 aGF0IGEgbG9jayB3aWxsIGJlCmNvbnRlbnRlZCBvbmx5IGluIHJhcmUsIHNsb3cgY2FzZXMsIHRo ZW4gdGFraW5nIGEgbG9jayBpcyBhLW9rLiAgT3IgYSBzZWNvbmRhcnkKTU1VIGNvdWxkIGRvIHRy eS1sb2NrIGFuZCBiYWlsIGlmIHRoZSBsb2NrIGlzIGNvbnRlbmRlZC4KClRoYXQgd2F5IEtWTSBj YW4gaG9ub3IgdGhlIGludGVudCBvZiB0aGUgQVBJIHdpdGggYW4gaW1wbGVtZW50YXRpb24gdGhh dCB3b3JrcwpiZXN0IGZvciBLVk0gX2FuZF8gZm9yIE1HUkxVLiAgSSdtIHN1cmUgdGhlcmUgd2ls bCBiZSBmdXR1cmUgYWRqdXN0bWVudHMgYW5kIGZpeGVzLApidXQgdGhhdCdzIGp1c3QgbW9yZSBt b3RpdmF0aW9uIGZvciB1c2luZyBzb21ldGhpbmcgbGlrZSAiZmFzdCBvbmx5IiBpbnN0ZWFkIG9m CiJsb2NrbGVzcyIuCgo+ID4gPiBJIG1hZGUgdGhpcyBsb2dpYyBjaGFuZ2UgYXMgcGFydCBvZiBy ZW1vdmluZyBiYXRjaGluZy4KPiA+ID4KPiA+ID4gSSdkIHJlYWxseSBhcHByZWNpYXRlIGd1aWRh bmNlIG9uIHdoYXQgdGhlIGNvcnJlY3QgdGhpbmcgdG8gZG8gaXMuCj4gPiA+Cj4gPiA+IEluIG15 IG1pbmQsIHdoYXQgd291bGQgd29yayBncmVhdCBpczogYnkgZGVmYXVsdCwgZG8gYWdpbmcgZXhh Y3RseQo+ID4gPiB3aGVuIEtWTSBjYW4gZG8gaXQgbG9ja2xlc3NseSwgYW5kIHRoZW4gaGF2ZSBh IEtjb25maWcgdG8gYWx3YXlzIGhhdmUKPiA+ID4gTUdMUlUgdG8gZG8gYWdpbmcgd2l0aCBLVk0g aWYgYSB1c2VyIHJlYWxseSBjYXJlcyBhYm91dCBwcm9hY3RpdmUKPiA+ID4gcmVjbGFpbSAod2hl biB0aGUgZmVhdHVyZSBiaXQgaXMgc2V0KS4gVGhlIHNlbGZ0ZXN0IGNhbiBjaGVjayB0aGUKPiA+ ID4gS2NvbmZpZyArIGZlYXR1cmUgYml0IHRvIGtub3cgZm9yIHN1cmUgaWYgYWdpbmcgd2lsbCBi ZSBkb25lLgo+ID4KPiA+IEkgc3RpbGwgZG9uJ3Qgc2VlIGhvdyB0aGF0IEtjb25maWcgaGVscHMu IE9yIHdoeSB0aGUgbmV3IHN0YXRpYyBicmFuY2gKPiA+IGlzbid0IGVub3VnaD8KPiAKPiBXaXRo b3V0IGEgc3BlY2lhbCBLY29uZmlnLCB0aGUgZmVhdHVyZSBiaXQganVzdCB0ZWxscyB1cyB0aGF0 IGFnaW5nCj4gd2l0aCBLVk0gaXMgcG9zc2libGUsIG5vdCB0aGF0IGl0IHdpbGwgbmVjZXNzYXJp bHkgYmUgZG9uZS4gRm9yIHRoZQo+IHNlbGYtdGVzdCwgaXQnZCBiZSBnb29kIHRvIGtub3cgZXhh Y3RseSB3aGVuIGFnaW5nIGlzIGJlaW5nIGRvbmUgb3IKPiBub3QsIHNvIGhhdmluZyBhIEtjb25m aWcgbGlrZSBMUlVfR0VOX0FMV0FZU19XQUxLX1NFQ09OREFSWV9NTVUgd291bGQKPiBoZWxwIG1h a2UgdGhlIHNlbGYtdGVzdCBzZXQgdGhlIHJpZ2h0IGV4cGVjdGF0aW9ucyBmb3IgYWdpbmcuCj4g Cj4gVGhlIEtjb25maWcgd291bGQgYWxzbyBhbGxvdyBhIHVzZXIgdG8ga25vdyB0aGF0LCBubyBt YXR0ZXIgd2hhdCwKPiB3ZSdyZSBnb2luZyB0byBnZXQgY29ycmVjdCBhZ2UgZGF0YSBmb3IgVk1z LCBldmVuIGlmLCBzYXksIHdlJ3JlIHVzaW5nCj4gdGhlIHNoYWRvdyBNTVUuCgpIZWgsIHVubGVz cyBLVk0gZmx1c2hlcywgeW91IHdvbid0IGdldCAiY29ycmVjdCIgYWdlIGRhdGEuCgo+IFRoaXMg aXMgc29tZXdoYXQgaW1wb3J0YW50IGZvciBtZS9Hb29nbGUgQ2xvdWQuIElzIHRoYXQgcmVhc29u YWJsZT8gTWF5YmUKPiB0aGVyZSdzIGEgYmV0dGVyIHNvbHV0aW9uLgoKSG1tLCBubz8gIFRoZXJl J3Mgbm8gcmVhc29uIHRvIHVzZSBhIEtjb25maWcsIGUuZy4gaWYgd2UgX3JlYWxseV8gd2FudCB0 byBwcmlvcml0aXplCmFjY3VyYWN5IG92ZXIgc3BlZWQsIHRoZW4gYSBLVk0gKHg4Nj8pIG1vZHVs ZSBwYXJhbSB0byBoYXZlIEtWTSB3YWxrIG5lc3RlZCBURFAKcGFnZSB0YWJsZXMgd291bGQgZ2l2 ZSB1cyB3aGF0IHdlIHdhbnQuCgpCdXQgYmVmb3JlIHdlIGRvIHRoYXQsIEkgdGhpbmsgd2UgbmVl ZCB0byBwZXJmb3JtIGR1ZSBkaWxlZ2VuY2UgKG9yIHByb3ZpZGUgZGF0YSkKc2hvd2luZyB0aGF0 IGhhdmluZyBLVk0gdGFrZSBtbXVfbG9jayBmb3Igd3JpdGUgaW4gdGhlICJmYXN0IG9ubHkiIEFQ SSBwcm92aWRlcwpiZXR0ZXIgdG90YWwgYmVoYXZpb3IuICBJLmUuIHRoYXQgdGhlIGFkZGl0aW9u YWwgYWNjdXJhY3kgaXMgaW5kZWVkIHdvcnRoIHRoZSBjb3N0LgoKX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxp c3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZy YWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo=