From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sean Christopherson Date: Wed, 29 May 2024 15:58:02 -0700 Subject: [PATCH v4 2/7] mm: multi-gen LRU: Have secondary MMUs participate in aging In-Reply-To: References: <20240529180510.2295118-1-jthoughton@google.com> <20240529180510.2295118-3-jthoughton@google.com> Message-ID: List-Id: To: kvm-riscv@lists.infradead.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit On Wed, May 29, 2024, Yu Zhao wrote: > On Wed, May 29, 2024 at 3:59?PM Sean Christopherson wrote: > > > > On Wed, May 29, 2024, Yu Zhao wrote: > > > On Wed, May 29, 2024 at 12:05?PM James Houghton wrote: > > > > > > > > Secondary MMUs are currently consulted for access/age information at > > > > eviction time, but before then, we don't get accurate age information. > > > > That is, pages that are mostly accessed through a secondary MMU (like > > > > guest memory, used by KVM) will always just proceed down to the oldest > > > > generation, and then at eviction time, if KVM reports the page to be > > > > young, the page will be activated/promoted back to the youngest > > > > generation. > > > > > > Correct, and as I explained offline, this is the only reasonable > > > behavior if we can't locklessly walk secondary MMUs. > > > > > > Just for the record, the (crude) analogy I used was: > > > Imagine a large room with many bills ($1, $5, $10, ...) on the floor, > > > but you are only allowed to pick up 10 of them (and put them in your > > > pocket). A smart move would be to survey the room *first and then* > > > pick up the largest ones. But if you are carrying a 500 lbs backpack, > > > you would just want to pick up whichever that's in front of you rather > > > than walk the entire room. > > > > > > MGLRU should only scan (or lookaround) secondary MMUs if it can be > > > done lockless. Otherwise, it should just fall back to the existing > > > approach, which existed in previous versions but is removed in this > > > version. > > > > IIUC, by "existing approach" you mean completely ignore secondary MMUs that > > don't implement a lockless walk? > > No, the existing approach only checks secondary MMUs for LRU folios, > i.e., those at the end of the LRU list. It might not find the best > candidates (the coldest ones) on the entire list, but it doesn't pay > as much for the locking. MGLRU can *optionally* scan MMUs (secondary > included) to find the best candidates, but it can only be a win if the > scanning incurs a relatively low overhead, e.g., done locklessly for > the secondary MMU. IOW, this is a balance between the cost of > reclaiming not-so-cold (warm) folios and that of finding the coldest > folios. Gotcha. I tend to agree with Yu, driving the behavior via a Kconfig may generate simpler _code_, but I think it increases the overall system complexity. E.g. distros will likely enable the Kconfig, and in my experience people using KVM with a distro kernel usually aren't kernel experts, i.e. likely won't know that there's even a decision to be made, let alone be able to make an informed decision. Having an mmu_notifier hook that is conditionally implemented doesn't seem overly complex, e.g. even if there's a runtime aspect at play, it'd be easy enough for KVM to nullify its mmu_notifier hook during initialization. The hardest part is likely going to be figuring out the threshold for how much overhead is too much. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 89E7F1CB32C for ; Wed, 29 May 2024 22:58:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717023487; cv=none; b=Np0AFgGFczinoBy3/iTUidU2pq+3HbetSreOQeoojJMRD6xXGYqBJoS41YGcNGwRloQn8QQ3H/iq2x7PXzZYuC1baIV1oG6FVgcMm2DxuzJssqReSDPdmolvv7GBrn8QmR9OsmIix9/rsYDzqyS0ktzWSizSpmK9OOumERFEjqU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717023487; c=relaxed/simple; bh=ILTrqYj7gGL+uJmOCLBAjzct0EhWj0aN1ME6oQovXBA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gaaZEoZLWfSbzT1OOcEEQ3gmfH0GmxnPbOZIRhSHFlWW7yQulLAYGaHWXjMvYIhFZyhnK2NwGj4vxbilCbNUBACqkj+j7NkfFFa3xX5BksY8djiKwbzmlnRRV8u6NU/ByypGkqHqvT/IiJ9AwEUW2OAKYsRmfRiT12+3rJ5ClGY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=kku/jNJJ; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="kku/jNJJ" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-df4dc93d0d3so386221276.2 for ; Wed, 29 May 2024 15:58:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1717023484; x=1717628284; darn=lists.linux.dev; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=lXQ4aAmXpFR+Rnn+LVSp/P9w+zocv3IAjl6jcVlXHAQ=; b=kku/jNJJ3MJ/wgXHujjn9udBjgTokp8ar0bLJgXGLwcVSxoo5nLFZGuxufsFlAjsxa kscfOs3eMZR8Zf7qOavtr/o2/N84yESsptPFu1W3JE4JzggoAY+6lIAhvPQJmGoAlN9x vlrZTZLGUydgpgVAQJHLpmbsNRu5yTLe3m+cT5gm1SZ/1ZLnlV53ZqnRoo9G9VNDlDaq VgobUi8B8Wwz2LDOyoXImXt65BOQeBFV6eeTlFHO3c3mYf97t3oQ3dtWwKK8kXnKtt+H inDak7dQHWCVxhiiOyaDPUBe/JeYFgIZrW/GB9Jup5/ZhexcYxj3PyfkqY4pUwHntbMT 2w7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717023484; x=1717628284; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=lXQ4aAmXpFR+Rnn+LVSp/P9w+zocv3IAjl6jcVlXHAQ=; b=m2o8j2N57J90TKEQFxN2UGRA7DwJGj+IPYh1Tvv2DzYvxYQ2sb+Z+6ANveRVk9M1Ix +f1KcX4Z5vTpQSpHir3Ez0jo3ZjHi7Gyx7A7LqDal5o+upq5Z9/xIN59y4Q6c9WGoED3 BFuYCCF8gwR7Ky4bpUQfMXbFTM49mrUB4i6XBTVlZMfO4e8hufk50JUfvmpgEFlEI+8Z oIfVRykA3qDET7CTIYOrcqYiijrldHOGtJFo+9/vaP3L12sHKy/lWvNJGvSPj/ewGvyo /6ShczsLsvuV0Gcak0rr8E4SOdt/xBjZaHVN12VHX4BdHmlgo95/Q7ZmmARWsONVhGfu 72Qg== X-Forwarded-Encrypted: i=1; AJvYcCXpk5U6GgV8tKzpi1JpfnTqQ7A/JrRNAWr8JcADYbozF+MXV1Nir2jQ7YkDsyAqoxsDtj/WJiA7W091xmYkvA7Dhr2aOdd2 X-Gm-Message-State: AOJu0YxcF7xIjIXL//YpOYOzR7TVuXpr+6iZevWL9F5quf9EBqZJXpOH RS4Q1K3X8072l0h1WCUOD2zlhpWhfzO9pPILy2RR2Td0P55zZ4+ZQBWgH+dJ7yUkK+Qww5G86Eh +jg== X-Google-Smtp-Source: AGHT+IEcoxWAWGQljmQC1ObvPASoASZSWeSUzQr2Szq7dgxCnuAYyloQCwOCyj3B7twEp9K1BnNOHbqA4OU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:1007:b0:dfa:5838:b919 with SMTP id 3f1490d57ef6-dfa5a68828dmr142777276.10.1717023484293; Wed, 29 May 2024 15:58:04 -0700 (PDT) Date: Wed, 29 May 2024 15:58:02 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240529180510.2295118-1-jthoughton@google.com> <20240529180510.2295118-3-jthoughton@google.com> Message-ID: Subject: Re: [PATCH v4 2/7] mm: multi-gen LRU: Have secondary MMUs participate in aging From: Sean Christopherson To: Yu Zhao Cc: James Houghton , Andrew Morton , Paolo Bonzini , Albert Ou , Ankit Agrawal , Anup Patel , Atish Patra , Axel Rasmussen , Bibo Mao , Catalin Marinas , David Matlack , David Rientjes , Huacai Chen , James Morse , Jonathan Corbet , Marc Zyngier , Michael Ellerman , Nicholas Piggin , Oliver Upton , Palmer Dabbelt , Paul Walmsley , Raghavendra Rao Ananta , Ryan Roberts , Shaoqin Huang , Shuah Khan , Suzuki K Poulose , Tianrui Zhao , Will Deacon , Zenghui Yu , kvm-riscv@lists.infradead.org, kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable On Wed, May 29, 2024, Yu Zhao wrote: > On Wed, May 29, 2024 at 3:59=E2=80=AFPM Sean Christopherson wrote: > > > > On Wed, May 29, 2024, Yu Zhao wrote: > > > On Wed, May 29, 2024 at 12:05=E2=80=AFPM James Houghton wrote: > > > > > > > > Secondary MMUs are currently consulted for access/age information a= t > > > > eviction time, but before then, we don't get accurate age informati= on. > > > > That is, pages that are mostly accessed through a secondary MMU (li= ke > > > > guest memory, used by KVM) will always just proceed down to the old= est > > > > generation, and then at eviction time, if KVM reports the page to b= e > > > > young, the page will be activated/promoted back to the youngest > > > > generation. > > > > > > Correct, and as I explained offline, this is the only reasonable > > > behavior if we can't locklessly walk secondary MMUs. > > > > > > Just for the record, the (crude) analogy I used was: > > > Imagine a large room with many bills ($1, $5, $10, ...) on the floor, > > > but you are only allowed to pick up 10 of them (and put them in your > > > pocket). A smart move would be to survey the room *first and then* > > > pick up the largest ones. But if you are carrying a 500 lbs backpack, > > > you would just want to pick up whichever that's in front of you rathe= r > > > than walk the entire room. > > > > > > MGLRU should only scan (or lookaround) secondary MMUs if it can be > > > done lockless. Otherwise, it should just fall back to the existing > > > approach, which existed in previous versions but is removed in this > > > version. > > > > IIUC, by "existing approach" you mean completely ignore secondary MMUs = that > > don't implement a lockless walk? >=20 > No, the existing approach only checks secondary MMUs for LRU folios, > i.e., those at the end of the LRU list. It might not find the best > candidates (the coldest ones) on the entire list, but it doesn't pay > as much for the locking. MGLRU can *optionally* scan MMUs (secondary > included) to find the best candidates, but it can only be a win if the > scanning incurs a relatively low overhead, e.g., done locklessly for > the secondary MMU. IOW, this is a balance between the cost of > reclaiming not-so-cold (warm) folios and that of finding the coldest > folios. Gotcha. I tend to agree with Yu, driving the behavior via a Kconfig may generate si= mpler _code_, but I think it increases the overall system complexity. E.g. distr= os will likely enable the Kconfig, and in my experience people using KVM with = a distro kernel usually aren't kernel experts, i.e. likely won't know that th= ere's even a decision to be made, let alone be able to make an informed decision. Having an mmu_notifier hook that is conditionally implemented doesn't seem = overly complex, e.g. even if there's a runtime aspect at play, it'd be easy enough= for KVM to nullify its mmu_notifier hook during initialization. The hardest pa= rt is likely going to be figuring out the threshold for how much overhead is too = much. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A23FCC25B7E for ; Wed, 29 May 2024 22:58:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=t4aNwr9pJmQQ1QlM9i6W7K6imXYLXQRGmhDkj3ZXK7o=; b=a3O5vsJsmmgt0S9DL7b7RofMqk A+uA0kqsMXu6vgH04UYE8C9bS9U8Jirr9WHHv7EUwZ4UHsieSldXCPUd/XMbOCYy+wsFRNMTOYPrE 0CmYy/orykCw7m+G9f6osGpypuGX2TSdHwQQyKGF0IXcB5mKHLQPYjFcWuIQuv4Q+Y78XeY33Z5kU /joDEeuugQ0Y+2VVWjg8CcWA28RR0ifTIGrIEBPjdjS+ks2ZazxzBHcd8FufT+I8lIMiUyP9DSjU3 rSNrG+O+ZT0Xon4e7TNo1JPHd7scKWaeobd++PckcuacnxM1ReheJi6x9RdqMn/qylUL4EhfJ/Wvx uI10dBrQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCSFQ-00000005lkv-35x6; Wed, 29 May 2024 22:58:20 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCSFM-00000005lem-01H1 for linux-riscv@lists.infradead.org; Wed, 29 May 2024 22:58:17 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-627f20cff42so3832537b3.0 for ; Wed, 29 May 2024 15:58:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1717023484; x=1717628284; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=lXQ4aAmXpFR+Rnn+LVSp/P9w+zocv3IAjl6jcVlXHAQ=; b=X55ojBrsRsz7qlX450m3rkuGFgaMOJjsY//QOFxltNZR8hM4CP92Bz94zoTZiKkUsk I5uT/Ppeh01kLIeXe0QHT+uhrGnZ5rqq8nslSTm/vad/hQ0VJel7S+d4bfE9iubJm4RV 2+fD3Qa3AZzSmlxKpc8Eu5Uc3MmkSF0TbhydP4PzDODqE/w2cfmBzHc6Tp1IkWRwe/I2 gTgvt1d4mxWoEqQGWX7N4xRxN2WTEAHZPOw4IPclDVntk2ih/atCKOYfFj8PxgO210Vw JN8qh7bqM1u/dt3rHjxVgFS7W4zxPD9EVFteADgyrXRWyGEO02nOYPOQ2djtWX/hCiof RSqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717023484; x=1717628284; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=lXQ4aAmXpFR+Rnn+LVSp/P9w+zocv3IAjl6jcVlXHAQ=; b=gfs17MHEibpD+qkUzCWMb/6AM+8XZFrgFyueZCZ9U0H/ELPOSzSsUJ9xoqHvhI5uOp Cioe7+jzaEvWrKhdaNZijlTrfrW2Lco5e4++KQehenobWRGk2jO3SwJvxJEDBdc20jJ8 3naCvwA4V+/jMlPWfg7OT98rSfdslQ5Y9l3Dz56Z9BR/BIDPy/ltBcMmOH6u1BZY351A u7uOaUjVjDg0K9P24N7JdbO+HlvYPh4Ss/4ohAJsEKjsWTqrt5jbn16mDsGFtUBOusa8 wElwwy2eJFnXBEEFjdfvemTVp8W6vDzln3elmDnHlWzwOxXXYZ/bCGFHHswPUrOumZv9 hagA== X-Forwarded-Encrypted: i=1; AJvYcCWVhASjrb4fdAcPRwaJkuqQkaciqHnjVzuKHCqWFZ4PAbcx53BBSeP5Irop+3JdYhlHkSu+FTnwgd0tUek3zXeu+mAEoo0V3YqWCtS4YSEs X-Gm-Message-State: AOJu0YxWNf5rx9Ti/BMXzTPpPBWcg67SZ9kJU/IQOioo5l6ufqMG7d/p daHQzuX2OhrLIJt2LDh5jsCQHTG4V8xIEWQ2nlo/NtWOtVnSPLVA7PijZBC+RBhfLgLKGM1B6Xb 8aw== X-Google-Smtp-Source: AGHT+IEcoxWAWGQljmQC1ObvPASoASZSWeSUzQr2Szq7dgxCnuAYyloQCwOCyj3B7twEp9K1BnNOHbqA4OU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:1007:b0:dfa:5838:b919 with SMTP id 3f1490d57ef6-dfa5a68828dmr142777276.10.1717023484293; Wed, 29 May 2024 15:58:04 -0700 (PDT) Date: Wed, 29 May 2024 15:58:02 -0700 In-Reply-To: Mime-Version: 1.0 References: <20240529180510.2295118-1-jthoughton@google.com> <20240529180510.2295118-3-jthoughton@google.com> Message-ID: Subject: Re: [PATCH v4 2/7] mm: multi-gen LRU: Have secondary MMUs participate in aging From: Sean Christopherson To: Yu Zhao Cc: James Houghton , Andrew Morton , Paolo Bonzini , Albert Ou , Ankit Agrawal , Anup Patel , Atish Patra , Axel Rasmussen , Bibo Mao , Catalin Marinas , David Matlack , David Rientjes , Huacai Chen , James Morse , Jonathan Corbet , Marc Zyngier , Michael Ellerman , Nicholas Piggin , Oliver Upton , Palmer Dabbelt , Paul Walmsley , Raghavendra Rao Ananta , Ryan Roberts , Shaoqin Huang , Shuah Khan , Suzuki K Poulose , Tianrui Zhao , Will Deacon , Zenghui Yu , kvm-riscv@lists.infradead.org, kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_155816_055574_0CD60D91 X-CRM114-Status: GOOD ( 28.61 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org T24gV2VkLCBNYXkgMjksIDIwMjQsIFl1IFpoYW8gd3JvdGU6Cj4gT24gV2VkLCBNYXkgMjksIDIw MjQgYXQgMzo1OeKAr1BNIFNlYW4gQ2hyaXN0b3BoZXJzb24gPHNlYW5qY0Bnb29nbGUuY29tPiB3 cm90ZToKPiA+Cj4gPiBPbiBXZWQsIE1heSAyOSwgMjAyNCwgWXUgWmhhbyB3cm90ZToKPiA+ID4g T24gV2VkLCBNYXkgMjksIDIwMjQgYXQgMTI6MDXigK9QTSBKYW1lcyBIb3VnaHRvbiA8anRob3Vn aHRvbkBnb29nbGUuY29tPiB3cm90ZToKPiA+ID4gPgo+ID4gPiA+IFNlY29uZGFyeSBNTVVzIGFy ZSBjdXJyZW50bHkgY29uc3VsdGVkIGZvciBhY2Nlc3MvYWdlIGluZm9ybWF0aW9uIGF0Cj4gPiA+ ID4gZXZpY3Rpb24gdGltZSwgYnV0IGJlZm9yZSB0aGVuLCB3ZSBkb24ndCBnZXQgYWNjdXJhdGUg YWdlIGluZm9ybWF0aW9uLgo+ID4gPiA+IFRoYXQgaXMsIHBhZ2VzIHRoYXQgYXJlIG1vc3RseSBh Y2Nlc3NlZCB0aHJvdWdoIGEgc2Vjb25kYXJ5IE1NVSAobGlrZQo+ID4gPiA+IGd1ZXN0IG1lbW9y eSwgdXNlZCBieSBLVk0pIHdpbGwgYWx3YXlzIGp1c3QgcHJvY2VlZCBkb3duIHRvIHRoZSBvbGRl c3QKPiA+ID4gPiBnZW5lcmF0aW9uLCBhbmQgdGhlbiBhdCBldmljdGlvbiB0aW1lLCBpZiBLVk0g cmVwb3J0cyB0aGUgcGFnZSB0byBiZQo+ID4gPiA+IHlvdW5nLCB0aGUgcGFnZSB3aWxsIGJlIGFj dGl2YXRlZC9wcm9tb3RlZCBiYWNrIHRvIHRoZSB5b3VuZ2VzdAo+ID4gPiA+IGdlbmVyYXRpb24u Cj4gPiA+Cj4gPiA+IENvcnJlY3QsIGFuZCBhcyBJIGV4cGxhaW5lZCBvZmZsaW5lLCB0aGlzIGlz IHRoZSBvbmx5IHJlYXNvbmFibGUKPiA+ID4gYmVoYXZpb3IgaWYgd2UgY2FuJ3QgbG9ja2xlc3Ns eSB3YWxrIHNlY29uZGFyeSBNTVVzLgo+ID4gPgo+ID4gPiBKdXN0IGZvciB0aGUgcmVjb3JkLCB0 aGUgKGNydWRlKSBhbmFsb2d5IEkgdXNlZCB3YXM6Cj4gPiA+IEltYWdpbmUgYSBsYXJnZSByb29t IHdpdGggbWFueSBiaWxscyAoJDEsICQ1LCAkMTAsIC4uLikgb24gdGhlIGZsb29yLAo+ID4gPiBi dXQgeW91IGFyZSBvbmx5IGFsbG93ZWQgdG8gcGljayB1cCAxMCBvZiB0aGVtIChhbmQgcHV0IHRo ZW0gaW4geW91cgo+ID4gPiBwb2NrZXQpLiBBIHNtYXJ0IG1vdmUgd291bGQgYmUgdG8gc3VydmV5 IHRoZSByb29tICpmaXJzdCBhbmQgdGhlbioKPiA+ID4gcGljayB1cCB0aGUgbGFyZ2VzdCBvbmVz LiBCdXQgaWYgeW91IGFyZSBjYXJyeWluZyBhIDUwMCBsYnMgYmFja3BhY2ssCj4gPiA+IHlvdSB3 b3VsZCBqdXN0IHdhbnQgdG8gcGljayB1cCB3aGljaGV2ZXIgdGhhdCdzIGluIGZyb250IG9mIHlv dSByYXRoZXIKPiA+ID4gdGhhbiB3YWxrIHRoZSBlbnRpcmUgcm9vbS4KPiA+ID4KPiA+ID4gTUdM UlUgc2hvdWxkIG9ubHkgc2NhbiAob3IgbG9va2Fyb3VuZCkgc2Vjb25kYXJ5IE1NVXMgaWYgaXQg Y2FuIGJlCj4gPiA+IGRvbmUgbG9ja2xlc3MuIE90aGVyd2lzZSwgaXQgc2hvdWxkIGp1c3QgZmFs bCBiYWNrIHRvIHRoZSBleGlzdGluZwo+ID4gPiBhcHByb2FjaCwgd2hpY2ggZXhpc3RlZCBpbiBw cmV2aW91cyB2ZXJzaW9ucyBidXQgaXMgcmVtb3ZlZCBpbiB0aGlzCj4gPiA+IHZlcnNpb24uCj4g Pgo+ID4gSUlVQywgYnkgImV4aXN0aW5nIGFwcHJvYWNoIiB5b3UgbWVhbiBjb21wbGV0ZWx5IGln bm9yZSBzZWNvbmRhcnkgTU1VcyB0aGF0Cj4gPiBkb24ndCBpbXBsZW1lbnQgYSBsb2NrbGVzcyB3 YWxrPwo+IAo+IE5vLCB0aGUgZXhpc3RpbmcgYXBwcm9hY2ggb25seSBjaGVja3Mgc2Vjb25kYXJ5 IE1NVXMgZm9yIExSVSBmb2xpb3MsCj4gaS5lLiwgdGhvc2UgYXQgdGhlIGVuZCBvZiB0aGUgTFJV IGxpc3QuIEl0IG1pZ2h0IG5vdCBmaW5kIHRoZSBiZXN0Cj4gY2FuZGlkYXRlcyAodGhlIGNvbGRl c3Qgb25lcykgb24gdGhlIGVudGlyZSBsaXN0LCBidXQgaXQgZG9lc24ndCBwYXkKPiBhcyBtdWNo IGZvciB0aGUgbG9ja2luZy4gTUdMUlUgY2FuICpvcHRpb25hbGx5KiBzY2FuIE1NVXMgKHNlY29u ZGFyeQo+IGluY2x1ZGVkKSB0byBmaW5kIHRoZSBiZXN0IGNhbmRpZGF0ZXMsIGJ1dCBpdCBjYW4g b25seSBiZSBhIHdpbiBpZiB0aGUKPiBzY2FubmluZyBpbmN1cnMgYSByZWxhdGl2ZWx5IGxvdyBv dmVyaGVhZCwgZS5nLiwgZG9uZSBsb2NrbGVzc2x5IGZvcgo+IHRoZSBzZWNvbmRhcnkgTU1VLiBJ T1csIHRoaXMgaXMgYSBiYWxhbmNlIGJldHdlZW4gdGhlIGNvc3Qgb2YKPiByZWNsYWltaW5nIG5v dC1zby1jb2xkICh3YXJtKSBmb2xpb3MgYW5kIHRoYXQgb2YgZmluZGluZyB0aGUgY29sZGVzdAo+ IGZvbGlvcy4KCkdvdGNoYS4KCkkgdGVuZCB0byBhZ3JlZSB3aXRoIFl1LCBkcml2aW5nIHRoZSBi ZWhhdmlvciB2aWEgYSBLY29uZmlnIG1heSBnZW5lcmF0ZSBzaW1wbGVyCl9jb2RlXywgYnV0IEkg dGhpbmsgaXQgaW5jcmVhc2VzIHRoZSBvdmVyYWxsIHN5c3RlbSBjb21wbGV4aXR5LiAgRS5nLiBk aXN0cm9zCndpbGwgbGlrZWx5IGVuYWJsZSB0aGUgS2NvbmZpZywgYW5kIGluIG15IGV4cGVyaWVu Y2UgcGVvcGxlIHVzaW5nIEtWTSB3aXRoIGEKZGlzdHJvIGtlcm5lbCB1c3VhbGx5IGFyZW4ndCBr ZXJuZWwgZXhwZXJ0cywgaS5lLiBsaWtlbHkgd29uJ3Qga25vdyB0aGF0IHRoZXJlJ3MKZXZlbiBh IGRlY2lzaW9uIHRvIGJlIG1hZGUsIGxldCBhbG9uZSBiZSBhYmxlIHRvIG1ha2UgYW4gaW5mb3Jt ZWQgZGVjaXNpb24uCgpIYXZpbmcgYW4gbW11X25vdGlmaWVyIGhvb2sgdGhhdCBpcyBjb25kaXRp b25hbGx5IGltcGxlbWVudGVkIGRvZXNuJ3Qgc2VlbSBvdmVybHkKY29tcGxleCwgZS5nLiBldmVu IGlmIHRoZXJlJ3MgYSBydW50aW1lIGFzcGVjdCBhdCBwbGF5LCBpdCdkIGJlIGVhc3kgZW5vdWdo IGZvcgpLVk0gdG8gbnVsbGlmeSBpdHMgbW11X25vdGlmaWVyIGhvb2sgZHVyaW5nIGluaXRpYWxp emF0aW9uLiAgVGhlIGhhcmRlc3QgcGFydCBpcwpsaWtlbHkgZ29pbmcgdG8gYmUgZmlndXJpbmcg b3V0IHRoZSB0aHJlc2hvbGQgZm9yIGhvdyBtdWNoIG92ZXJoZWFkIGlzIHRvbyBtdWNoLgoKX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtcmlzY3Yg bWFpbGluZyBsaXN0CmxpbnV4LXJpc2N2QGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3Rz LmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1yaXNjdgo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45FB5C25B75 for ; Wed, 29 May 2024 23:06:44 +0000 (UTC) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20230601 header.b=NzfKHTcz; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4VqPvL48bWz7B1f for ; Thu, 30 May 2024 08:58:54 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20230601 header.b=NzfKHTcz; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=flex--seanjc.bounces.google.com (client-ip=2607:f8b0:4864:20::1149; helo=mail-yw1-x1149.google.com; envelope-from=3_ljxzgykdpktfbokdhpphmf.dpnmjovyqqd-efwmjtut.p0mbct.psh@flex--seanjc.bounces.google.com; receiver=lists.ozlabs.org) Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4VqPtR6rM7z79Ll for ; Thu, 30 May 2024 08:58:06 +1000 (AEST) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-62a0825e365so4039717b3.1 for ; Wed, 29 May 2024 15:58:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1717023484; x=1717628284; darn=lists.ozlabs.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=lXQ4aAmXpFR+Rnn+LVSp/P9w+zocv3IAjl6jcVlXHAQ=; b=NzfKHTczE266gqQ/xVfeAnGfcOg4r3SFetpnZiEm6VBpFmPpluvu+XOKg5LYVIqCnX 5Vb1YU/KNkuWUqCu0+GES7kdj3yuv1hifd6ARy/4dB7Cxo4FiPO2Eu53tmRydpHPpFe0 tMYz0oKkETlH8tP2TINCNmohbZ/dParBzrtn43uVJo219irATiDharAHAvXJacYOPZJu 8ObKtM/r5zhP0KmpWqIfRuObu/9RQNh3gBkWDYQq8lhmc9Z5/miwwBj6o7XAqT9qls+9 pHGr29gg5D+RzFtaEbksf//BAm4PBKXCpuJEu4y+ejDkPH1R3Lw/cld92Ajagv5fywiK lTXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717023484; x=1717628284; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=lXQ4aAmXpFR+Rnn+LVSp/P9w+zocv3IAjl6jcVlXHAQ=; b=srEtc5IOG/t1H16cjyu9EjkHtfQJukP77/AOwMShrywL1KVTCLtKI+kzLKEO9+9Rsc 6GERHQHLMFbd7yE1CDXxo+TnJaGZ5V/vlejH3JK3Ld3n7P/r8D5YRGMf+emlID6oynpi 7cbt7oyoBgHyJU8FBxLrDIw8XojP1bEFEeeo5A1qLvhAiFsap79WQEDJJ+iPPLb8XyPt lcy2GtAtb6BMnm0rkQyhAhxaeHLJAIw46erL6OLCmsR09fmjc4lLu0Vp848GHKNadqqP jxIHx6VR+1C55ru066eKpxM0uV3Dv1Wv0rqFHKKNZZuFB5KqUqtYpaHbOmk4d/uLOo2P +IWw== X-Forwarded-Encrypted: i=1; AJvYcCXhXmHqR9h4sYym1PZJVn87VTg4FITVSHb9XRUrwJH0/+yA9UN+PUpj1zzCfK2U5LM1BP/hoaX1FuoI2STsfymhdJ92ttaDmw/fnkzXEA== X-Gm-Message-State: AOJu0YxWWsry9/QHslzLLrH0YlteTo4Iuib2g6z13bWdnaquUMvSKhfG CXuY3dln82GGFWrMbuKDSsdZa96t2kfwpRdcMQL2fKnEcm3sGG1NsBYa3QPjX1D1Vk6+y/EcT+d u3A== X-Google-Smtp-Source: AGHT+IEcoxWAWGQljmQC1ObvPASoASZSWeSUzQr2Szq7dgxCnuAYyloQCwOCyj3B7twEp9K1BnNOHbqA4OU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:1007:b0:dfa:5838:b919 with SMTP id 3f1490d57ef6-dfa5a68828dmr142777276.10.1717023484293; Wed, 29 May 2024 15:58:04 -0700 (PDT) Date: Wed, 29 May 2024 15:58:02 -0700 In-Reply-To: Mime-Version: 1.0 References: <20240529180510.2295118-1-jthoughton@google.com> <20240529180510.2295118-3-jthoughton@google.com> Message-ID: Subject: Re: [PATCH v4 2/7] mm: multi-gen LRU: Have secondary MMUs participate in aging From: Sean Christopherson To: Yu Zhao Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: James Houghton , kvm@vger.kernel.org, linux-doc@vger.kernel.org, Catalin Marinas , Atish Patra , linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, linux-kselftest@vger.kernel.org, Raghavendra Rao Ananta , linux-riscv@lists.infradead.org, Shuah Khan , Jonathan Corbet , Anup Patel , Huacai Chen , David Rientjes , Zenghui Yu , Axel Rasmussen , linux-mips@vger.kernel.org, Albert Ou , Ryan Roberts , Will Deacon , Suzuki K Poulose , Shaoqin Huang , Nicholas Piggin , Bibo Mao , loongarch@lists.linux.dev, Paul Walmsley , David Matlack , Palmer Dabbelt , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, Ankit Agrawal , Oliver Upton , James Morse , kvm-riscv@lists.infradead.org, Marc Zyngier , Paolo Bonzini , Andrew Morton , Tianrui Zhao , linuxppc-dev@lists.ozlabs.org Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Wed, May 29, 2024, Yu Zhao wrote: > On Wed, May 29, 2024 at 3:59=E2=80=AFPM Sean Christopherson wrote: > > > > On Wed, May 29, 2024, Yu Zhao wrote: > > > On Wed, May 29, 2024 at 12:05=E2=80=AFPM James Houghton wrote: > > > > > > > > Secondary MMUs are currently consulted for access/age information a= t > > > > eviction time, but before then, we don't get accurate age informati= on. > > > > That is, pages that are mostly accessed through a secondary MMU (li= ke > > > > guest memory, used by KVM) will always just proceed down to the old= est > > > > generation, and then at eviction time, if KVM reports the page to b= e > > > > young, the page will be activated/promoted back to the youngest > > > > generation. > > > > > > Correct, and as I explained offline, this is the only reasonable > > > behavior if we can't locklessly walk secondary MMUs. > > > > > > Just for the record, the (crude) analogy I used was: > > > Imagine a large room with many bills ($1, $5, $10, ...) on the floor, > > > but you are only allowed to pick up 10 of them (and put them in your > > > pocket). A smart move would be to survey the room *first and then* > > > pick up the largest ones. But if you are carrying a 500 lbs backpack, > > > you would just want to pick up whichever that's in front of you rathe= r > > > than walk the entire room. > > > > > > MGLRU should only scan (or lookaround) secondary MMUs if it can be > > > done lockless. Otherwise, it should just fall back to the existing > > > approach, which existed in previous versions but is removed in this > > > version. > > > > IIUC, by "existing approach" you mean completely ignore secondary MMUs = that > > don't implement a lockless walk? >=20 > No, the existing approach only checks secondary MMUs for LRU folios, > i.e., those at the end of the LRU list. It might not find the best > candidates (the coldest ones) on the entire list, but it doesn't pay > as much for the locking. MGLRU can *optionally* scan MMUs (secondary > included) to find the best candidates, but it can only be a win if the > scanning incurs a relatively low overhead, e.g., done locklessly for > the secondary MMU. IOW, this is a balance between the cost of > reclaiming not-so-cold (warm) folios and that of finding the coldest > folios. Gotcha. I tend to agree with Yu, driving the behavior via a Kconfig may generate si= mpler _code_, but I think it increases the overall system complexity. E.g. distr= os will likely enable the Kconfig, and in my experience people using KVM with = a distro kernel usually aren't kernel experts, i.e. likely won't know that th= ere's even a decision to be made, let alone be able to make an informed decision. Having an mmu_notifier hook that is conditionally implemented doesn't seem = overly complex, e.g. even if there's a runtime aspect at play, it'd be easy enough= for KVM to nullify its mmu_notifier hook during initialization. The hardest pa= rt is likely going to be figuring out the threshold for how much overhead is too = much. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 34BFEC25B75 for ; Wed, 29 May 2024 22:58:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=QB2ShnnCwMqv/4r4gSNMU1pI0cFtojijxOkudoCSpqs=; b=qRtB+E3tFRR4G3hlzPGpYwsZXR HNc0uxgWL8WsTHo/cemdOR7dNNXFunAUHq9EeAva9dhu6znfMzyPHqFxO/k3IT1JtAzy5Khy59a0i QoWwkRPgP68AlQsZkIpQAbesRshUntZ4nUgbHiU8YB4tU4050Kt3AcJxNSbAVrKkylfSCaUywP8n+ Tlxz6880KOJ8YVovH9Oa+Z6eODGzt2qn0WurLUBAsIXI6JF/7m2G1c8p5IJQX6pnYq4K+PnUprUkD d0OwTQ/NeuyfBiJe4+fsgwUdwBOx7MVMzjeKrdTJQV9sMGBiYTly1xdjMpHBsqAQKNIJDDIqY/QU+ K1BrqvZQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCSFQ-00000005lkf-0Ol6; Wed, 29 May 2024 22:58:20 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sCSFL-00000005leW-48w2 for linux-arm-kernel@lists.infradead.org; Wed, 29 May 2024 22:58:17 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-df4dc93d0d3so386220276.2 for ; Wed, 29 May 2024 15:58:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1717023484; x=1717628284; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=lXQ4aAmXpFR+Rnn+LVSp/P9w+zocv3IAjl6jcVlXHAQ=; b=X55ojBrsRsz7qlX450m3rkuGFgaMOJjsY//QOFxltNZR8hM4CP92Bz94zoTZiKkUsk I5uT/Ppeh01kLIeXe0QHT+uhrGnZ5rqq8nslSTm/vad/hQ0VJel7S+d4bfE9iubJm4RV 2+fD3Qa3AZzSmlxKpc8Eu5Uc3MmkSF0TbhydP4PzDODqE/w2cfmBzHc6Tp1IkWRwe/I2 gTgvt1d4mxWoEqQGWX7N4xRxN2WTEAHZPOw4IPclDVntk2ih/atCKOYfFj8PxgO210Vw JN8qh7bqM1u/dt3rHjxVgFS7W4zxPD9EVFteADgyrXRWyGEO02nOYPOQ2djtWX/hCiof RSqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717023484; x=1717628284; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=lXQ4aAmXpFR+Rnn+LVSp/P9w+zocv3IAjl6jcVlXHAQ=; b=NDub4cKJ90ZUEKo7k0yZbZmqrbi5/c7nLu9HQoFeIY1sD1kfoSN1MGSU82G+Cfdwm7 UMHKawdL1gyno1FrnfE/obPE8zp4Nza9q8wr6hnUEApsi3z2+p69mnw3zBx5Q1o55og0 yKmA2STg2YJJq6u6dSms+Y8JlHBM3vMRC22JluZVxrDt1hsI0sGv1+DG0jOi8N0cPYzB NCdl7GZtHRrDAbG7voe6VoDaKZmlfMAIqQ+wg9c17XoPaLklhm6OEemHcFFNYH3pXGjV 6z5vPBvaX0XAFJxbC0NeQiloxsdmd+3UJwl52JwwcerGZKAzNq/gRvmc7Dmkfxa4/pq3 ofhw== X-Forwarded-Encrypted: i=1; AJvYcCV0A7Phuzj6P9RpyZICEgmGHT6oATPjmiZ4CCWfiUX7UG4UmOUyV9427sBSGhhKUBSRoSuaFC15vdM37YEs/woVCgDEHEA9vgCrr14r5uaBKfD+eko= X-Gm-Message-State: AOJu0YwaGaRRPLO3JhvFArVe9StH0oFHeGbDQhed4tXg5ZYeAW81flEO N9uM+afQqSgtgMsIXjGAjJN1x2VF0f55EFQ9XBWfJxpTKTjbQeIy/LytoaWF5MU2rNoyrHWBJdp 4kA== X-Google-Smtp-Source: AGHT+IEcoxWAWGQljmQC1ObvPASoASZSWeSUzQr2Szq7dgxCnuAYyloQCwOCyj3B7twEp9K1BnNOHbqA4OU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a05:6902:1007:b0:dfa:5838:b919 with SMTP id 3f1490d57ef6-dfa5a68828dmr142777276.10.1717023484293; Wed, 29 May 2024 15:58:04 -0700 (PDT) Date: Wed, 29 May 2024 15:58:02 -0700 In-Reply-To: Mime-Version: 1.0 References: <20240529180510.2295118-1-jthoughton@google.com> <20240529180510.2295118-3-jthoughton@google.com> Message-ID: Subject: Re: [PATCH v4 2/7] mm: multi-gen LRU: Have secondary MMUs participate in aging From: Sean Christopherson To: Yu Zhao Cc: James Houghton , Andrew Morton , Paolo Bonzini , Albert Ou , Ankit Agrawal , Anup Patel , Atish Patra , Axel Rasmussen , Bibo Mao , Catalin Marinas , David Matlack , David Rientjes , Huacai Chen , James Morse , Jonathan Corbet , Marc Zyngier , Michael Ellerman , Nicholas Piggin , Oliver Upton , Palmer Dabbelt , Paul Walmsley , Raghavendra Rao Ananta , Ryan Roberts , Shaoqin Huang , Shuah Khan , Suzuki K Poulose , Tianrui Zhao , Will Deacon , Zenghui Yu , kvm-riscv@lists.infradead.org, kvm@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240529_155816_055591_66DE372E X-CRM114-Status: GOOD ( 30.01 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gV2VkLCBNYXkgMjksIDIwMjQsIFl1IFpoYW8gd3JvdGU6Cj4gT24gV2VkLCBNYXkgMjksIDIw MjQgYXQgMzo1OeKAr1BNIFNlYW4gQ2hyaXN0b3BoZXJzb24gPHNlYW5qY0Bnb29nbGUuY29tPiB3 cm90ZToKPiA+Cj4gPiBPbiBXZWQsIE1heSAyOSwgMjAyNCwgWXUgWmhhbyB3cm90ZToKPiA+ID4g T24gV2VkLCBNYXkgMjksIDIwMjQgYXQgMTI6MDXigK9QTSBKYW1lcyBIb3VnaHRvbiA8anRob3Vn aHRvbkBnb29nbGUuY29tPiB3cm90ZToKPiA+ID4gPgo+ID4gPiA+IFNlY29uZGFyeSBNTVVzIGFy ZSBjdXJyZW50bHkgY29uc3VsdGVkIGZvciBhY2Nlc3MvYWdlIGluZm9ybWF0aW9uIGF0Cj4gPiA+ ID4gZXZpY3Rpb24gdGltZSwgYnV0IGJlZm9yZSB0aGVuLCB3ZSBkb24ndCBnZXQgYWNjdXJhdGUg YWdlIGluZm9ybWF0aW9uLgo+ID4gPiA+IFRoYXQgaXMsIHBhZ2VzIHRoYXQgYXJlIG1vc3RseSBh Y2Nlc3NlZCB0aHJvdWdoIGEgc2Vjb25kYXJ5IE1NVSAobGlrZQo+ID4gPiA+IGd1ZXN0IG1lbW9y eSwgdXNlZCBieSBLVk0pIHdpbGwgYWx3YXlzIGp1c3QgcHJvY2VlZCBkb3duIHRvIHRoZSBvbGRl c3QKPiA+ID4gPiBnZW5lcmF0aW9uLCBhbmQgdGhlbiBhdCBldmljdGlvbiB0aW1lLCBpZiBLVk0g cmVwb3J0cyB0aGUgcGFnZSB0byBiZQo+ID4gPiA+IHlvdW5nLCB0aGUgcGFnZSB3aWxsIGJlIGFj dGl2YXRlZC9wcm9tb3RlZCBiYWNrIHRvIHRoZSB5b3VuZ2VzdAo+ID4gPiA+IGdlbmVyYXRpb24u Cj4gPiA+Cj4gPiA+IENvcnJlY3QsIGFuZCBhcyBJIGV4cGxhaW5lZCBvZmZsaW5lLCB0aGlzIGlz IHRoZSBvbmx5IHJlYXNvbmFibGUKPiA+ID4gYmVoYXZpb3IgaWYgd2UgY2FuJ3QgbG9ja2xlc3Ns eSB3YWxrIHNlY29uZGFyeSBNTVVzLgo+ID4gPgo+ID4gPiBKdXN0IGZvciB0aGUgcmVjb3JkLCB0 aGUgKGNydWRlKSBhbmFsb2d5IEkgdXNlZCB3YXM6Cj4gPiA+IEltYWdpbmUgYSBsYXJnZSByb29t IHdpdGggbWFueSBiaWxscyAoJDEsICQ1LCAkMTAsIC4uLikgb24gdGhlIGZsb29yLAo+ID4gPiBi dXQgeW91IGFyZSBvbmx5IGFsbG93ZWQgdG8gcGljayB1cCAxMCBvZiB0aGVtIChhbmQgcHV0IHRo ZW0gaW4geW91cgo+ID4gPiBwb2NrZXQpLiBBIHNtYXJ0IG1vdmUgd291bGQgYmUgdG8gc3VydmV5 IHRoZSByb29tICpmaXJzdCBhbmQgdGhlbioKPiA+ID4gcGljayB1cCB0aGUgbGFyZ2VzdCBvbmVz LiBCdXQgaWYgeW91IGFyZSBjYXJyeWluZyBhIDUwMCBsYnMgYmFja3BhY2ssCj4gPiA+IHlvdSB3 b3VsZCBqdXN0IHdhbnQgdG8gcGljayB1cCB3aGljaGV2ZXIgdGhhdCdzIGluIGZyb250IG9mIHlv dSByYXRoZXIKPiA+ID4gdGhhbiB3YWxrIHRoZSBlbnRpcmUgcm9vbS4KPiA+ID4KPiA+ID4gTUdM UlUgc2hvdWxkIG9ubHkgc2NhbiAob3IgbG9va2Fyb3VuZCkgc2Vjb25kYXJ5IE1NVXMgaWYgaXQg Y2FuIGJlCj4gPiA+IGRvbmUgbG9ja2xlc3MuIE90aGVyd2lzZSwgaXQgc2hvdWxkIGp1c3QgZmFs bCBiYWNrIHRvIHRoZSBleGlzdGluZwo+ID4gPiBhcHByb2FjaCwgd2hpY2ggZXhpc3RlZCBpbiBw cmV2aW91cyB2ZXJzaW9ucyBidXQgaXMgcmVtb3ZlZCBpbiB0aGlzCj4gPiA+IHZlcnNpb24uCj4g Pgo+ID4gSUlVQywgYnkgImV4aXN0aW5nIGFwcHJvYWNoIiB5b3UgbWVhbiBjb21wbGV0ZWx5IGln bm9yZSBzZWNvbmRhcnkgTU1VcyB0aGF0Cj4gPiBkb24ndCBpbXBsZW1lbnQgYSBsb2NrbGVzcyB3 YWxrPwo+IAo+IE5vLCB0aGUgZXhpc3RpbmcgYXBwcm9hY2ggb25seSBjaGVja3Mgc2Vjb25kYXJ5 IE1NVXMgZm9yIExSVSBmb2xpb3MsCj4gaS5lLiwgdGhvc2UgYXQgdGhlIGVuZCBvZiB0aGUgTFJV IGxpc3QuIEl0IG1pZ2h0IG5vdCBmaW5kIHRoZSBiZXN0Cj4gY2FuZGlkYXRlcyAodGhlIGNvbGRl c3Qgb25lcykgb24gdGhlIGVudGlyZSBsaXN0LCBidXQgaXQgZG9lc24ndCBwYXkKPiBhcyBtdWNo IGZvciB0aGUgbG9ja2luZy4gTUdMUlUgY2FuICpvcHRpb25hbGx5KiBzY2FuIE1NVXMgKHNlY29u ZGFyeQo+IGluY2x1ZGVkKSB0byBmaW5kIHRoZSBiZXN0IGNhbmRpZGF0ZXMsIGJ1dCBpdCBjYW4g b25seSBiZSBhIHdpbiBpZiB0aGUKPiBzY2FubmluZyBpbmN1cnMgYSByZWxhdGl2ZWx5IGxvdyBv dmVyaGVhZCwgZS5nLiwgZG9uZSBsb2NrbGVzc2x5IGZvcgo+IHRoZSBzZWNvbmRhcnkgTU1VLiBJ T1csIHRoaXMgaXMgYSBiYWxhbmNlIGJldHdlZW4gdGhlIGNvc3Qgb2YKPiByZWNsYWltaW5nIG5v dC1zby1jb2xkICh3YXJtKSBmb2xpb3MgYW5kIHRoYXQgb2YgZmluZGluZyB0aGUgY29sZGVzdAo+ IGZvbGlvcy4KCkdvdGNoYS4KCkkgdGVuZCB0byBhZ3JlZSB3aXRoIFl1LCBkcml2aW5nIHRoZSBi ZWhhdmlvciB2aWEgYSBLY29uZmlnIG1heSBnZW5lcmF0ZSBzaW1wbGVyCl9jb2RlXywgYnV0IEkg dGhpbmsgaXQgaW5jcmVhc2VzIHRoZSBvdmVyYWxsIHN5c3RlbSBjb21wbGV4aXR5LiAgRS5nLiBk aXN0cm9zCndpbGwgbGlrZWx5IGVuYWJsZSB0aGUgS2NvbmZpZywgYW5kIGluIG15IGV4cGVyaWVu Y2UgcGVvcGxlIHVzaW5nIEtWTSB3aXRoIGEKZGlzdHJvIGtlcm5lbCB1c3VhbGx5IGFyZW4ndCBr ZXJuZWwgZXhwZXJ0cywgaS5lLiBsaWtlbHkgd29uJ3Qga25vdyB0aGF0IHRoZXJlJ3MKZXZlbiBh IGRlY2lzaW9uIHRvIGJlIG1hZGUsIGxldCBhbG9uZSBiZSBhYmxlIHRvIG1ha2UgYW4gaW5mb3Jt ZWQgZGVjaXNpb24uCgpIYXZpbmcgYW4gbW11X25vdGlmaWVyIGhvb2sgdGhhdCBpcyBjb25kaXRp b25hbGx5IGltcGxlbWVudGVkIGRvZXNuJ3Qgc2VlbSBvdmVybHkKY29tcGxleCwgZS5nLiBldmVu IGlmIHRoZXJlJ3MgYSBydW50aW1lIGFzcGVjdCBhdCBwbGF5LCBpdCdkIGJlIGVhc3kgZW5vdWdo IGZvcgpLVk0gdG8gbnVsbGlmeSBpdHMgbW11X25vdGlmaWVyIGhvb2sgZHVyaW5nIGluaXRpYWxp emF0aW9uLiAgVGhlIGhhcmRlc3QgcGFydCBpcwpsaWtlbHkgZ29pbmcgdG8gYmUgZmlndXJpbmcg b3V0IHRoZSB0aHJlc2hvbGQgZm9yIGhvdyBtdWNoIG92ZXJoZWFkIGlzIHRvbyBtdWNoLgoKX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtYXJtLWtl cm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0 dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5l bAo=