From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF58AC27C53 for ; Wed, 12 Jun 2024 17:31:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 35B556B007B; Wed, 12 Jun 2024 13:31:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 309D06B009B; Wed, 12 Jun 2024 13:31:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1ABD66B009C; Wed, 12 Jun 2024 13:31:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id EA06C6B009B for ; Wed, 12 Jun 2024 13:31:32 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 9A3E71218C0 for ; Wed, 12 Jun 2024 17:31:32 +0000 (UTC) X-FDA: 82222928424.03.611348D Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf03.hostedemail.com (Postfix) with ESMTP id B82B02000A for ; Wed, 12 Jun 2024 17:31:30 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=NxID0ypP; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf03.hostedemail.com: domain of 3cdtpZgYKCAo2okxtmqyyqvo.mywvsx47-wwu5kmu.y1q@flex--seanjc.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3cdtpZgYKCAo2okxtmqyyqvo.mywvsx47-wwu5kmu.y1q@flex--seanjc.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718213490; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZttrWbEybPZrU6j9LIdJZge1/Pd2kmv/UPX9FdzbJ4o=; b=CMFIuj3N/WIK0pW7S9UNqw/s+p8hWTepyl50Gh2gjtfO12EzsjHXqUzAqSIlIB8MPrII+y 3hSYn+NfULyCfma6PT/EiFAj8J1fzNkLofp1POc3NQXcqWOBYOonAwY7cum0SsfyRM/tiR 9Y/XdL+iFZjFCg5HBB+22PA5qtKvdbo= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=NxID0ypP; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf03.hostedemail.com: domain of 3cdtpZgYKCAo2okxtmqyyqvo.mywvsx47-wwu5kmu.y1q@flex--seanjc.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3cdtpZgYKCAo2okxtmqyyqvo.mywvsx47-wwu5kmu.y1q@flex--seanjc.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718213490; a=rsa-sha256; cv=none; b=YY7sXlxgxu8RtvXOmZjIzJrH1XacUmTzNlqcf71vtGxSAm2AFHkzIld7zHXqkcDMoKW/Ys 4mzB+IIqLmfc9yeEZP0Y/nWo6lSiDzhkcqorNSxfImcZprhcnUfx4w0kC2odirQy9zpS21 bpCRzhCl+ZLKTQQv8K/Mqp4o6z6KSc0= Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-1f73174cacbso851135ad.2 for ; Wed, 12 Jun 2024 10:31:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1718213489; x=1718818289; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ZttrWbEybPZrU6j9LIdJZge1/Pd2kmv/UPX9FdzbJ4o=; b=NxID0ypPvPPVfmJY5jMkE320ugAFtsoV7uvdBohy6loe8PUs0VOmGrG+t5r78+nNCK ilIqCVgqz0mdv2pd94gp32tB0zUd74zHUkwkrrvfsmhxy/c+BKNa6pr1MzPYU6dEDwL8 ctJN6UtNABDLKztJHOmj/OJhArpyDsS3KZn/jWPTKolABGZaWqG88lg8xgBxbbglxmEQ 9gx9TxBIaMcCrSgyrZomrKdwr6nUZS0G+h9vBF7dA5pi3zhDkq1pxnwtTnGyiJE2ouxn t7GiDG2YWJA159/R913ijV+Zfvz4VhCyF2RZ4Of3jyMjrDbotOqFCL0TXbDYI8h223RJ D6oQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718213489; x=1718818289; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ZttrWbEybPZrU6j9LIdJZge1/Pd2kmv/UPX9FdzbJ4o=; b=u6sWYTDoIx4MUhvEHGMy3cDd33SBCGX6iqKHfWk6GTd9D6WsrNQCeDIve+0oHSZvyz ukzj3G5DVyNG+5XhHR62PfgNnLX5HjPGoECmOeLiqlBxKwtcusavyoZRyGhm3hVqriwt AykqeF6UrBC99NBtHxTNaGnCPyNGsZ0why37qOifKBgGAOshBIhAIgz65REZ/E3Jgo5B JZsJa7XpPJcca4/MdOymmx6dGw1rPo4sqKylFXj2gZ5qxXng1RNiGo6/eD6pp/XnfNBU 9KnLB0O3huCjvod8aoYDeYO/uhWfkcvEaD+mCZQFevCC/QBI06SGuSzMV7vwDojVWKCV fvxw== X-Forwarded-Encrypted: i=1; AJvYcCUznvMFqXj2MxVePwryxz1O0PMEHt8IKOZPLhl9Dtp41xLRLkysyJghtdJ55/9m9njt2ZG2THWmnpwjnDifVVk8fsI= X-Gm-Message-State: AOJu0YxnGepVvEHcTHRAuoPRWli6FltW2AgKBECeUuaPOzf8LJFJ8I6S CsI1SwCFDMh3IKbSaOAbFYrI7BXp7Rnt+HbM9+IY2KtlsNAxdtf2orC9mwbH7Lzc5hwbRNIkNFj Xig== X-Google-Smtp-Source: AGHT+IEfEUqjAJd5HwtcI3SellD9wJyKICC45AOG2hDSPfUBXvAeRlQriWUGwdY0fjJOl8E1dOYOBVVBc8s= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5c37]) (user=seanjc job=sendgmr) by 2002:a17:903:182:b0:1f3:4b2a:1acb with SMTP id d9443c01a7336-1f83b23ddb1mr79315ad.0.1718213489334; Wed, 12 Jun 2024 10:31:29 -0700 (PDT) Date: Wed, 12 Jun 2024 10:31:27 -0700 In-Reply-To: Mime-Version: 1.0 References: <20240405115815.3226315-2-pbonzini@redhat.com> <20240412104408.GA27645@willie-the-truck> <86jzl2sovz.wl-maz@kernel.org> <86h6g5si0m.wl-maz@kernel.org> <20240418141932.GA1855@willie-the-truck> <20240419112432.GB2972@willie-the-truck> Message-ID: Subject: Re: [PATCH 1/4] KVM: delete .change_pte MMU notifier callback From: Sean Christopherson To: Will Deacon Cc: Marc Zyngier , Paolo Bonzini , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Oliver Upton , Tianrui Zhao , Bibo Mao , Thomas Bogendoerfer , Nicholas Piggin , Anup Patel , Atish Patra , Andrew Morton , David Hildenbrand , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org Content-Type: text/plain; charset="us-ascii" X-Rspamd-Queue-Id: B82B02000A X-Stat-Signature: 79hz6gp9z98ydap3ci9xn1tsbzg63mey X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1718213490-831773 X-HE-Meta: U2FsdGVkX19ZWoR+54grBjrz+BPsxwCREdYrDSCSuC5zox/OlzNSm7HkPn1VJ7vxfu7ZEi0VyL2pNBfTBsP+4k9IWBtNONe6XFVtBjxl9HwUUe4kgbX7/wS0Fk/mvRIWs6ctfhBxKvJpSYNWXLezab9Ncn8tm+8Y8v1kJ0qr322M/FIQvQ64jO0jv1jU6Bzbr46tacdq8/5qm3vaN9I5S7zQnuaTXKheN/Idy2gdbD3Hz7X/PxIlWQGuyykVZFd6QDVJuJw2kkfH9Ow/QurToIAPy0nAW/6Vj7/mVcbqNX0TrVD95P+Z6USf2HM8rk7oY4dAC6J6arMFAKXXhqSYdrpZwX6Du/vRsxRs8ivlyZP0+Z+BN6ajifqfzXhRix/V7Ql1e+LpfaT1hV8QXZjPx+mHIqJAx8Lg7A2lpSPPp3cYJetnhECyH6psX/Xk6KMJGT8gxgN37D9xxu761qPq1+z+cesR3NlKhVlyWv1F3GGZWJVTW4vTsH1nWzfyqaXCBvpCRIEnhcEuFBuoO9ReHLPIeMr8jsetlfOPl471t64/kiG+Xe41CG1WtjF7mNvA2GN+lMdFHYDsuvSkCXwRFXMNdDMk7KnjTjkLTHA7qCODodBjOfnEVyJAdXuRYEcTkQOL1FaN/PFSNDE7Yir6R7RIl/uH9Ll6mPRAm5i8vQuIGfFa2ACAj2Rbtj0pirHBcftkx+3lS4ZaXqVDelrx+3bdCN5gMV2HYreaehq/ji0TKTFamtSBloTOwflk9RZXEWIwUi7QUF+eNkLf7TliK8pajRWV1ocO/ytrgWML6odrhtfsQxPmDbu6GaCkWt/HVJ0QHcxo9XfwA6v8HUJJ1kAc+ijSyK4UiSeGOpOGVx/TZHMeOJfV3Mlffh6aerlngUqTqNSv3w8MZbYcvqW2/OgIO5yRUXlu9X5d/xQf6h6A8dOUW6UUaBCovynUCJ2YDsLrq2oEaQKiqxVNArA FMtJGnav X6V/zkXhmOapjCRX4avVaTxz0eANPi4ZUcjgBMDHTIrjxxf1y7x0zqgleZmZXAOQOvXXuYQ3Yov1dPEaG8PIJji9r5a9Liufejgy0KTzuHanmnfkxoNAvVgomzf+VdXW3Vu/2OfK/Wod41VI5dOxrgU7mb/GX1vSBsaw6MUwd2c7cIVd3gpiKGyhmY4/4hR9Ce+LE4aTnyilcYlTDjdc87R624rU+0OkXKUCjCxvzrJbes5XSVoPGVd6gu4KxODhQFGDXlm8Ulz+ggThXotMzFIhX59OSF1nKdVyXGVJZN/DgiTt2dYju57nSqTrbM6ZtJvddhk4WIMv0z7z7ICdQYv6LJjhwl/NBfspB1lnEz24ohQD99H8odLqowwstIfbKbwWxub5mxwCVdPR76TjcHpxWkeIIc0MFZAfK1XEU+cdWYdkdPCu4xO+8v5Ttq7T7tZuBTDrQ7HnGQooINAuTiinHvoxL3beH+WUZE4Ckn8ReMhBG5h9zSonIipcZhXyyeTIa/rAsQrcaNZG2Xub50Z1q/WqE5mqA9O/yRsRDNI8d/i92GAK1ceUFX6LGbLAuFhlnKNTlcIXkvTmzF/iJi55gUQ3wIzkWs7SjIVvAzyHoTUKkeAdicgmjgvkPSRr24RGxa90G7e5EdGTQ/eCoDMTgZtEzMZB4HD6NUj2pr4P1kO/eQceL31HUmV8Sg8WVvZzLRg7T6dA6kXBVdXQHY6dmyg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jun 12, 2024, Sean Christopherson wrote: > On Fri, Apr 19, 2024, Will Deacon wrote: > > On Thu, Apr 18, 2024 at 12:53:26PM -0700, Sean Christopherson wrote: > > > On Thu, Apr 18, 2024, Will Deacon wrote: > > > > > I assume the idea would be to let arch code do single-page invalidations of > > > > > stage-2 entries for each gfn? > > > > > > > > Right, as it's the only code which knows which ptes actually ended up > > > > being aged. > > > > > > > > > Unless I'm having a brain fart, x86 can't make use of that functionality. Intel > > > > > doesn't provide any way to do targeted invalidation of stage-2 mappings. AMD > > > > > provides an instruction to do broadcast invalidations, but it takes a virtual > > > > > address, i.e. a stage-1 address. I can't tell if it's a host virtual address or > > > > > a guest virtual address, but it's a moot point because KVM doen't have the guest > > > > > virtual address, and if it's a host virtual address, there would need to be valid > > > > > mappings in the host page tables for it to work, which KVM can't guarantee. > > > > > > > > Ah, so it sounds like it would need to be an arch opt-in then. > > > > > > Even if x86 (or some other arch code) could use the precise tracking, I think it > > > would make sense to have the behavior be arch specific. Adding infrastructure > > > to get information from arch code, only to turn around and give it back to arch > > > code would be odd. > > > > Sorry, yes, that's what I had in mind. Basically, a way for the arch code > > to say "I've handled the TLBI, don't worry about it." > > > > > Unless arm64 can't do the invalidation immediately after aging the stage-2 PTE, > > > the best/easiest solution would be to let arm64 opt out of the common TLB flush > > > when a SPTE is made young. > > > > > > With the range-based flushing bundled in, this? > > > > > > --- > > > include/linux/kvm_host.h | 2 ++ > > > virt/kvm/kvm_main.c | 40 +++++++++++++++++++++++++--------------- > > > 2 files changed, 27 insertions(+), 15 deletions(-) > > > > > > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h > > > index afbc99264ffa..8fe5f5e16919 100644 > > > --- a/include/linux/kvm_host.h > > > +++ b/include/linux/kvm_host.h > > > @@ -2010,6 +2010,8 @@ extern const struct kvm_stats_header kvm_vcpu_stats_header; > > > extern const struct _kvm_stats_desc kvm_vcpu_stats_desc[]; > > > > > > #ifdef CONFIG_KVM_GENERIC_MMU_NOTIFIER > > > +int kvm_arch_flush_tlb_if_young(void); > > > + > > > static inline int mmu_invalidate_retry(struct kvm *kvm, unsigned long mmu_seq) > > > { > > > if (unlikely(kvm->mmu_invalidate_in_progress)) > > > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c > > > index 38b498669ef9..5ebef8ef239c 100644 > > > --- a/virt/kvm/kvm_main.c > > > +++ b/virt/kvm/kvm_main.c > > > @@ -595,6 +595,11 @@ static void kvm_null_fn(void) > > > } > > > #define IS_KVM_NULL_FN(fn) ((fn) == (void *)kvm_null_fn) > > > > > > +int __weak kvm_arch_flush_tlb_if_young(void) > > > +{ > > > + return true; > > > +} > > > > I tend to find __weak functions a little ugly, but I think the gist of the > > diff looks good to me. Thanks for putting it together! > > Circling back to this, I don't think we should pursue this specific tweak, at > least not without hard data for a concrete use case. Ha, I spoke too soon. Based on the learning from the KVM+MGLRU thread[*], it looks like KVM should omit the TLB flush when aging pages whenever possible. If that's not doable on all architectures for whatever reason, then something like this is probably the way to go. [*] https://lore.kernel.org/all/CAOUHufYCmYNngmS=rOSAQRB0N9ai+mA0aDrB9RopBvPHEK42Ng@mail.gmail.com