From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7E2BC43334 for ; Thu, 23 Jun 2022 22:36:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229575AbiFWWgq (ORCPT ); Thu, 23 Jun 2022 18:36:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33736 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229502AbiFWWgp (ORCPT ); Thu, 23 Jun 2022 18:36:45 -0400 Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8228C51339 for ; Thu, 23 Jun 2022 15:36:44 -0700 (PDT) Received: by mail-pj1-x1029.google.com with SMTP id x1-20020a17090abc8100b001ec7f8a51f5so4020578pjr.0 for ; Thu, 23 Jun 2022 15:36:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=JetDtHU3X+1B2pqv06q6XrBYV3K3bGShzpXgJBS5x0g=; b=dyEgVIex9SIwN8/Oyba1p0Nc/YJDQKTUUM+mY41FwvpygkyZzf0UtRzkKtS4EIBHHA Xck2+vMALl3GyO0YO8LzH+UzcYB+LWSq4wyqHpkd6GB7JHgdEgY4GmC2kYqywMW9wsls zHvDQWimUQ2Amyryb6JyWPUCRISy1yFQeB2SK1K2Q1zmns78u8LqMrF2dCe0rnOnEtOS s4EI3EPGuqp63XvDPn4eK8iO6Q9nl13rFabYy1ssU8BiQGN2cC5gYvW0Kh0SSQATdEo1 YR4qTvqz2T8gAwdcHcKLG32P3KZmowtxSSGghozM+FEM9d62/53a/FQfUV/bBixZfntZ 9BUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=JetDtHU3X+1B2pqv06q6XrBYV3K3bGShzpXgJBS5x0g=; b=uEAC+1SNg8mGt1Ss1hzDxOmYhBfTj5NKaT2P2yRL25aCXcZh1Sn6cOs+uqDYS0Tpmz 60KCnr43JdJdQPM+s4VJrcGYaVf+PFccJxGMEaWswCuVsoGQPYqudouVZ0EpZuvslyQw snYH12xCm59ucSYactok4Zwhd6FGQAoTQYoVf1eeCgvWv8r+6bwGBiT78QIkjoVFO2F5 sI/TowHdDQiHtegnftZfQAz7Kfndto5zqcvuQ69VEUk9hCXw8lneviXNRoFacEg07CUP D6w6PZfsHtRxrCjQfMVkftw2aAPtYP448YbtYqZ6f83eKNZZvXxmG5ihmJt2sM3HQaaQ aSxQ== X-Gm-Message-State: AJIora/illGj8DRISbIAd575EZPIZoEPFoRngKrws7tGtUagY2xzWtRI XQNeaTbemune6Bp7s30LrYiPqA== X-Google-Smtp-Source: AGRyM1tvIami/If6+RQSfpCBjEgTqjpFW/6mqfqXNmMf74n8X2WXBnNOJEHAYhBFEwQGjNfyTfwJcg== X-Received: by 2002:a17:903:10a:b0:16a:2762:88e2 with SMTP id y10-20020a170903010a00b0016a276288e2mr21456838plc.22.1656023803838; Thu, 23 Jun 2022 15:36:43 -0700 (PDT) Received: from google.com (223.103.125.34.bc.googleusercontent.com. [34.125.103.223]) by smtp.gmail.com with ESMTPSA id ca27-20020a056a00419b00b00525133f98adsm154148pfb.146.2022.06.23.15.36.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Jun 2022 15:36:42 -0700 (PDT) Date: Thu, 23 Jun 2022 22:36:37 +0000 From: David Matlack To: Sean Christopherson Cc: Paolo Bonzini , LKML , kvm list , Marc Zyngier , Anup Patel , Ben Gardon , Peter Xu , "Maciej S. Szmigiero" , KVMARM , LinuxMIPS , "open list:KERNEL VIRTUAL MACHINE FOR RISC-V (KVM/riscv)" , Peter Feiner , Lai Jiangshan Subject: Re: [PATCH v7 22/23] KVM: x86/mmu: Extend Eager Page Splitting to nested MMUs Message-ID: References: <20220622192710.2547152-1-pbonzini@redhat.com> <20220622192710.2547152-23-pbonzini@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-mips@vger.kernel.org On Thu, Jun 23, 2022 at 07:48:02PM +0000, Sean Christopherson wrote: > On Thu, Jun 23, 2022, David Matlack wrote: > > On Wed, Jun 22, 2022 at 12:27 PM Paolo Bonzini wrote: > > Please trim replies. > > > > +static int topup_split_caches(struct kvm *kvm) > > > +{ > > > + int r; > > > + > > > + lockdep_assert_held(&kvm->slots_lock); > > > + > > > + /* > > > + * It's common to need all SPLIT_DESC_CACHE_MIN_NR_OBJECTS (513) objects > > > + * when splitting a page, but setting capacity == min would cause > > > + * KVM to drop mmu_lock even if just one object was consumed from the > > > + * cache. So make capacity larger than min and handle two huge pages > > > + * without having to drop the lock. > > > > I was going to do some testing this week to confirm, but IIUC KVM will > > only allocate from split_desc_cache if the L1 hypervisor has aliased a > > huge page in multiple {E,N}PT12 page table entries. i.e. L1 is mapping > > a huge page into an L2 multiple times, or mapped into multiple L2s. > > This should be common in traditional, process-level, shadow paging, > > but I think will be quite rare for nested shadow paging. > > Ooooh, right, I forgot that that pte_list_add() needs to allocate if and only if > there are multiple rmap entries, otherwise rmap->val points that the one and only > rmap directly. > > Doubling the capacity is all but guaranteed to be pointless overhead. What about > buffering with the default capacity? That way KVM doesn't have to topup if it > happens to encounter an aliased gfn. It's arbitrary, but so is the default capacity > size. > > E.g. as fixup LGTM > > --- > arch/x86/kvm/mmu/mmu.c | 26 +++++++++++++++----------- > 1 file changed, 15 insertions(+), 11 deletions(-) > > diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c > index 22b87007efff..90d6195edcf3 100644 > --- a/arch/x86/kvm/mmu/mmu.c > +++ b/arch/x86/kvm/mmu/mmu.c > @@ -6125,19 +6125,23 @@ static bool need_topup_split_caches_or_resched(struct kvm *kvm) > > static int topup_split_caches(struct kvm *kvm) > { > - int r; > - > - lockdep_assert_held(&kvm->slots_lock); > - > /* > - * It's common to need all SPLIT_DESC_CACHE_MIN_NR_OBJECTS (513) objects > - * when splitting a page, but setting capacity == min would cause > - * KVM to drop mmu_lock even if just one object was consumed from the > - * cache. So make capacity larger than min and handle two huge pages > - * without having to drop the lock. > + * Allocating rmap list entries when splitting huge pages for nested > + * MMUs is rare as KVM needs to allocate if and only if there is more > + * than one rmap entry for the gfn, i.e. requires an L1 gfn to be > + * aliased by multiple L2 gfns, which is very atypical for VMMs. If > + * there is only one rmap entry, rmap->val points directly at that one > + * entry and doesn't need to allocate a list. Buffer the cache by the > + * default capacity so that KVM doesn't have to topup the cache if it > + * encounters an aliased gfn or two. > */ > - r = __kvm_mmu_topup_memory_cache(&kvm->arch.split_desc_cache, > - 2 * SPLIT_DESC_CACHE_MIN_NR_OBJECTS, > + const int capacity = SPLIT_DESC_CACHE_MIN_NR_OBJECTS + > + KVM_ARCH_NR_OBJS_PER_MEMORY_CACHE; > + int r; > + > + lockdep_assert_held(&kvm->slots_lock); > + > + r = __kvm_mmu_topup_memory_cache(&kvm->arch.split_desc_cache, capacity, > SPLIT_DESC_CACHE_MIN_NR_OBJECTS); > if (r) > return r; > > base-commit: 436b1c29f36ed3d4385058ba6f0d6266dbd2a882 > -- >