From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60FF4C43217 for ; Fri, 31 Dec 2021 02:27:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9B6206B0099; Thu, 30 Dec 2021 21:27:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 93D9C6B009A; Thu, 30 Dec 2021 21:27:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7915C6B009B; Thu, 30 Dec 2021 21:27:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0154.hostedemail.com [216.40.44.154]) by kanga.kvack.org (Postfix) with ESMTP id 624F46B0099 for ; Thu, 30 Dec 2021 21:27:24 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 1E8C78771C for ; Fri, 31 Dec 2021 02:27:24 +0000 (UTC) X-FDA: 78976502808.09.D613113 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by imf23.hostedemail.com (Postfix) with ESMTP id B0B3D140003 for ; Fri, 31 Dec 2021 02:27:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1640917643; x=1672453643; h=date:from:to:cc:subject:message-id:reply-to:references: mime-version:in-reply-to; bh=GWTpTesJcb+925QqthMUbTz8/8tLtbXxt+lqNt52Aec=; b=nyKXvtzvT8cehP+OGMmI5Co2TeLGJ0seVbtd9Lm+Dq+y0mMB8xCEIjJt bhSw4oMzGHlEYUZmuxHPVEm2Qozu4ugWBasV6QyzvJJqi+5FHKQGSwOcp s68dVKTYkBRYP9pMHXL9Bpdp4+kTtFnXT/IF8Nu6G+nCAwuLgzDqXeCBZ 9G2GXxYf8KJ2fulg9BLyqZMvK5YxwhN3zV1jNXNBBaDAXV+wb1XEgrVI7 VK77bTtis3+O99/BRIMlxnyv4xSUi4AiQ8BjZSh9dFkWUWZ5sujZ+psG5 ymv67zj7kYSlSQMs84L4QO/dOk+Z2wKuqU0Nhp2KjV4eW2TNHHhv4oXTt Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10213"; a="305123266" X-IronPort-AV: E=Sophos;i="5.88,250,1635231600"; d="scan'208";a="305123266" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Dec 2021 18:27:21 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,250,1635231600"; d="scan'208";a="666705441" Received: from chaop.bj.intel.com (HELO localhost) ([10.240.192.101]) by fmsmga001.fm.intel.com with ESMTP; 30 Dec 2021 18:27:12 -0800 Date: Fri, 31 Dec 2021 10:26:36 +0800 From: Chao Peng To: Sean Christopherson Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, qemu-devel@nongnu.org, Paolo Bonzini , Jonathan Corbet , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, "H . Peter Anvin" , Hugh Dickins , Jeff Layton , "J . Bruce Fields" , Andrew Morton , Yu Zhang , "Kirill A . Shutemov" , luto@kernel.org, john.ji@intel.com, susie.li@intel.com, jun.nakajima@intel.com, dave.hansen@intel.com, ak@linux.intel.com, david@redhat.com Subject: Re: [PATCH v3 kvm/queue 05/16] KVM: Maintain ofs_tree for fast memslot lookup by file offset Message-ID: <20211231022636.GA7025@chaop.bj.intel.com> Reply-To: Chao Peng References: <20211223123011.41044-1-chao.p.peng@linux.intel.com> <20211223123011.41044-6-chao.p.peng@linux.intel.com> <20211224035418.GA43608@chaop.bj.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=nyKXvtzv; dmarc=pass (policy=none) header.from=intel.com; spf=none (imf23.hostedemail.com: domain of chao.p.peng@linux.intel.com has no SPF policy when checking 134.134.136.100) smtp.mailfrom=chao.p.peng@linux.intel.com X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: B0B3D140003 X-Stat-Signature: gifo34xtnsh93fu4bmd4b3m4sf5snxdo X-HE-Tag: 1640917631-127336 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Dec 28, 2021 at 09:48:08PM +0000, Sean Christopherson wrote: > On Fri, Dec 24, 2021, Chao Peng wrote: > > On Thu, Dec 23, 2021 at 06:02:33PM +0000, Sean Christopherson wrote: > > > On Thu, Dec 23, 2021, Chao Peng wrote: > > > > > > In other words, there needs to be a 1:1 gfn:file+offset mapping. Since userspace > > > likely wants to allocate a single file for guest private memory and map it into > > > multiple discontiguous slots, e.g. to skip the PCI hole, the best idea off the top > > > of my head would be to register the notifier on a per-slot basis, not a per-VM > > > basis. It would require a 'struct kvm *' in 'struct kvm_memory_slot', but that's > > > not a huge deal. > > > > > > That way, KVM's notifier callback already knows the memslot and can compute overlap > > > between the memslot and the range by reversing the math done by kvm_memfd_get_pfn(). > > > Then, armed with the gfn and slot, invalidation is just a matter of constructing > > > a struct kvm_gfn_range and invoking kvm_unmap_gfn_range(). > > > > KVM is easy but the kernel bits would be difficulty, it has to maintain > > fd+offset to memslot mapping because one fd can have multiple memslots, > > it need decide which memslot needs to be notified. > > No, the kernel side maintains an opaque pointer like it does today, But the opaque pointer will now become memslot, isn't it? That said, kernel side should maintain a list of opaque pointer (memslot) instead of one for each fd (inode) since a fd to memslot mapping is 1:M now. >KVM handles > reverse engineering the memslot to get the offset and whatever else it needs. > notify_fallocate() and other callbacks are unchanged, though they probably can > drop the inode. > > E.g. likely with bad math and handwaving on the overlap detection: > > int kvm_private_fd_fallocate_range(void *owner, pgoff_t start, pgoff_t end) > { > struct kvm_memory_slot *slot = owner; > struct kvm_gfn_range gfn_range = { > .slot = slot, > .start = (start - slot->private_offset) >> PAGE_SHIFT, > .end = (end - slot->private_offset) >> PAGE_SHIFT, > .may_block = true, > }; > > if (!has_overlap(slot, start, end)) > return 0; > > gfn_range.end = min(gfn_range.end, slot->base_gfn + slot->npages); > > kvm_unmap_gfn_range(slot->kvm, &gfn_range); > return 0; > } I understand this KVM side handling, but again one fd can have multiple memslots. How shmem decides to notify which memslot from a list of memslots when it invokes the notify_fallocate()? Or just notify all the possible memslots then let KVM to check? Thanks, Chao