From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0A59C369CB for ; Wed, 23 Apr 2025 12:27:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 113E46B00A4; Wed, 23 Apr 2025 08:27:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C3466B00A5; Wed, 23 Apr 2025 08:27:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ECD086B00A6; Wed, 23 Apr 2025 08:27:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id CD7D16B00A4 for ; Wed, 23 Apr 2025 08:27:03 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B2D8CC01EC for ; Wed, 23 Apr 2025 12:27:03 +0000 (UTC) X-FDA: 83365233126.20.53779CC Received: from tor.source.kernel.org (tor.source.kernel.org [172.105.4.254]) by imf08.hostedemail.com (Postfix) with ESMTP id 26559160003 for ; Wed, 23 Apr 2025 12:27:02 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf08.hostedemail.com: domain of cmarinas@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1745411222; a=rsa-sha256; cv=none; b=4flfgdwvybSUU28Idlkdw6Q/qSj/wp2rgPOGLRl+1xSKi/zI1U9qpyZx0z3xi9I5L9DGCv KKec/Sz405kMkYd1HciEz/bq/CIYXIwSmLebLwaw0hg5dNYZ4notGkbdrdRaialRzykBUt LOiNd+ZFCV7dSZUX4c1d5pZfMbE9fGo= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=arm.com (policy=none); spf=pass (imf08.hostedemail.com: domain of cmarinas@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=cmarinas@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1745411222; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jK4eLOZAO+NeR0mENoE8rBEEAUzeupCz2Uf9vs4VHm4=; b=tuYc1RKYOHNzPWFcYThAFBBD5qDr+vQI/GtfHkpZQCxLtPphmlZa7w3Z4u5e87af0txtKC 3OInZHXBQWMWQIR+QWmNGqC1g8ZvpADkQ3IFjdMH/17m7jOCD4sRV0vxARDFqmhNMFhua4 WatTFJgqTXNqCpdbdCRXVE+2xHLKyIY= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by tor.source.kernel.org (Postfix) with ESMTP id 8F4E8614C6; Wed, 23 Apr 2025 12:26:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 679B1C4CEE2; Wed, 23 Apr 2025 12:26:54 +0000 (UTC) Date: Wed, 23 Apr 2025 13:26:51 +0100 From: Catalin Marinas To: Jason Gunthorpe Cc: Oliver Upton , Ankit Agrawal , Sean Christopherson , Marc Zyngier , "joey.gouly@arm.com" , "suzuki.poulose@arm.com" , "yuzenghui@huawei.com" , "will@kernel.org" , "ryan.roberts@arm.com" , "shahuang@redhat.com" , "lpieralisi@kernel.org" , "david@redhat.com" , Aniket Agashe , Neo Jia , Kirti Wankhede , "Tarun Gupta (SW-GPU)" , Vikram Sethi , Andy Currid , Alistair Popple , John Hubbard , Dan Williams , Zhi Wang , Matt Ochs , Uday Dhoke , Dheeraj Nigam , Krishnakant Jaju , "alex.williamson@redhat.com" , "sebastianene@google.com" , "coltonlewis@google.com" , "kevin.tian@intel.com" , "yi.l.liu@intel.com" , "ardb@kernel.org" , "akpm@linux-foundation.org" , "gshan@redhat.com" , "linux-mm@kvack.org" , "ddutile@redhat.com" , "tabba@google.com" , "qperret@google.com" , "kvmarm@lists.linux.dev" , "linux-kernel@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" Subject: Re: [PATCH v3 1/1] KVM: arm64: Allow cacheable stage 2 mapping using VMA flags Message-ID: References: <20250422135452.GL823903@nvidia.com> <20250422170324.GB1645809@nvidia.com> <20250422233556.GB1648741@nvidia.com> <20250423120243.GD1648741@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250423120243.GD1648741@nvidia.com> X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 26559160003 X-Stat-Signature: w5xwoa9n5ab4391qr6jgmq3q5yhnfhe8 X-Rspam-User: X-HE-Tag: 1745411222-13599 X-HE-Meta: U2FsdGVkX18RTmOyLKkRBpkWvhGgB7PM9Dqesimh91OyzOvJN7KzM9K5MJlGUnftvqYaqpnwMuwthxG+jfr1kyqu320oiOZrzn3lpudMtfsgSAL3QFmIZWQPMb9+aCwxSR4M0mDg12Pt25e3G95yyFXyYuk8L4Eq+67/sPSoj1UJnKOtQrHlHDQBPQSxQ5u65Jpvqi7wgC3+OsrFxarqvhZKbGgEc8XU1/4flZ2geI8oxeWjewIM/yc7/l2Tx9/6JyuDi/vad/JpXW26L9cEH2aA8jRhf416PVSD7mmqGEUZuZ5+n6U+3U2+YiMs3PchmdKwq6le/WJGY8tnFTqSu9pEKN61jNa9yj0G9n69uVt0cJ1SuuSyN/1EU5N5OePAke+zLa5wwnDZOdFlHdHcE/tg08yV/rkw3zBAjOqqVsuwfFf3IXEbi1siC6soFmjUXdeB9HOUwP6KKeKTOiGLyYQ4a5f5MsVziaVkssB/wu5ZT/NYuYlYgB1iT7FCrUsRxS6gbaJmXlZTD9zQDPk6ItAiI1Z1Zcz4Ptm+MVONbP9kncjUnV8dNFhb0Q9AEZ1c4w/EJAih5zPLgcCEWfEdc0hp6EScFwKV47vf95D/jZ7nD84nuLDp70CFadydgLRdGRHQkGwxX94CKrmHY9btBAQ3iTfo1RwGf9eU8XcoyXp7fqavOI4CcfRYo6NRiD6XTV2ZerPD4c9ZcR3lEUuCIQrofmJDIS9BfUYFak6rZBwm7H3PyrLv7NC4QtDp08rvAuIoxi275j62PwjPTi+rAF+NA0qn1wvvRVBNYndWx+pKndg08cou8Atm3swyiW1GEeGnhjaI1BfJ6UM5zmcSG3mUaC1nKbkYUyUKc1+r5+RW1gzrGjQShV93r8l6pUqinFci3Nbt3OUdQukN50EENxuDMSPsVRY10sbxwZtwpRNO0XnoLiZYzAb07M3/MJ6lSRqVcNcDEj82kXludGA WSteAJgz /bCVsP/EryremcUB7fWmIplT5sitlmAE38POwVdyGoec/tc93wObw9jeNuBJAae+lCKW8CBeOdyET3gVt9eKKdzaOfEIgSdir83SXdhyhRqHv8HHD6xib/LCSujjYwmOfB4spGlHzrDNnshhsnPVVkk15rer3keUC7L7JX2OFCxYUoLDRdHYTUG9FfJ16YCU5bLJcemrwIlhvzmxYxZHmWCtmHhqTfIHBku+c9KjBHzIuggVnswnPmBKEgtlYhg3IQT4hAdRk4xWUErjhdyb1ovFOnuoa71wk+KYQvgLbRqH02oAdktq2zuK9Ko6FttnoU+tdfgYUcvQRRpSe6AyXlgGrpoo3ZTma7wiqGpx+zM+BNxX4OHS5SeBTwGkRrHmMc5x65cDWjLf+mg8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Apr 23, 2025 at 09:02:43AM -0300, Jason Gunthorpe wrote: > On Wed, Apr 23, 2025 at 11:45:04AM +0100, Catalin Marinas wrote: > > On Tue, Apr 22, 2025 at 08:35:56PM -0300, Jason Gunthorpe wrote: > > > On Tue, Apr 22, 2025 at 02:28:18PM -0700, Oliver Upton wrote: > > > > Wait, so userspace simultaneously doesn't know the cacheability at host > > > > stage-1 but *does* for stage-2? > > > > > > No, it doesn't know either. The point is the VMM doesn't care about > > > any of this. It just wants to connect KVM to VFIO and have the kernel > > > internally negotiate the details of how that works. > > > > > > There is zero value in the VMM being aware that KVM/VFIO is using > > > cachable or non-cachable mappings because it will never touch this > > > memory anyhow, and arguably it would be happier if it wasn't even in a > > > VMA in the first place. > > > > I think this was discussed at some point in the past - what about > > ignoring the VMA altogether and using an fd-based approach? Keep the > > current VFIO+vma ABI non-cacheable and introduce a new one for cacheable > > I/O mappings, e.g. KVM_MEM_IOFD. Is there a way for VFIO to communicate > > the attributes to KVM on the type of mapping in the absence of a VMA > > (e.g. via the inode)? If not: > > I hope to see that someday. The CC people are working in that > direction, but we are still far from getting enough agreement from all > the stakeholders on how that will work. > > > Stage 2 could default to non-cacheable and we can add a > > KVM_MEMORY_ATTRIBUTE_IO_WB as an option to > > I don't think we should ever allow KVM to create a non-cachable > mapping of an otherwise cachable object?? Ah, good point. So we want a strict cacheable mapping even if there is no user/VMM mapping alias. Is it because KVM cannot do cache maintenance on the PFNMAP or because the device does not support other memory types? Both valid reasons though. In this case KVM still needs to know the properties of the device. Not sure how it could best do this without a vma. -- Catalin