From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qk1-f174.google.com (mail-qk1-f174.google.com [209.85.222.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 386EB301719 for ; Mon, 6 Apr 2026 22:05:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.174 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775513125; cv=none; b=AZBsDi5RaiY91rZqZ9d+gIQrLkPeOyG6YjB6Mjr5GcXp/1oi7YxfKQOBHL0BDfP6u3Kp1kp2586Nrg4FsCfYugprtkImFZVdgnbjsIwvXl/5aR6L8Va8rMH/JTl75u3he2wGJuRxZOx5LIbtMZ4lp8tDhtoJUDNtTI6ptDAgfGU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775513125; c=relaxed/simple; bh=oK9QofoXFukYbZ1c/cswd1bebDb7z9qu2bCtB6Eab70=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=BRsZJUGHCYVjJAOVrjXpPliAkPRL2z/B/eYICjV6gn8pRFwXcu25HVRHJLKUhODNu5Doz9dzwjpdys07JHq0krOPTxNKEMi6YnudRhHd1990VJiLMIHM0b9cmhiR87G7unFLuvafUz3/DcfSJFl2rNlQ+wSHWYGpg7loFeXaseQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca; spf=pass smtp.mailfrom=ziepe.ca; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b=RVmy1JPg; arc=none smtp.client-ip=209.85.222.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ziepe.ca Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="RVmy1JPg" Received: by mail-qk1-f174.google.com with SMTP id af79cd13be357-8d736211595so123070685a.0 for ; Mon, 06 Apr 2026 15:05:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1775513123; x=1776117923; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=EGZao/IIzct9FspuaHKhJftGREYwsc0pd1AtS7x/cpc=; b=RVmy1JPga9OP1fAYlBdCYxO5p1+NGTZBvNC1KuBLpT466N7Isgs+iKH5WT+SI/ROzv yoHVWTtoIRc5V8m6KHP9CHMbAWIv8a8mW5oiPlKd84adiK6/r+zv22SOEoHy3lafQN9e Ppx6aw2Tizpatlz00J0wp50QbbpAMXE5wJ1/6wlJPNKKRO7AnnxaBi+Ai7IGup59P7zI nHVBfP2wgxCfxbg/eANCqyPufMfBzyPEbilOwLOHXKCxDD195xeJB//VTM/Is34+sddx /uRwDjVsWwAAm4oB4gT5XJ5Tl8T8mTwvtRrPo0NbGANms9By8Bxw2O9Si3DM6tHyHOWy lowQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775513123; x=1776117923; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EGZao/IIzct9FspuaHKhJftGREYwsc0pd1AtS7x/cpc=; b=sHcwlE7JtxK6uk70mGk34/uz12MMg3PD+beewZ3BiaeRXO1S/Dirk1zmjpJlWMJrzy CoHpYtcFByr/MKXEiqULQeufrXWtRcjrvKcKEAPz0LcsbzhESGlg92tkrMb+RzQmJ/P4 xokiQJ5e9VFQyilZaucZw70hVSW1Eo2nGALvfINpy40efoCfznmqusd2JL0qiHiyH9Wx kTTi+3KO9qtYdXh4Z5wemrdooamvg58jTy9BdoT1M0rguTHrlepV2vK+Pu9s48fYpm44 F7RkJM1rajq33rNfmcHouQYtUX0HAzUCMrRunbbp1Zk50ud4w6GnwI4tDrs73c4Ks692 sN2A== X-Forwarded-Encrypted: i=1; AJvYcCXrPcKS7bvvIUDdMgoqtAIGfc0SLyg53JN3xFNBr8KvwAdCLs8MepfT3jSR0d+TewHgUpRgefb8Dv0rux4=@vger.kernel.org X-Gm-Message-State: AOJu0YzeewVoZmMGJQ0XLuWzDeQQ4E2ibbxn+QXtFpERtn4jvwecygGF HzkdECK2bEHXTBFnwiPv7C+DMMUUQYkCeoAkoclu9NpCLrOVOb4pOSJ3m+G5cWTC9As= X-Gm-Gg: AeBDietrhqjzxS7mu9xiFIQep3jnuZrKyhWQZiN+bQobQZ+GPKtCaRIpJ6YgtIivOYr XKMjkgICGH7HUSIF6KCJQVISaHrK7DCOZ8iw/X7pEOI9RkHNgzVMIrwa/12cB70e+QCz4ebIast cmJamCOSJyofH4LkJmwXWOrPT6dWKze+eSYEOxCv9MtnKjZ5oE0UT68X6e3zm2kodqE9KwltIy7 wupDh8jAXvFa78xzqzyFkm4l0ZJcBZvEtfsaZ8g4Y5Tbl78lRAObJfm5epLmtXaaH0FAFLVwI9a nI5oyo44H6LGIeM21ykaILd4hdtO3fkc8NjsCT/78TW08wpgVFVKQxcMQYIlUM7IqmAxRKSlurh DexWO88qEFs4uovxexNCE7AhjhyPXmAi+QP53qg8rNTh0X7AuAbH61wnJRvylGfwS2/zcG9VaY9 vIvlKWMKRPzE+q5cTuiROHUL8LrxmtKX5WqxnrtRu4aXsWePChg3wyADHZR50mm2UM21smAqRrH i8EODbN X-Received: by 2002:a05:620a:294d:b0:8cf:bfcf:eb8f with SMTP id af79cd13be357-8d41bcd15d3mr1922226485a.65.1775513123076; Mon, 06 Apr 2026 15:05:23 -0700 (PDT) Received: from ziepe.ca (mctnnbsa70w-159-2-73-22.dhcp-dynamic.fibreop.nb.bellaliant.net. [159.2.73.22]) by smtp.gmail.com with ESMTPSA id af79cd13be357-8d2a5c5cb5csm1165614585a.15.2026.04.06.15.05.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Apr 2026 15:05:22 -0700 (PDT) Received: from jgg by wakko with local (Exim 4.97) (envelope-from ) id 1w9s4P-0000000E4mE-3CKd; Mon, 06 Apr 2026 19:05:21 -0300 Date: Mon, 6 Apr 2026 19:05:21 -0300 From: Jason Gunthorpe To: Gregory Price Cc: Dan Williams , mhonap@nvidia.com, alwilliamson@nvidia.com, jonathan.cameron@huawei.com, dave.jiang@intel.com, alejandro.lucero-palau@amd.com, dave@stgolabs.net, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dmatlack@google.com, shuah@kernel.org, yishaih@nvidia.com, skolothumtho@nvidia.com, kevin.tian@intel.com, ankita@nvidia.com, vsethi@nvidia.com, cjia@nvidia.com, targupta@nvidia.com, zhiw@nvidia.com, kjaju@nvidia.com, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-cxl@vger.kernel.org, kvm@vger.kernel.org Subject: Re: [PATCH v2 16/20] vfio/cxl: Register regions with VFIO layer Message-ID: <20260406220521.GG2551565@ziepe.ca> References: <20260401143917.108413-1-mhonap@nvidia.com> <20260401143917.108413-17-mhonap@nvidia.com> <69d0169115c06_1b0cc6100a5@dwillia2-mobl4.notmuch> <20260404185330.GD2551565@ziepe.ca> <69d1685538f45_556821002f@djbw-dev.notmuch> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Mon, Apr 06, 2026 at 05:22:06PM -0400, Gregory Price wrote: > On Sat, Apr 04, 2026 at 12:36:53PM -0700, Dan Williams wrote: > > Jason Gunthorpe wrote: > > > > > > The entire thing needs to be mmapable in a cache coherent way since > > > that is what the HW semantic is. If you try to do something else you > > > will break KVM support since it follows the VMA. > > > > Then I assume it matters that memremap() sometimes silently falls back > > to the direct map. The "VFIO owns" expectation needs to guard against > > some helpful platform firmware mapping accelerator memory as System RAM. > > > > At a minimum having VFIO fail to map in that case helps with the > > argument I have been making that "no, EFI_CONVENTIONAL_MEMORY type + > > EFI_SPECIFIC_PURPOSE flag" is not suitable for accelerators with private > > CXL memory. Those want to be enforcing "EFI_RESERVED". > > Agree - in fact I would argue any potential user of private nodes should > have some kind of splat saying the memory should be marked reserved in > the first place, otherwise it's a firmware bug. I would expect this to happen via the request_resource mechanism, if MM is using the CXL range it should be locked in the resource tree and vfio should request it and then fail during the initial startup phases. > I need to read up a little bit on this area, but i don't see the "needs > to be mmap'able in a cache coherent way" to be an argument for one > particular method or another (hotplug, private node, memremap, etc). It is speaking to what VFIO has to do. It is the exclusive owner of the physical range, it does not have struct pages, it must be cachable for VFIO and KVM to work - not alot of choices here. Turn the phys_addr_t's into large cachable special PTEs inside a VMA. Jason