From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-dl1-f43.google.com (mail-dl1-f43.google.com [74.125.82.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0920C3644CB for ; Thu, 11 Jun 2026 14:40:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.43 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781188829; cv=none; b=ZSnPNGogcX28n4fHogfIYh+O9Crw5CXIwQNoFFVIap/PmU+I+i/CQl7VoGD+w7dWeY0MK7IVJYtW5pW6+iTOmnpFq2HNpJmgSuQUCD0YLoBhwX3BYxa0RE/ZgDfGnaF/B9ujZP9gSWkYFWTKZwccMFWrroxsdMV/njOT4BDkyB8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781188829; c=relaxed/simple; bh=YEvXA7hxzCZPI3z9TCMIdWjx2UA7u7fi73iaafd2xIA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=pUP3C8aAOc2rCFewMsrc9qBxHkiz+rC3ityziwRcUmfTm+01dXH2fRRv1iHXHQkwFDvYjMyzX0/AJjnDS+5iMsDGpHL+aZ39c4seHRqDtfESq4LNzyI8+NEb9tSLbTJ7qY55fCSZtNVxmrs2euIgmsJ0Jka2Xjlx12953tkP/9A= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Rp6TueOS; arc=none smtp.client-ip=74.125.82.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Rp6TueOS" Received: by mail-dl1-f43.google.com with SMTP id a92af1059eb24-1336742714fso10973c88.1 for ; Thu, 11 Jun 2026 07:40:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1781188826; x=1781793626; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=jc1obbf7us2qhujC6W8Kt/1pNv4AnXWLRx9wD4plXQM=; b=Rp6TueOSrB8gba3wp+LlGVP3CuxtZyLhD4+Kd+q09587qsrHBl3yH2ulZNBdvHAi6C zx1lSqBi8XznqVKar9GCTQ8WnmBW6OIvpK7q4IlhEkLCHFRoXA3/ytY6/AYtBi+fs23R IMuSKfYgR131OZ+f/FhHWteMk9u3517MY8rsGecA9qxdnOa4a+3YabDEjQWtxnjgkNUL Nabt8BqCQg/se1aY4NPNO/VWyh4xhtmc0f+e9HjlzQROyN6B+mPniHKxK2rj9L8/jXpb tx3TTMKwB6Z5PuBjPSG+xiezyjPStEwBY58kwHBKM8eivB6/WTMLXEkV4p9ntMI5SJUk bawg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781188826; x=1781793626; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jc1obbf7us2qhujC6W8Kt/1pNv4AnXWLRx9wD4plXQM=; b=rqWmURjtuJHYVF74+TlQ7VDwi9baNfFnzVj2gApMBtnpr5Uco7AYiZgHJqn6NPNPut eOJsOCHeFyNeBehY6jgklQthOm6m3MGFH4KjGYDV/rykzBCAJ6SZ6EicaFL/i7ZFma4q DlCxcr5y2t9v7C/sdLxSeA2qrFDoAgeHvCGRIHzYVcS2HBLoCwmFkJ63UcBWXw50iTHc HTRmW/GYtH+t6mO0my3kFs+eNdzrl4W97ulbc1taddO2+4ZbkAIHhKb2GY9VILSjC3ai 9aYaLJ2t7tl8cJbS6tEByTXZCkVEWxAq+rl5gcZKkogyylzzHFj3MwwLcDSZ9oZ7jiKE fSjQ== X-Forwarded-Encrypted: i=1; AFNElJ8sR/czjJmqOQgvtcgliLoB6H/etGjFNK3vgNgZHVFsYpQl2RHl/UtXTaIAHtIuG7MYK7WuD/7dYhQAao8=@vger.kernel.org X-Gm-Message-State: AOJu0YwrVzw9AI9aaVbHnZljeQU+UYJpm5xAhVNUbybYPEGF+0WrDWwp 7aJhrUljttVWcf+1rXevUhl9hRBUCK/udD6LSExeNL3L11o8x4PirO5hT6wyiDVLIg== X-Gm-Gg: Acq92OHO7pJW0A+lhKOa8oKfgeJFCFrf27Vybj0CA5bk9mfkrRgfq+iMZ36Ae2OlpM2 Zolkwzj+Y5uTBI+SiKJGbmZMiTRG0TJhrSE9PdgQKOjxDCLthIHSf5FFYl3LnbhIosPVEHy6kXn Ikut4o+2huwrOL/Cmf+TStd5NJeoQzx4f8PU8Q7FW5ObtRYeqnvgL9hKnrAZ2qPxReL3EWaTc3/ Q7aURYU5PHLgX45XYQEQ1ubEGWMOmGDUuPbXBskhrqH+zJozxYiGVXrqa6HIEOCNXG/i9nLLJeo LpZbqMFZeoMhNNBYfwBqUS3OpQDgagHxg7y7Q88hzEmoIA4MeS6CbSYxTrZL0XzqhoCPwE6Rbci 5fXLKUWKariq0xzKDwB6PjsvsFVKLjjBO4MWAzpnRvH91cKkSQVS5uQtb+zJIE2eyXQ8e9n4b6i oWYYO0ZMTGZGaVtOqQARRMGdjAdSj9svYeQsUc1YLlunSetthrAlBPLWqJa1N31pw0k3Z7vXy0K NrYib6bLw== X-Received: by 2002:a05:7022:513:b0:138:888:32e0 with SMTP id a92af1059eb24-13841ec18e4mr169957c88.32.1781188824336; Thu, 11 Jun 2026 07:40:24 -0700 (PDT) Received: from google.com (199.255.142.34.bc.googleusercontent.com. [34.142.255.199]) by smtp.gmail.com with ESMTPSA id 5a478bee46e88-30806c45dcasm2284632eec.7.2026.06.11.07.40.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jun 2026 07:40:23 -0700 (PDT) Date: Thu, 11 Jun 2026 14:40:17 +0000 From: Pranjal Shrivastava To: Jason Gunthorpe Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Bjorn Helgaas , Logan Gunthorpe , Alex Williamson , Kevin Tian , Ankit Agrawal , Matt Evans , Vivek Kasireddy , Leon Romanovsky , Shivaji Kant , Samiullah Khawaja Subject: Re: [RFC PATCH 0/5] vfio/pci: Support ZONE_DEVICE-backed P2P Registration Message-ID: References: <20260610151853.3608948-1-praan@google.com> <20260610162848.GO2764304@ziepe.ca> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260610162848.GO2764304@ziepe.ca> On Wed, Jun 10, 2026 at 01:28:48PM -0300, Jason Gunthorpe wrote: > On Wed, Jun 10, 2026 at 03:18:48PM +0000, Pranjal Shrivastava wrote: > > > Users utilize the standard sysfs p2pmem/allocate interface for managing > > memory slices once a BAR is registered. > > I'm shocked someone wants to use API, what are you expecting to do > with it?? Our primary use-case is PCIe BAR (DDR / HBM) -> NFS via P2PDMA while the PCIe device is managed by a user-space driver based on vfio-pci. While kernel drivers (e.g.drm) can register BARs with ZONE_DEVICE natively to enable this, VFIO currently lacks an equivalent mechanism. > > > An alternative implementation has been explored which integrates with the > > ongoing VFIO DMABUF-mmap refactor [1]. In that approach, rather than > > registering a BAR as a system-wide P2P provider, VFIO optionally > > allocates ZONE_DEVICE pages only for specifically exported DMABUFs via a > > new VFIO_DMA_BUF_FLAG_ALLOC_STRUCT_PAGES flag. > > That's probably more sensible but you can't have a DMABUF mmap > actually install non-special memory. The native vfio mmap still can, > but not mmap on the dmabuf fd. That's still workable, just keep in > mind. Ack. I guess, we could have a separate mmap path in case of BARs that are struct page backed which doesn't go through the dmabuf exporter. That said, we don't mind moving away from the old API if VFIO mmap can support this, we don't mind open-coding with devmem_remap_pages. Effectively, we just need a struct page-backed BAR exporter, preferably, via VFIO. > > What do you even intend to do with this? With the new work to tie > dmabuf directly into io_uring I really wonder if this is worth doing > for VFIO? I agree that Pavel's io_uring series is great for the Block Layer [1], However, it doesn't help with NFS + O_DIRECT which still relies on struct page for DMA mapping. We have a series to modernize NFS to support P2PDMA [2][3] and while I understand native dmabuf read/writes are coming into play, their integration into NFS is a distant future (which probably we'll have a stab at once Pavel's series settles). Thus, we'd like to provide a standard, VFS-compatible P2P path via VFIO we're willing to help maintain this if needed. Thanks, Praan [1] https://lore.kernel.org/all/cover.1777475843.git.asml.silence@gmail.com/ [2] https://lore.kernel.org/all/20260603053033.3300318-1-praan@google.com/ [3] [RFC] https://lore.kernel.org/all/20260401194501.2269200-1-praan@google.com/