From: Pranjal Shrivastava <praan@google.com>
To: David Laight <david.laight.linux@gmail.com>
Cc: "David Hu" <xuehaohu@google.com>,
"Sumit Semwal" <sumit.semwal@linaro.org>,
"Christian König" <christian.koenig@amd.com>,
"Jason Gunthorpe" <jgg@ziepe.ca>,
"Nicolin Chen" <nicolinc@nvidia.com>,
"Leon Romanovsky" <leon@kernel.org>,
"Kevin Tian" <kevin.tian@intel.com>,
"Ankit Agrawal" <ankita@nvidia.com>,
"Alex Williamson" <alex@shazbot.org>,
linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org,
linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org,
iommu@lists.linux.dev, jmoroni@google.com, kpberry@google.com,
chriscli@google.com, sashiko-bot@kernel.org,
stable@vger.kernel.org
Subject: Re: [PATCH v2] dma-buf: Split sgl into page-aligned 2G chunks
Date: Tue, 23 Jun 2026 20:55:32 +0000 [thread overview]
Message-ID: <ajryxMaT5evDUxaq@google.com> (raw)
In-Reply-To: <20260623094446.4a8fc2ed@pumpkin>
On Tue, Jun 23, 2026 at 09:44:46AM +0100, David Laight wrote:
Hi David,
> On Tue, 23 Jun 2026 01:54:59 +0000
> David Hu <xuehaohu@google.com> wrote:
>
> > Currently, `fill_sg_entry()` splits the scatterlist using `UINT_MAX`.
> > This creates a non-page-aligned DMA length (`0xFFFFFFFF`) for the
> > first entry, resulting in non-page-aligned DMA addresses for all
> > subsequent entries.
>
> There is a separate issue of whether this code is even needed at all.
> Where can transfers over 2G (never mind 4G) actually come from.
>
> The read, write and similar system calls limit transfers to INT_MAX
> (even on 64bit) and a lot of driver code will need fixing it longer
> lengths are allowed though.
> io_uring better enforce the same limits.
> So the transfers can come directly from userspace.
>
> Not only that but you also need a single physically contiguous buffer.
> Good luck allocating that!
>
> Now maybe there are some peer-to-peer places where the large buffer
> is device memory, but they will be unusual and probably need
> special treatment anyway.
>
I agree that traditional VFS read/write face the MAX_RW_COUNT limit
(~2GB), and io_uring has its limits, but I'm a little confused by the
push to enforce these limits here in the SGL code?
File I/O seems to be only one side of the picture. In my view, this fix
is necessary and certainly has a use-case:
For example, the RDMA subsystem has the capability to import dmabufs [1],
which gives rise to use cases for dmabuf beyond standard file ops
(via VFS/io_uring).
In these scenarios, GPU HBM can be exported as dmabufs. With recent GPUs,
HBM capacity can be in the order of hundreds of GBs [2]. RDMA can employ
infrastructure like the vfio-dmabuf-exporter [3] or similar dmabuf
exporters to frequently move huge blocks of data via P2PDMA.
If we restrict incoming dmabuf transfers to fit within VFS-centric
limits (2GB), we impose unnecessary overhead on the RDMA stack, forcing
it to manage a significantly higher number of memory registrations. By
cleanly splitting these massive contiguous device buffers into
page-aligned SGL entries, we directly improve the efficiency of P2P
transfers and memory registration.
Since this change doesn't seem to have a negative impact on standard file
I/O or break existing VFS constraints, I'm curious why we shouldn't
support splitting these >4GB P2P transfers? Am I missing something?
Thanks,
Praan
[1] https://elixir.bootlin.com/linux/v7.1.1/source/drivers/infiniband/core/umem_dmabuf.c#L174
[2] https://nvdam.widen.net/s/fdvdqvfvj2/hopper-h200-nvl-product-brief (Table 2-2)
[3] https://elixir.bootlin.com/linux/v7.1.1/source/drivers/vfio/pci/vfio_pci_dmabuf.c#L297
prev parent reply other threads:[~2026-06-23 20:56 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-06-21 22:21 [PATCH] dma-buf: Split sgl by largest page-aligned chunk David Hu
2026-06-22 8:13 ` David Laight
2026-06-22 21:26 ` David Hu
2026-06-23 8:25 ` David Laight
2026-06-23 21:03 ` David Hu
2026-06-23 1:54 ` [PATCH v2] dma-buf: Split sgl into page-aligned 2G chunks David Hu
2026-06-23 8:44 ` David Laight
2026-06-23 20:55 ` Pranjal Shrivastava [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ajryxMaT5evDUxaq@google.com \
--to=praan@google.com \
--cc=alex@shazbot.org \
--cc=ankita@nvidia.com \
--cc=chriscli@google.com \
--cc=christian.koenig@amd.com \
--cc=david.laight.linux@gmail.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=iommu@lists.linux.dev \
--cc=jgg@ziepe.ca \
--cc=jmoroni@google.com \
--cc=kevin.tian@intel.com \
--cc=kpberry@google.com \
--cc=leon@kernel.org \
--cc=linaro-mm-sig@lists.linaro.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=nicolinc@nvidia.com \
--cc=sashiko-bot@kernel.org \
--cc=stable@vger.kernel.org \
--cc=sumit.semwal@linaro.org \
--cc=xuehaohu@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox