From: Pranjal Shrivastava <praan@google.com>
To: David Laight <david.laight.linux@gmail.com>
Cc: "David Hu" <xuehaohu@google.com>,
"Sumit Semwal" <sumit.semwal@linaro.org>,
"Christian König" <christian.koenig@amd.com>,
"Jason Gunthorpe" <jgg@ziepe.ca>,
"Nicolin Chen" <nicolinc@nvidia.com>,
"Leon Romanovsky" <leon@kernel.org>,
"Kevin Tian" <kevin.tian@intel.com>,
"Ankit Agrawal" <ankita@nvidia.com>,
"Alex Williamson" <alex@shazbot.org>,
linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org,
linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org,
iommu@lists.linux.dev, jmoroni@google.com, kpberry@google.com,
chriscli@google.com, sashiko-bot@kernel.org,
stable@vger.kernel.org
Subject: Re: [PATCH v2] dma-buf: Split sgl into page-aligned 2G chunks
Date: Tue, 23 Jun 2026 20:55:32 +0000 [thread overview]
Message-ID: <ajryxMaT5evDUxaq@google.com> (raw)
In-Reply-To: <20260623094446.4a8fc2ed@pumpkin>
On Tue, Jun 23, 2026 at 09:44:46AM +0100, David Laight wrote:
Hi David,
> On Tue, 23 Jun 2026 01:54:59 +0000
> David Hu <xuehaohu@google.com> wrote:
>
> > Currently, `fill_sg_entry()` splits the scatterlist using `UINT_MAX`.
> > This creates a non-page-aligned DMA length (`0xFFFFFFFF`) for the
> > first entry, resulting in non-page-aligned DMA addresses for all
> > subsequent entries.
>
> There is a separate issue of whether this code is even needed at all.
> Where can transfers over 2G (never mind 4G) actually come from.
>
> The read, write and similar system calls limit transfers to INT_MAX
> (even on 64bit) and a lot of driver code will need fixing it longer
> lengths are allowed though.
> io_uring better enforce the same limits.
> So the transfers can come directly from userspace.
>
> Not only that but you also need a single physically contiguous buffer.
> Good luck allocating that!
>
> Now maybe there are some peer-to-peer places where the large buffer
> is device memory, but they will be unusual and probably need
> special treatment anyway.
>
I agree that traditional VFS read/write face the MAX_RW_COUNT limit
(~2GB), and io_uring has its limits, but I'm a little confused by the
push to enforce these limits here in the SGL code?
File I/O seems to be only one side of the picture. In my view, this fix
is necessary and certainly has a use-case:
For example, the RDMA subsystem has the capability to import dmabufs [1],
which gives rise to use cases for dmabuf beyond standard file ops
(via VFS/io_uring).
In these scenarios, GPU HBM can be exported as dmabufs. With recent GPUs,
HBM capacity can be in the order of hundreds of GBs [2]. RDMA can employ
infrastructure like the vfio-dmabuf-exporter [3] or similar dmabuf
exporters to frequently move huge blocks of data via P2PDMA.
If we restrict incoming dmabuf transfers to fit within VFS-centric
limits (2GB), we impose unnecessary overhead on the RDMA stack, forcing
it to manage a significantly higher number of memory registrations. By
cleanly splitting these massive contiguous device buffers into
page-aligned SGL entries, we directly improve the efficiency of P2P
transfers and memory registration.
Since this change doesn't seem to have a negative impact on standard file
I/O or break existing VFS constraints, I'm curious why we shouldn't
support splitting these >4GB P2P transfers? Am I missing something?
Thanks,
Praan
[1] https://elixir.bootlin.com/linux/v7.1.1/source/drivers/infiniband/core/umem_dmabuf.c#L174
[2] https://nvdam.widen.net/s/fdvdqvfvj2/hopper-h200-nvl-product-brief (Table 2-2)
[3] https://elixir.bootlin.com/linux/v7.1.1/source/drivers/vfio/pci/vfio_pci_dmabuf.c#L297
next prev parent reply other threads:[~2026-06-23 20:56 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-06-21 22:21 [PATCH] dma-buf: Split sgl by largest page-aligned chunk David Hu
2026-06-21 22:34 ` sashiko-bot
2026-06-22 8:13 ` David Laight
2026-06-22 21:26 ` David Hu
2026-06-23 8:25 ` David Laight
2026-06-23 21:03 ` David Hu
2026-06-23 1:54 ` [PATCH v2] dma-buf: Split sgl into page-aligned 2G chunks David Hu
2026-06-23 2:08 ` sashiko-bot
2026-06-23 8:44 ` David Laight
2026-06-23 20:55 ` Pranjal Shrivastava [this message]
2026-06-23 22:53 ` David Laight
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ajryxMaT5evDUxaq@google.com \
--to=praan@google.com \
--cc=alex@shazbot.org \
--cc=ankita@nvidia.com \
--cc=chriscli@google.com \
--cc=christian.koenig@amd.com \
--cc=david.laight.linux@gmail.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=iommu@lists.linux.dev \
--cc=jgg@ziepe.ca \
--cc=jmoroni@google.com \
--cc=kevin.tian@intel.com \
--cc=kpberry@google.com \
--cc=leon@kernel.org \
--cc=linaro-mm-sig@lists.linaro.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=nicolinc@nvidia.com \
--cc=sashiko-bot@kernel.org \
--cc=stable@vger.kernel.org \
--cc=sumit.semwal@linaro.org \
--cc=xuehaohu@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.